How to make a search engine by Java ?

  • Context: Java 
  • Thread starter Thread starter Todee
  • Start date Start date
  • Tags Tags
    Engine Java Search
Join the discussion
Ask a follow-up here, or get your own question answered by working scientists, mathematicians and engineers — people, not an autocomplete.
Real named experts · corrections over time · the nuance an AI answer skips
2 replies · 5K views
Todee
Messages
6
Reaction score
0
want some sites to teach me how to develop a simple web-based search engine that demonstrates the main features of a search engine (web crawling, indexing and ranking) and the interaction between them.
Using Java :confused:
 
Physics news on Phys.org
I'm going to suggest that you take the CS101 course over at Udacity because it will teach you exactly what you want to know, how to build a search engine. They teach it using Python, but the code is not complex and you could easily adapt it to Java.

One thing to be aware of. At the end of the course, you don't so much have a working search engine, as you have all the components that are required. The reason they don't give you a working program is because a search engine involves a fair amount of web etiquette - meaning you have the power to hit web servers with thousands upon thousands of requests, and before you unleash yours onto the world, you want to make sure that you are acting in a courteous manner. Particularly in the testing phase.

The actual code for web-crawling and indexing involves making a request to some seed page, getting the HTML back, parsing the HTML for links, and then recursively following those links and parsing the new HTML for more links, until you run out of room in your index, or the links stop.

Ranking can be done in many ways. Google's algorithm is fairly well documented around the web. It basically says, for any page, the rank is a measure of how many other pages link to this page, and the rank of those other pages. A high ranked page linking to your page, increases your rank by a larger factor than a low ranked page linking to your page.