Online search engine development is a technology called a web spider or network robot. The term "robot" is very popular in the programmer. Computer Robot refers to a software program that a certain task can be performed in a manner that cannot be implemented in a manner. Since the "robot" program dedicated to retrieving information is climbed in the network, the "robot" program of the search engine is called the "spider" program. The first "robot" program for monitoring the size of the Internet development is the World Wide Web Wanderer developed by Matthew Gray. Just starting it only used to count the number of servers on the Internet, and later developed to retrieve the site domain name. Compared with Wanderer, Martin Koster created AliWeb in October 1993, which is an ARCHIE HTTP version. ALIWEB does not use the Robot program, but rely on the website to submit information to establish your own link index, similar to what we are familiar with now. Yahoo. With the rapid development of the Internet, it is more difficult to retrieve all new web pages. Therefore, on the basis of Matthew Gray Wanderer, some programmers have improved traditional "spider" procedures. It is idea that since all the webpages may have links to other websites, then start from the link to track a website, it is possible to retrieve the entire Internet. By the end of 1993, some search engines based on this principle have emerged, in which JumpStation, The World Wide Web Worm and Repository-Based Software Engineering (RBSE) Spider are the most prestigious. However, JumpStation and WWW WORMs are only in order to find the search results in the database in the database, so there is no information correlation. RBSE is the first engine that introduces a key string matching degree concept in the search results. The earliest search engine appeared in July 1994. At that time, Michael Mauldin entered the John Leavitt's spider program to its index program, creating the Lycos that everyone is now familiar. In April, Stanford, two doctoral students, David Filo and American Gerry Yang, found a super directory index Yahoo, and successfully enabled the concept of search engines into the heart. Since then, search engines have entered a period of high-speed development. At present, there are hundreds of surname engine on the Internet have reached hundreds, and the amount of information retrieved is also in the same day before. For example, the recent winds of Google, the webpage stored in its database has reached 3 billion! With the sharp expansion of the Internet, a search engine has not been able to adapt to the current market situation, so it is now starting a division of labor, and there is a professional search engine technology and search database services. Business. It is not directly to the user's search engine, but provides a full text page search service to other search engines including Overture (original goto), Looksmart, MSN, Hotbot, etc.. Domestic Baidu also belongs to this class (note), Sohu and the new are its technology. So in this sense, they are search engines search engine. From "China Software"
