3 results about "Web crawler" patented technology

In a data mining system, data is gathered into a data store using, e.g., a Web crawler. The data is classified into entities. Data miners use rules to process the entities and append respective keys to the entities representing characteristics of the entities as derived from rules embodied in the miners. With these keys, characteristics of entities as defined by disparate expert authors of the data miners are identified for use in responding to complex data requests from customers.

The present invention belongs to the technical field of the Internet, and in particular relates to a web crawler method and system based on improved PageRank. The method includes: (1) crawling web pages; (2) obtaining web page relationships; (3) obtaining relationship matrices; 4) Obtain the initial probability matrix; (5) PageRank calculation; after obtaining the relationship matrix, the initial probability matrix and the damping coefficient, calculate the PR value of the web page, and iteratively calculate until the probability matrix converges and terminate the iteration; by the method provided by the present invention, solve The problem of web page deception in which some pages reuse keywords in web crawlers to improve search rankings. The web crawler system provided by the present invention is easy to use and has easy-to-accept storage, conversion, and calculation forms, and can efficiently and quickly calculate the weight of each web page; at the same time, it includes a relatively complete crawler program, which shows the crawler in real life. Applications, including picture and file download, Baidu Encyclopedia search, video playback, web page relationship visualization and other functions.

The invention discloses a method for rapidly and accurately searching for a target book by a web crawler technology. The process of the method comprises inputting electronic books and establishing an electronic library; classifying and placing the input books into different sub-webpages; inputting keywords of the book which needs to be read; crawling book webpages related to the target book by the web crawler technology; performing analysis on the crawled webpages; outputting filtered books after the analysis; selecting the target book to read. According to the method for rapidly and accurately searching for the target book by the web crawler technology, new electronic books can be input into the electronic library timely and accordingly the richness of the book classes of the books of the electronic library is ensured; the target book can be obtained rapidly and accurately by the web crawler technology; the book classes are not limited to the word class and also comprise the picture class, the video class and the like; the method is suitable for the electronic library and also suitable for an electronic book website and accordingly certain promotion is brought to the development of the electronic books.