Crawler
The part of a search engine that goes through the internet page by page and follows the links found in these pages. Also called spider or robot, the crawler creates a copy of the found web pages and stores them in the index of the search engine.
Back
Contact us