WordGarden.io
Back to search

crawler

B2
noun

Pronunciation

UK

  • /krˈɔːlə/

US

  • /ˈkrɔlər/

Description

A "crawler" can be many things. Most literally, it is a person or animal that moves on its belly or close to the ground—think of a baby learning to move, or an insect like a caterpillar. In the digital world, however, a crawler (also known as a spider) is a program that automatically visits web pages and follows links, typically to index websites for search engines like Google. These web crawlers help search engines find and organize information online. You might also hear "crawler" used for a slow, tracked machine built to move over rough ground, or for the scrolling text bar on TV or websites (a news crawler).

Think of a baby crawling across the floor—slow but determined. Then, imagine a tiny digital robot "crawling" through the internet to collect data. Both are excellent ways to remember how a crawler operates!

Examples

  1. 1

    Baby development

    By nine months, Leo was already a fast crawler.

  2. 2

    Daycare groups

    At the daycare, the crawlers stay in a different room from the walkers.

  3. 3

    Search engines

    The site's `robots.txt` file tells search-engine crawlers which pages to avoid.

    • Domain

      in tech, a crawler

      a program that automatically visits websites

Forms and spellings

1 form open this card.

Main spelling

  • crawlernoun