---

crawler → CapableOf → gather information	0.68
crawler → CapableOf → visit website	0.67
web crawler (subgroup of crawler) → CapableOf → call spider	0.66
crawler → CapableOf → visit page	0.65
crawler → CapableOf → find page	0.62
search engine → HasA → crawler	0.62
crawler → CapableOf → find link	0.61
crawler → CapableOf → crawl page	0.60
crawler → CapableOf → collect datum	0.59
crawler → ReceivesAction → used by search engines	0.56
crawler → HasProperty → active	0.50
web crawler (subgroup of crawler) → CapableOf → browse the world wide web	0.46
crawler → HasPrerequisite → a starting point	0.41
web crawler (subgroup of crawler) → IsA → program	0.41
crawler → CapableOf → index the content	0.40
crawler → CapableOf → use link	0.38
crawler → CapableOf → read the content	0.38
crawler → ReceivesAction → programmed to visit site	0.38
crawler → CapableOf → read the file	0.36
crawler → CapableOf → crawl content	0.36
crawler → Desires → download page	0.36
crawler → CapableOf → send request	0.36
crawler → CapableOf → access page	0.36
crawler → CapableOf → use a visitedlinksservice	0.36
crawler → CapableOf → write page	0.36
web crawler (subgroup of crawler) → CapableOf → call seed	0.34
web crawler (subgroup of crawler) → CapableOf → start with a list of urls	0.34
crawler → CapableOf → scour the internet	0.33
crawler → ReceivesAction → designed	0.33
crawler → CapableOf → collect tweet	0.29
crawler → ReceivesAction → written in python	0.29
crawler → ReceivesAction → thought as automated web browsers	0.29
crawler → CapableOf → build index	0.29
crawler → CapableOf → assemble datum	0.29
crawler → CapableOf → start to crawl independent parts of the sites	0.29
crawler → HasProperty → useful	0.27
crawler → HasProperty → intelligent	0.25
crawler → CapableOf → played key role	0.25
crawler → CapableOf → tackle the steepest gradients	0.25
crawler → CapableOf → read the text	0.25
crawler → CapableOf → feed on leave	0.25
crawler → CapableOf → understand context	0.25
crawler → CapableOf → settle on twig	0.25
crawler → HasProperty → yellow	0.25
crawler → CapableOf → insert their mouthparts	0.25
crawler → HasProperty → configurable	0.25
crawler → ReceivesAction → expected	0.25
crawler → CapableOf → avoid requesting resource	0.25
crawler → CapableOf → traverse website	0.25
crawler → ReceivesAction → stopped	0.25
crawler → CapableOf → parse page	0.25
crawler → CapableOf → access known urls of a host	0.25
crawler → CapableOf → take linear path	0.25
crawler → CapableOf → appear as black-colored creature	0.25
crawler → HasProperty → strong	0.25
crawler → CapableOf → follow all the links	0.25
crawler → HasA → a boom length	0.25
crawler → CapableOf → utilize computational linguistics	0.25
crawler → CapableOf → never crawl x-robots-tag	0.25
crawler → CapableOf → see x-robots-tag	0.25
crawler → CapableOf → implement a producer/consumer pattern	0.25
crawler → ReceivesAction → placed	0.25
web crawler (subgroup of crawler) → CapableOf → attempt to download page	0.25
crawler → HasProperty → slow	0.21
crawler → CapableOf → get stuck	0.21
crawler → ReceivesAction → written in java	0.16
crawler → HasProperty → flexible	0.16
crawler → ReceivesAction → killed	0.16
crawler → CapableOf → overcome nearly every obstacle	0.16
web crawler (subgroup of crawler) → CapableOf → avoid retrieving duplicate content	0.16