Skip to content

Listing liveness: alive, dead and blocking directories

Liveness is whether a directory's listing pages respond when our crawler fetches them. A directory is live when they do, dead when they stop, and blocking when it refuses automated access. The same idea applies one level down: an individual listing is live while the directory still shows it.

The directory states

"Alive" means listing pages answered on the most recent checks. "Dead" means they stopped answering; directories in this market shut down often, usually without notice. "Blocks crawler" means the site refuses our crawler, for example with repeated 403 or 429 responses. We record a block and publish it as its own state; we never work around one. How each state is decided is on the methodology page.

There is a fourth state, and today it is the commonest one: a directory we track but have not yet crawled reads "not verified yet". It is not a verdict about the directory. It means we have not looked, and it is the reason such a row carries no rating and no indexation rate either.

When a listing disappears

Every listing we observe is a dated record: when we first saw it and when we last saw it. A listing that vanishes from consecutive crawls is closed with its disappearance date, not deleted. The history stays, which is what makes a question like when a link quietly died answerable, and it is why a scan can show lost listings alongside live ones.

Where to see it: each directory's state is shown on its page in the index, state changes appear in the changelog, and a scan lists your live and lost listings.