lalela

How Lalela works

Lalela sees requests where they arrive, at the edge or on the server, classifies the agent behind each one, and infers the question that caused it.

Types of AI agent traffic

  • User-triggered fetches: an assistant fetches a page because a person asked it something just now. Examples: ChatGPT-User, Claude-User, Perplexity-User. These are the most valuable visits.
  • Agentic browsers: an AI agent drives a browser to complete a task such as comparing products or signing up.
  • AI search indexers: crawlers that build the indexes AI assistants search, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot.
  • Training crawlers: crawlers that collect pages for training models, such as GPTBot and ClaudeBot.
  • AI referrals: people who click through to a site from an AI assistant's answer.

Verifying agents

User-agent strings are easy to fake, so Lalela labels each visit with a confidence tier:

  • Verified: the request came from an IP range the operator publishes or a network registered to the operator, or carries a valid cryptographic signature (Web Bot Auth).
  • Declared: the user agent names a known agent, but nothing verifies it.
  • Suspected: the request doesn't name itself as an agent, but its behaviour points to automation, such as a browser user agent from a cloud provider's network.

Inferring the question

An agent never sends the user's question to the site. Lalela infers it from evidence:

  1. A query the agent sent to the site's /ask endpoint or site search, which is close to the literal question.
  2. Which pages the agent fetched together, in what order and how quickly.
  3. What those pages say.

Every inferred question is labelled as inferred, with its evidence and a confidence score measured against real prompts.

The /ask endpoint

Sites using Lalela can offer agents an /ask endpoint, listed in their llms.txt. An agent sends a search query and gets back matching passages from the site's own pages, with links. Answers come only from the site's content.