To make your website agent-ready, give personal AI agents three things: content they can read, a sanctioned way to ask questions and complete tasks, and rules that give them the customer's authority and no more. The 12 points below cover all three, in the order most teams can ship them.
Personal agents such as Instinct, Muse, Grok bot and Town, plus general assistants like ChatGPT, Claude and Perplexity, now research, compare, buy, cancel and contact support for the people who use them. Serving those agents well is what B2A (business to agent) means. This page is the build list. If you are still choosing between methods, read WebMCP vs public endpoint vs llms.txt first. If you want to know whether agents are already in your queue, start with the 7 signs AI agents are contacting your support team.
Key takeaways
Readable comes first. Agents can't use what they can't fetch. Audit bot walls, JavaScript-only content and robots.txt before you build anything new.
Publish facts in plain text. Pricing, fees, eligibility, returns and cancellation terms should live on stable, plain-text URLs that match what your support team says.
Give agents a front door. A sanctioned endpoint lets any agent ask a question in plain HTTP. WebMCP lets supporting browser agents call tools on your page. Most sites will want both over time.
Machine traffic needs machine rules. Rate limits, a clear "check back in N seconds" and idempotent retries stop polling from turning into bans or duplicate tickets.
Actions need identity. Anonymous agents get public answers. Anything that touches an account goes through the same verification a human would face, and the agent never gets more authority than the customer.
Measure and test. Tag agent traffic, read the conversations, and run a real agent through your flows before your customers' agents do it for you.
What "agent-ready" means in practice
An agent-ready website is one where a customer's agent can find accurate answers and get legitimate tasks done without scraping, guessing or tripping your fraud controls. It is a support and operations project as much as a web project, because the questions agents ask are the same ones customers ask: what does this cost, am I eligible, where is my order, how do I cancel.
The checklist is split into three layers:
Readable (points 1 to 4): agents can fetch and understand your public content.
Reachable (points 5 to 7): agents have a sanctioned way to ask and act, at a pace your systems can handle.
Governed (points 8 to 12): identity, policies, disclosure, measurement and testing keep it safe and improving.
You don't need all 12 on day one. Points 1 to 4 are mostly configuration and content. Points 5 to 11 are where a support platform does the heavy lifting. Point 12 never really ends.
1. Keep your site crawlable, with no accidental bot walls
Why it matters: many sites block agents without meaning to. A challenge page set to "block all automated traffic", a WAF rule written for scrapers, or a pricing table rendered only by client-side JavaScript can all leave an agent with an empty page. The agent then answers from stale third-party sources, or tells its user it couldn't find anything.
What to do:
List every bot-management and WAF rule that returns a challenge, a 403 or a CAPTCHA, and note which paths it covers. Help center, pricing, policy and status pages should almost never sit behind a challenge.
Render key facts in the HTML response. If a price, fee or eligibility rule only appears after JavaScript runs, many agents will never see it.
Keep your help center public. Gating it behind a login hides answers from agents and from search engines at the same time.
Separate "block abusive traffic" from "block automated traffic". Scrapers and credential stuffing deserve blocking. A customer's agent reading your refund policy does not.
How to check: fetch your top ten support and pricing URLs with a plain HTTP client (curl is fine) and read the raw response. If the answer to a common question isn't in that text, an agent probably can't see it either.
2. Make deliberate robots.txt choices
Why it matters: robots.txt is the first file many automated clients read. The Robots Exclusion Protocol is now an IETF standard, RFC 9309, and it is worth reading the short spec because a few details catch teams out.
RFC 9309 states plainly that "these rules are not a form of access authorization." robots.txt asks crawlers to behave; it doesn't enforce anything. Use real access controls for anything private.
If your robots.txt returns a server error (5xx), the RFC says crawlers must assume complete disallow. A flaky robots.txt can quietly remove you from every well-behaved crawler.
If it returns a 4xx, crawlers may access any resource. A missing file means "everything is allowed".
Crawlers should not use a cached copy for more than 24 hours, so changes take effect within about a day for compliant clients.
What to do: decide separately for training crawlers, search crawlers and user-initiated fetches. Vendors document these differently. OpenAI, for example, documents GPTBot for training, OAI-SearchBot for search, and ChatGPT-User for pages ChatGPT visits when a user asks it something, and notes that because ChatGPT-User actions are user-initiated, robots.txt rules may not apply. Blocking a training crawler is a reasonable policy choice. Blocking the fetches your own customers trigger is usually not what you meant.
How to check: request /robots.txt and confirm it returns a 200, parses cleanly and doesn't disallow your help center, pricing or policy paths for the agents you want to serve.
3. Publish an llms.txt
Why it matters: an llms.txt file gives agents a short, curated map of your site in Markdown, instead of making them infer your structure from navigation menus. It is a proposed convention, described at llmstxt.org, and not a ratified standard. The proposal notes that thousands of sites publish one and that Chrome's Lighthouse checks for it as part of its agentic browsing audits.
What to do: follow the format in the proposal:
an H1 with your company or site name (the only required section);
a blockquote with a short summary of what you do;
optional paragraphs of context;
H2 sections containing lists of links, each with a short note on what the page covers.
For a support use case, the useful sections are usually pricing, plans and fees; policies (refunds, cancellations, privacy); how to contact support; and, once you have one, how an agent can ask questions directly. The proposal also suggests offering clean Markdown versions of key pages at the same URL with .md appended, which saves agents from parsing layout HTML.
How to check: open /llms.txt in a browser. Every link should resolve, every note should still be accurate, and nothing in it should contradict your help center.
4. Publish clean, machine-readable facts
Why it matters: the most common agent questions are factual: price, fees, rates, eligibility, delivery times, cancellation terms. When those facts are scattered across marketing pages, PDFs and old blog posts, agents pick up whichever version they find first. Your support team then spends time correcting answers your own website gave out.
What to do:
Pick one canonical URL per fact set: one pricing page, one fees page, one refunds page, one cancellation page.
Write the facts in plain sentences and simple tables. "The monthly fee is $12. You can cancel at any time from Settings, and the fee stops at the end of the billing period" beats a clever layout.
Put a "last updated" date on policy pages, and remove or redirect superseded versions.
Where it fits, add structured data (for example, product, offer or FAQ markup) so the same facts are available in a typed form.
Make sure your support knowledge base and these public pages say the same thing. An agent that gets two different answers will ask again, often on another channel.
How to check: ask three general assistants your top five pricing and policy questions and compare their answers to your canonical pages. Every mismatch is a content fix, and often a stale page to retire.
5. Offer a sanctioned endpoint for questions
Why it matters: reading pages only gets an agent so far. Questions like "does this plan cover my situation" or "what happens to my balance if I close the account" need a conversation. Without a sanctioned way to ask, agents fill your contact form, open chat sessions in a headless browser, or call your phone line, and their traffic looks like a bot attack.
What to do: publish one plain-HTTP endpoint that any agent can call with a question and get a plain JSON answer. The design choices that matter:
Same brain as your support. Answers should come from the same knowledge, policies and workflows that serve human customers, not a separate bot FAQ that drifts out of date.
Conversation continuity. An agent should be able to ask a follow-up in the same conversation without repeating context.
Self-describing responses. Each response should tell the agent what to do next, so it can follow the protocol without hardcoding it.
Discoverable. Link it from llms.txt and from a short notice on the site, so agents find it before they reach for your contact form.
A usage policy. State that agents should only call it with their user's knowledge and send only the question itself, with no personal information unless the user asks for it.
We run one on lorikeetcx.ai. Our llms.txt has an "Ask our support agent (for AI assistants)" section, and the footer notice points agents to GET https://api.lorikeetcx.ai/v1/ask/<public key>?q=.... The response is plain JSON and includes instructions for checking back and asking follow-ups. For the security side of exposing any endpoint, see our guide to secure API and webhook integrations for AI support.
How to check: call the endpoint from a terminal with a real customer question. You should get an answer, or a clear instruction on when to check back, without a browser, login or API key.
6. Register WebMCP tools where supported
Why it matters: WebMCP is a proposed web standard that lets a page expose functions or HTML forms as "tools" with natural-language descriptions and structured schemas, which browser-based agents can call directly. Its explainer describes pages using it as in-page MCP servers: the site's own code performs the action and keeps the visible UI in sync, instead of the agent scraping and clicking.
What to do:
Treat WebMCP as an addition to the endpoint, not a replacement. It only works in browsers and agents that support the proposal. The project's implementation status page lists origin trials in Chrome and Edge, experimental support in Brave, and support in ChatGPT Desktop, with Firefox and Safari positions still open. Most agents today will use your endpoint or read your pages.
Start with a small set of tools that map to real customer jobs: ask a support question, check an order, start a booking. Name and describe them plainly.
Route tool calls through the same policies as every other channel. A tool is a new way in, not a new set of rules.
Expect the API to change while the proposal matures, and keep the implementation thin.
The tradeoffs between WebMCP, a public endpoint and llms.txt get a full treatment in our three-way comparison.
How to check: open your site in a browser with WebMCP testing enabled and confirm your tools register and return results. If your target customers don't use a supporting browser yet, this point can wait.
7. Set rate limits and a check-back protocol
Why it matters: agents are patient and fast. They poll, retry and wait for release times in ways no human would. In September 2026, a Resy user's agent sent about 200 requests an hour, polling every 10 minutes and every 0.4 seconds around the daily table release, and the account was deactivated for acting like a bot before being reinstated two days later. The blocking vs serving breakdown covers that case in full. The lesson for builders: if you don't tell agents how fast they may go, your fraud rules will decide for them, and the person who loses is your customer.
What to do:
Set explicit per-key or per-conversation rate limits and return HTTP 429 when they're exceeded. RFC 6585 defines 429 Too Many Requests and allows a Retry-After header saying how long to wait.
For slow answers, respond immediately with a pending status and a check-back URL, and let the agent poll at a stated interval instead of holding a connection open.
Make retries idempotent. The same question retried five times should produce one conversation and at most one ticket, not five.
Explain the rules in the response body in plain language. Agents read instructions; humans reading logs later will too.
How to check: script 50 rapid requests against your endpoint. You should see clear 429s with a wait time, one conversation in your support tool, and no account flags on a test customer.
8. Require identity step-up before any account action
Why it matters: an anonymous agent asking "what's your refund policy" is harmless. An agent asking "refund my last order" is acting on an account, and it has to prove it speaks for the account holder. Agents also push on the edges of login flows. One agent that couldn't sign in with a saved password went through the one-time-code path instead, reading the code from the user's email, as described in Personal agents are coming. No one is ready.
What to do:
Treat endpoint conversations as anonymous by default. Public information only until the customer is verified.
Before any action or account-specific answer, run the same verification you use on chat, email and phone: one-time codes to the registered contact, knowledge checks, or a signed-in handoff.
Make the step-up explicit in the conversation ("to change this booking I need to verify the account holder; a code has been sent to the email on file") so the agent can relay it to its user.
Watch for patterns such as repeated one-time-code requests or failed verifications across channels, and treat them as risk signals, not automatic bans.
How to check: try to change a test account through your endpoint without verifying. It should refuse politely and explain the step-up. Then verify and confirm the same action works.
9. Write action policies: the customer's authority and no more
Why it matters: once an agent is verified, the question becomes what it may do. The simplest rule that holds up is that an agent gets exactly the authority the customer has on your other channels, under the same policies, and sees only what the customer would see.
What to do:
List the actions customers can take themselves today (update details, book, reschedule, cancel, request a refund) and make those available to verified agents under the same limits.
Keep the same eligibility rules, fees, cooling-off periods and retention offers. An agent cancelling a subscription should get the same process a person gets, no harder and no easier.
Mark actions that always need a human: high-value changes, disputes, complaints, hardship or anything regulated that your team reviews today.
Put these rules in one place that every channel reads from, so web chat, email, voice and agents can't drift apart.
How to check: pick your five most common account actions and run each through a human channel and the agent channel. The outcome, the checks and the escalation point should match.
10. Tell agents what they're talking to and what the rules are
Why it matters: agents follow instructions well when they're given clearly. Disclosure also protects the customer: their agent should know it is talking to your AI concierge, what it may do, and when a human will get involved.
What to do:
Add a short, visible notice on your site that tells agents where the endpoint is, how to use it, and what the usage policy is. On lorikeetcx.ai this sits in the footer.
Include the rules in the responses: rate limits, check-back timing, what needs verification, and how to reach a person.
Say plainly when the answer comes from an AI concierge and when a case has been handed to a human.
Keep the tone the same as your human-facing support. Agents quote you back to their users.
How to check: read one full agent conversation from your logs as if you were the customer. Could you tell who answered, what was done, and what happens next?
11. Set up analytics for agent traffic
Why it matters: you can't improve what you can't see, and agent traffic hides well. It shows up as odd-hour contacts, identical phrasing, precise follow-ups and bursts around release times. Our 7 signs guide lists the patterns in detail.
What to do:
Log every endpoint and WebMCP conversation as a ticket in your support tool, tagged as agent traffic, so it sits next to the human conversations on the same topics.
Group conversations by topic. The questions agents ask most are a direct list of gaps in your public content.
Track outcomes: answered, escalated, action completed, verification failed. Watch escalation rate by topic as your main quality signal.
In web analytics, segment documented user-agent strings and referrers from assistants separately, while remembering that many agents browse with ordinary browser strings.
How to check: at the end of the first week, you should be able to answer three questions from your data: how many agent conversations happened, what they were about, and how many were resolved without a human.
12. Test with a real agent, then keep monitoring
Why it matters: your customers' agents will run your flows whether or not you have. It is better to find the broken step yourself.
What to do:
Give a general assistant a realistic task: "Find out whether this company charges a fee to cancel, and cancel my plan if it doesn't." Watch where it reads, where it asks, and where it gets stuck.
Test each layer: can it find the facts (points 1 to 4), reach the endpoint (5 to 7), and get stopped correctly before an unverified action (8 to 9)?
Re-run the same tests after every pricing, policy or site change. Stale answers are the most common failure, and they arrive silently.
Review a sample of agent conversations weekly, the same way you QA human ones.
How to check: keep a short test script of ten tasks, run it monthly, and track pass rates over time.
The printable agent-ready checklist
Copy this table into your project tracker. "Done when" is the test that tells you the point is finished, not just started.
# | Point | Done when | Usual owner |
|---|---|---|---|
1 | Crawlable, no accidental bot walls | Top 10 support and pricing URLs return full answers to a plain HTTP fetch | Web / security |
2 | robots.txt choices | Returns 200, parses cleanly, policy set per crawler type, help and policy paths allowed | Web / SEO |
3 | llms.txt | Published at /llms.txt in the llmstxt.org format, every link resolves | Web / content |
4 | Machine-readable facts | One canonical page per fact set, dated, matching the support knowledge base | Content / support ops |
5 | Sanctioned question endpoint | Any agent gets an answer or check-back instruction over plain HTTP | Support ops / platform |
6 | WebMCP tools | Tools register and return results in a supporting browser (or consciously deferred) | Web engineering |
7 | Rate limits and check-back | Bursts get 429s with a wait time; retries create one ticket | Platform |
8 | Identity step-up | Unverified account actions are refused with a clear step-up | Support ops / security |
9 | Action policies | Top 5 account actions behave the same for humans and agents | Support ops / compliance |
10 | Disclosure to agents | Site notice live; responses state rules, AI status and human handoff | Support ops / legal |
11 | Agent-traffic analytics | Agent conversations tagged, sorted by topic, with outcomes tracked | Support ops / analytics |
12 | Test and monitor | Ten-task agent test script runs monthly and after every policy change | Support QA |
Where to start if you only have a week
Days 1 and 2: points 1 and 2. Fix bot walls on public support pages and clean up robots.txt. This is configuration, not a project.
Days 3 and 4: points 3 and 4. Consolidate pricing and policy facts, then publish an llms.txt that points to them.
Day 5: run point 12 once. The failures you see will tell you whether points 5 to 11 are urgent for you now or next quarter.
If you already see agent contacts in your queue, skip ahead to points 5, 7 and 8. An endpoint with rate limits and identity step-up changes the experience for those agents immediately, and it takes pressure off the fraud rules that are currently deciding for you.
How Lorikeet covers points 5 to 11
Points 1 to 4 are your website and content. Points 5 to 11 are where most of the build effort sits, and they are what Lorikeet B2A handles. B2A gives agents a sanctioned front door to the same AI concierge that already serves your customers on chat, email and voice, with the same knowledge, workflows, actions and guardrails.
Checklist point | How Lorikeet B2A handles it |
|---|---|
5. Sanctioned endpoint | A public endpoint any agent can call in plain HTTP, returning plain JSON with instructions for follow-up questions in the same conversation |
6. WebMCP tools | Agents that support WebMCP (a proposed web standard) call tools registered on your website, reaching the same concierge |
7. Rate limits and check-back | An agent asks, then checks back for the answer. Rate limits cap the volume, and a retry doesn't open a second ticket |
8. Identity step-up | Endpoint conversations are anonymous by default. Verifying the customer uses the same identity verification skills and policies the concierge applies on other channels |
9. Action policies | Agents get no more authority than the customer: the same guardrails and action policies, and they see only what a customer would see |
10. Disclosure | The concierge explains the rules to agents, including request limits and when to check back |
11. Analytics | Every agent conversation lands in Lorikeet as a normal ticket, sorted by topic, so you can see what agents want |
There is no backend work: the concierge answers from your existing knowledge. Setup is four steps:
Create an account.
Coach helps you build and test a concierge for your website.
Turn on the agent-facing endpoint.
Paste the snippet Coach gives you into your website.
Because B2A runs on the same concierge as your human support, a conversation that starts with a question can end in a booking, a sale or a resolution. For proof of the concierge itself (separate from B2A, which launched on 17 September 2026), Wonderschool went from a 10% to a 100% answer rate in its first full month live with Lorikeet. You can also test our own endpoint today: the instructions are in our llms.txt.
Lorikeet doesn't do bot management or payments. For blocking scrapers or processing agentic checkout, you'll want a specialist alongside it. Our roundup of the best platforms for handling AI agent traffic in customer support covers who does what.
If agents are already showing up in your queue, we should talk
Most teams find agent traffic before they plan for it: a spike of identical questions, an account flagged for polling, a customer asking why their assistant got blocked. If that sounds familiar, we can walk through where you are on this checklist and what it would take to serve those agents with rules instead of bans. Get a demo and bring your hardest agent conversation.
