How Readable works

#What Readable does

You give Readable a website address. It reads up to 30 public pages of that site, removes navigation, scripts, cookie banners and hidden elements, converts what is left to markdown, and writes three things:

• llms.txt: the site's name, a one-line summary, and links to every page, grouped into About, Tokenomics, Docs and Optional.
• llms-full.txt: the markdown of every page in one file.
• pages/*.md: one markdown file per page.

You download them as a zip and upload them to your own site. Readable never hosts them for you, and never needs a login, a wallet or access to your hosting.

The same input always gives the same output, byte for byte: no timestamps, no random order.

#What llms.txt is

llms.txt is a community proposal: a markdown file at the root of a website (or of a folder) that tells language models and AI agents what the site is and where its important pages are, ideally as plain markdown. It is not an official standard, and no AI product is required to read it.

Version 2 of the proposal (August 2026) also describes where a page's markdown version lives: at the page URL with .md appended (page.html.md) or with the extension replaced (page.md), or wherever the page points with <link rel="alternate" type="text/markdown">. The checker accepts all of these. Readable writes its copies to pages/<page>.md and links them from llms.txt.

#Upload your files

Upload everything to your site's web root, so these addresses work:

https://your-site.com/llms.txt
https://your-site.com/llms-full.txt
https://your-site.com/pages/<page>.md

Your site runs on	Put the files in
Next.js, Vite, Astro, SvelteKit, most frameworks	public/ (in Next.js, not the pages/ folder)
WordPress	the folder that contains wp-config.php
GitHub Pages, Netlify, Cloudflare Pages, static hosts	the folder you publish
Webflow, Wix, Squarespace, Framer	usually not possible: these builders don't allow it

Then run the checker to confirm your score.

#Scanning a folder

Enter a folder address ending in / (for example a GitHub Pages project site, https://you.github.io/project/) and Readable scans only that folder. The files are then meant for that folder: https://you.github.io/project/llms.txt and …/project/pages/*.md. Version 2 of the proposal says an llms.txt covers the pages under its folder. Enter the homepage to scan the whole site.

#Optional: point pages to their copies

The result page also gives you snippets that let agents go straight from a page to its markdown copy, as version 2 of the proposal describes:

• a <link rel="describedby" href="/llms.txt"> tag for every page,
• a <link rel="alternate" type="text/markdown" href="/pages/….md"> tag per page,
• or the same as Link: headers, in the _headers format of Netlify and Cloudflare Pages.

In Next.js, set alternates.types['text/markdown'] in a page's metadata. All of this is optional: agents that read llms.txt find the copies anyway.

Readable keeps every copy in one pages/ folder on purpose. The proposal also allows copies next to each page (/about/index.md), but that creates real folders like /about/, and on WordPress and many PHP or Nginx setups a real folder takes priority over the page with the same address, which can break it.

#The score

The checker fetches /llms.txt and the pages it lists, the way an agent would. It checks at most 10 links per site. For a URL in a folder (like a GitHub Pages project site), it uses the most specific llms.txt it finds, as version 2 of the proposal describes.

Check	Points
/llms.txt exists, has an H1 title (# Name) and at least one section (## Section)	20
A summary blockquote (> …) under the title	15
Markdown copies of listed pages: the link is itself markdown, or the page has a markdown version	25, proportional
Listed pages show at least 200 characters of text without running JavaScript	20, proportional
Links in llms.txt answer with 2xx	20, proportional

80 and up is AGENT-READABLE, 50 to 79 NEEDS WORK, below 50 NOT READABLE. Results are cached for 10 minutes.

#Pages that need JavaScript

Readable does not run JavaScript and has no headless browser. If a page shows almost no text until scripts run (a typical single-page app), Readable flags it and leaves it out. To fix it, render the page on the server (static generation or server-side rendering), or write its markdown copy by hand.

#Badge

https://agentreadable.site/b/your-site.com.svg
https://agentreadable.site/b/your-site.com.svg?theme=light

The badge shows your live score and is cached for 24 hours. Make one on the badge page.

#Command line

The same scan and checker run in your terminal with Node.js 20 or later. Nothing goes through agentreadable.site:

npx agentreadable scan your-site.com --out public
npx agentreadable check your-site.com --min 80

• scan writes llms.txt, llms-full.txt and pages/*.md into --out (default readable/) and prints the score the site will get once they are uploaded.
• check prints the five checks and the score. With --min, it exits with code 1 when the score is lower, so a CI build can fail when agents can no longer read your site. --json prints the result as JSON.
• --allow-private lets it fetch a development server on localhost.

In GitHub Actions:

- uses: actions/setup-node@v4
  with:
    node-version: 22
- run: npx --yes agentreadable@0.1 check https://your-site.com --min 80

#The crawler

Readable fetches pages with this user agent:

Readable/0.1 (+https://agentreadable.site/docs#crawler)

• It only fetches pages when someone asks for it: a scan, a check, or a badge view. The only scheduled fetches are the daily re-check of sites someone chose to monitor and the weekly Pons Readability Index.
• It reads robots.txt before anything else and obeys it for the Readable user agent and for *, in scans, checks, badges, the monitor and the index, llms.txt included. If robots.txt can't be fetched because of a server error, it fetches nothing. Links in your llms.txt that robots.txt blocks are skipped, not counted as broken.
• It never visits login, sign-up, admin, account or dashboard paths, and never sends cookies.
• At most 30 pages per scan, 2 MB per page, 3 redirects per request, and only public http/https addresses: private networks are blocked.

To keep Readable off your site:

User-agent: Readable
Disallow: /

#Monitor

The monitor re-checks a site about once a day and alerts you on Telegram or by webhook when:

• the score drops below 80 (while it stays low, you get a reminder at most once a week),
• a link in your llms.txt stops working,
• a page linked from the watched page is missing from your llms.txt. Pages already there at the first check count as the baseline.

Each check reads your llms.txt, up to 3 of the links it lists, and the watched page. The last 30 scores are kept.

Telegram: message the Readable bot; it replies with your chat ID.

Webhook: an https address on the public internet. Redirects are not followed. Readable sends a POST with a JSON body like:

{
  "type": "readable.alert",
  "url": "https://your-site.com/",
  "score": 55,
  "label": "NEEDS WORK",
  "reasons": ["The score is 55/100 (NEEDS WORK)."],
  "brokenLinks": [],
  "newPages": ["/staking"]
}

There is no account. When you start watching, you get a private link to see the history or stop the watch; we only store a hash of its secret, so keep the link safe. Capacity is limited to keep the service free (250 sites).

#Pons Readability Index

The index ranks the websites of tokens launched on Pons, the token launchpad on Robinhood Chain, by the same score as the checker.

• Which tokens: tokens that graduated from their bonding curve (most launches never do) and whose creator entered a website. Links to X, Telegram, Discord, Linktree, GitHub, chart sites and the like are not websites and are left out. The index holds the 150 most recent of them.
• Where the data comes from: the public launch records of the Pons V2 factory on Robinhood Chain (chain ID 4663), read through the chain's public RPC: graduation events, and each token's name, symbol and website as stored in its contract. Nothing is signed and no wallet is involved.
• How it updates: sites are checked through the week, each once, with up to 5 links from its llms.txt. Every Monday (UTC) the finished week is published. The 7-day change compares a token's score with last week's index; "new" means the token was not in it.
• Launched this week means the token launched in the same week it is listed for.
• A site whose robots.txt blocks Readable is left out. So is, for that week, a site that needs too many requests to check (long redirect chains).

The index says nothing about a token's value, team or safety. It is not investment advice, and it is not affiliated with Pons or Robinhood. To keep your site off the list, block Readable in robots.txt; it drops out within two weekly updates.

#Privacy

• No accounts, no cookies, no tracking scripts.
• A scan runs in your browser. Our server only fetches the pages for you and does not store them. The result stays in your browser tab.
• If you click Publish result page, we store your llms.txt and the scores for 60 days at a public link.
• If you start watching a site, we store its address, your Telegram chat ID or webhook URL, and the last 30 scores until you stop the watch.
• We don't keep request logs. Cloudflare runs our servers and processes requests to deliver them. Fonts load from Google Fonts.

#Limits

• 30 pages per scan, closest pages first; 2 MB per page.
• Scans, checks and publishing are rate-limited per network to keep the service free.
• The checker scores at most 10 links of an llms.txt.
• Published result pages expire after 60 days.

#Not guaranteed today

• AI products are not required to read llms.txt. Some agents and developer tools fetch it when they find it; many don't.
• llms.txt is a proposal, and its format can still change.
• It is not an SEO tool. Don't expect search ranking changes.

#What this is not

• Not a security audit, and not a safety check for any token.
• Not investment advice.
• Not hosting, and not a browser: no JavaScript is run.