Longhand AlmanacThe crawler
LonghandAlmanacBot/0.1 (+https://longhandalmanac.com/bot)
You saw this in your logs. Here’s what it is.
Longhand Almanac is building an index of creative writing published on websites people built themselves. Poems, stories, essays. Your site came up, most likely because another writer linked to you.
What it does not do
Worth answering first, because it’s why most people look a crawler up.
- It does not train an AI model on your work. Nothing your site publishes is used as training data, here or anywhere else.
- It does not republish your writing. No copies, no mirrors, no full text. A search result shows a title and a short excerpt, and the link goes to your page.
- It does not sell anything. Not your data, not ads against it, not a subscription.
What it does
- Requests pages the way a browser does, one per second at most, and obeys your robots.txt.
- Keeps a copy of the HTML it received, so the same page never has to be fetched twice.
- Works out whether the site is one person’s own writing rather than a magazine, a shop, or a review blog. A small model on a laptop does some of that, reading the page to tell a poem from a product description. It runs locally and sends nothing anywhere.
- Records who links to whom, because that’s how it finds the next writer.
Why bother. Writing on a personal site is close to unfindable now. Search engines rank the sites that work at being ranked, and somebody posting poems on a domain they hand-built is never going to be one of them. This index is built the other way around: no platforms, no publications, and nothing to gain from playing the ranking game.
How to keep it out
Add this to your robots.txt. It’s checked before every request, and a block takes effect on the next visit.
User-agent: LonghandAlmanacBot
Disallow: /
To allow it but keep part of your site private, list those paths instead of /. If you’d rather be removed entirely, including anything already stored, send the domain to the address below and it goes, no questions.
How to be included
Nothing is required. If the crawler finds you and you publish your own writing, you’re in.
Two things make you easier to find, and both are worth having whatever happens with this project: an RSS or Atom feed, and a page that gathers your work in one place, like /poetry or /fiction or /stories. Linking to writers you read helps more than anything else, because that’s the trail this follows.
Who’s behind it
One person, working on it a few hours a week. It isn’t a company and there’s nothing to sign up for.