TopeziaWidget

TopeziaWidget is the crawler behind Topezia's site chat. It reads a website so that the chat assistant on that same website can answer visitors' questions from the site's own pages.

It only visits sites that asked for it

This is not a general web crawler. It never discovers sites on its own and never follows links off the site it was pointed at. It visits a website only after the owner of that website has signed in to Topezia and either typed their own domain in or installed our WordPress plugin on their own site.

If TopeziaWidget is reading your site, someone with access to your Topezia account asked it to. If that wasn't you and you'd like it to stop, the last section tells you how — and we'd like to hear about it.

How to recognise it

Every request carries this user agent:

TopeziaWidget/1.0 (+https://www.topezia.com)

A user agent string can be copied by anyone, so requests are also cryptographically signed using Web Bot Auth (HTTP Message Signatures, RFC 9421). Each request carries Signature, Signature-Input and Signature-Agent headers, and our public key is published at:

https://www.topezia.com/.well-known/http-message-signatures-directory

That signature is what proves a request is genuinely ours. We don't publish an IP range, because we don't have dedicated addresses to publish — anything claiming to be TopeziaWidget from a particular IP is not something we can vouch for, but a valid signature is.

What it does

• Requests /robots.txt first, and obeys it. Rules addressed to TopeziaWidget take priority over *. Disallowed paths are never requested.
• Reads your sitemap if you have one, otherwise follows links from your homepage, one level deep.
• Fetches up to a few hundred pages, five at a time, and stops. A scan takes well under a minute on a typical site.
• Retries once, after a pause, on 429 or 503 — and only once.
• Requests GET only. It never submits forms, never signs in, and never touches anything behind a login.

When it visits

Only when there is a reason to: when a website is first connected, and when its owner presses re-scan. It does not poll, and it does not crawl on a schedule.

What it keeps

The readable text of your public pages, so the assistant can quote and link to them when answering a visitor. On a shop it also reads product names, prices and links, so it can point shoppers at the right product and hand them to your own checkout. Nothing is used to train models, and nothing is shared with anyone but the site's own Topezia account.

How to block it

Add this to your robots.txt and it will stop on the next scan:

User-agent: TopeziaWidget
Disallow: /

To remove the chat entirely, the site owner can turn it off in their Topezia dashboard, or delete the website from their account, which stops the crawler and deletes what it read.

Contact

Questions, or a crawl you didn't expect: hello@topezia.com. We answer.