StriiveSiteReader: who we are and how to block us
If you found StriiveSiteReader in your server logs, the visit came from Striive, a Danish website builder. Here is why it came, what it read, and how to stop it.
StriiveSiteReader/1.0 (+https://striiveai.com/bot)Why it visited
The reader reads a website only when someone gives us its address. Usually that is one of our customers, who wants their own website rebuilt on Striive. Now and then we read a business’s website ourselves to prepare a demo for them.
It never crawls the web on its own.
What it reads
On one visit, the reader fetches:
- robots.txt
- the sitemap: /sitemap.xml, /sitemap_index.xml, or the Sitemap: lines in robots.txt
- the home page
- the pages the customer picked, at most 15
- up to 5 more company pages, such as about, contact or opening hours, read for facts only
- the stylesheets those pages load
- the photos shown on the picked pages
What it leaves alone
It does not load scripts or fonts, and it never goes into logged-in or private areas.
How much it asks of your server
One visit is at most 400 requests, 60 MB and 3 minutes. At most 4 requests run at once against your site.
If your robots.txt sets a Crawl-delay, the reader waits that long between requests, up to 2 seconds.
What we keep
The customer’s new website keeps the text and photos it uses. Everything else we read is deleted once the build is done with it, and after one hour at the latest.
We keep a record of which addresses we read and when. We never keep the content.
How to block it
The reader always follows robots.txt, on every website, including our customers’ own. Its token is StriiveSiteReader. A group that names it takes precedence over User-agent: *, so you can block it without changing how other bots see your site.
User-agent: StriiveSiteReader
Disallow: /User-agent: StriiveSiteReader
Disallow: /private/A blocked website cannot be read. If it is your own website and you want to build it on Striive, you can upload screenshots instead.
Questions
Write to us at admin@striiveai.com.