TeknikPrime Bot
About our crawler
teknikprime fetches a page once, when a user asks for it, and analyses it. It does not crawl continuously, discover sites, or store content.
The user agent we send
TeknikPrimeBot/1.0 (+https://teknikprime.online/bot; site audit on user request)Rules we follow
- We obey robots.txt, and raise our own pause to match a Crawl-delay when one is declared.
- We open at most 1 connection to a server at a time. We never send requests back-to-back: consecutive requests to one server are at least 250 ms apart.
- A single audit sends at most 60 requests to the target, and once its 26-second budget is spent the remaining samples are never sent.
- We will not audit the same site twice within 30 seconds, and never more than 12 times an hour — no matter who asks.
- Each request times out at 8 seconds and we stop reading bodies larger than 3 MB.
- We do not retain raw page HTML or analysis results.
- When testing AI crawler access we send those crawlers’ user agents; it is the only way to measure whether a server blocks them. Those requests carry our contact address in a From header.
The honest part about these limits
The numbers above are enforced with counters held in server memory. On a serverless platform that means the counters are per instance: a load spread across many instances can reach a higher total share than it would on one. What actually happens in practice — one script hammering one endpoint — is stopped by these counters. A deployment that needs a hard domain-wide ceiling wants a shared counter (Vercel KV, Upstash) or the platform firewall in front of it. We are not hiding that.
If you would rather block us
Add the two lines below to your robots.txt; we obey them, and your site will not be audited again.
User-agent: TeknikPrimeBot
Disallow: /Contact: bot@teknikprime.online
timeout 8s · body cap 3MB · max 5 redirects