Proxies for Screaming Frog
A private IPv4 for Screaming Frog.
Proxy plans for Screaming Frog
Every plan includes IPv4 and SOCKS5 together, at no extra cost.
Threads are simultaneous connections. The more threads, the higher the parallel load your software can handle.
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 1,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 1,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 1,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 1,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 3,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 3,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 3,000 threads
Pool of 12,000-15,000 IPs
Address rotation
Unlimited traffic
Up to 3,000 threads
Proxy catalog for your task
Why Screaming Frog needs proxies
Screaming Frog SEO Spider scans a site and collects technical data for audits. On large and frequent crawls, requests from one address hit site limits and the crawl slows down.
A crawler produces a dense stream of requests to one domain, and on large sites it runs into limits on the receiving side long before it runs into channel speed. A proxy keeps the crawl from reading as load arriving from a single point.
Features of proxies for Screaming Frog
Screaming Frog SEO Spider crawls large sites with thousands of URLs during technical audits, and a big crawl from one IP hits server limits. A pool spreads the crawl across addresses.
The pool hands over private IPv4 by HTTP, HTTPS or SOCKS5, as IP:PORT bound to your address or IP:PORT:LOGIN:PASS. Where the program takes a single exit, binding by IP covers it, and where it takes a list, address count follows crawl threads.
What proxies you need for Screaming Frog
| Protocol | HTTP, HTTPS or SOCKS5 |
|---|---|
| Single exit | works by binding your own IP |
| List of exits | counted by crawl threads |
| Traffic | unlimited, with nothing extra to pay |
| List refresh | every 15 minutes |
| Authorisation | by binding your own IP in settings, up to 2 addresses |
How to connect a proxy in Screaming Frog
Crawlers differ in how many addresses they will accept: some work from a list, some take only one exit for the whole session. In the second case the arrangement is simple, enter your own IP in the dashboard settings, work by binding, and lower crawl speed through the program's own settings.
Where a list is accepted, take addresses by the number of concurrent crawl threads, with headroom. The thread limit comes with the package and does not stack across packages, so a large project means a larger package rather than a second identical one.
Traffic adds up quickly during a crawl, since full pages come down with their markup and assets. It is unmetered here, nothing is charged per gigabyte, and the limit is expressed in threads alone. A parallel case is covered on the page about Netpeak Spider.
- Where the program takes one exit, work by binding
- With a list, address count follows crawl thread count
- Traffic is unmetered, the limit is expressed in threads
- Crawl speed is regulated in the program's own settings
How to connect proxies in Screaming Frog
To make the pool work, bind the IP of the machine running your proxy software in the settings, a server or a home PC, no binding means no access. The primary output format is IP:PORT, with an additional IP:PORT:LOGIN:PASS for programs that require a login and password.
In Screaming Frog proxies are set in Configuration → System → Proxy: address and port, format IP:PORT, with authorization. Type HTTP.
The setup is checked on a small crawl. How many addresses you need for your site size is visible on the free test.
How to buy proxies for Screaming Frog
The process is simple. Register in the dashboard and request a free test up to 2 hours, bind your IP and message the operator in a ticket to check the setup with Screaming Frog on your own tasks.
When everything fits, pick a package by number of threads and term, pay in cryptocurrency, and access opens right after payment. The test is provided once before purchase, and support replies in a ticket.
FAQ
Will these proxies work with Screaming Frog?
Screaming Frog connects a proxy by host and port in the configuration, with HTTP supported. A suitable private address from the pool is checked on the free test.
Can I change the address between crawls?
Addresses in the pool refresh on rotation every 15 minutes, so between projects you work with different IPs. A long crawl is convenient to split into sessions.
Which protocol does Screaming Frog support?
Screaming Frog works with proxies over HTTP, set under System, Proxy. The test shows how the crawl of your site goes.
See also: proxies for A-Parser, proxies for Netpeak Spider, proxies for Key Collector, proxies for parsing, proxies for price monitoring, proxies for SEO, private IPv4 proxies, proxies for CapMonster, full catalog
© WinGate · source of the material · wingate.me/en/proxy/screaming-frog-proxy