Authoritas Structured Keyword Ranking API¶

Asynchronous API¶

Due to its nature, SERPs parsing is not an immediate operation, this API enables you to request multiple background jobs (one job for one keyword and one search engine) which will fetch structured search results from different search engines.

From the initial request you will receive a job id which can be used via a different request (see Get Search Results) to determine whether a job has been completed and retrieve the available data.

Rate Throttling¶

We get SERPs pages one at a time like real users do and do not use any dodgy SEO URL parameters that might identify us as a bot and get our SERP request blocked or worse still sent to a sandbox which returns inaccurate data (yes, it can happen!).

So if you want ask for 10 pages of results (approximately 100 organic results) then this will use 10 API Queries from your monthly API package allowance.

You may send large numbers of keywords to us in one go, but please do not expect results for 1 million keywords to be returned immediately. Depending upon the parameters you request a typical SERP request for 10 pages may take 1 to 1.5 minutes or so.

So when you send us large batches of thousands of keywords in one go, they are sent to your specific queue which is allocated multiple workers to make the SERP requests simultaneously. You will receive a unique Job ID for each keyword which you can later call to see if the SERP is ready. The number of workers assigned to your queue depends upon your API package allowance per month and any specific agreements you may have made with your Account Manager.

We don’t limit how many keyword jobs you can send to us simultaneously. In fact, we recommend sending them in large batch sizes of 1,000 keywords or more.

However, we limit the number of workers assigned to your queue, which may limit the rate at which your queue can process queries to the Authoritas SERPs API. We allocate resources according to the size of the API package and assume that your API queries will be spread evenly across each 24 hour period in each calendar month. We take your monthly API allowance and divide it by 30 days and then 86,400 to generate an expected daily rate of API Queries per second; e.g. 1,000,000 API Query allowance equates to 0.39 API Queries per second. We then allocate resources using this expected API Query rate per second number and we add a contingency of 30%. In this example, this would increase the rate to 0.5 API Queries per second.

We can often allocate more workers to speed up your queue if you have specific requirements. So please speak to your Account Manager so we can work together to accommodate your specific needs.

Request Limitations¶

The default limit is 70 POST requests per second.

Recommended limit: Send a maximum of 5 concurrent requests per second to avoid hitting the limit.

Batch Processing¶

A single POST request now takes ~200 ms, which means one thread can create up to 5 jobs per second.

With 5 concurrent threads, you can create up to 90,000 jobs per hour, depending on network conditions and distance.

Sending Jobs¶

Send all jobs at once with a callback URL.

Wait for the callback from our system to confirm job completion.

Avoiding Gaps in Processing¶

Do not rely on batches where you send, for example, 500 jobs, wait for their completion, and then send another batch.

This method can cause processing gaps due to the time needed to finish the last jobs of the first batch and the creation of new jobs for the second batch.

Job Polling¶

We suggest waiting a minimum of 3 minutes to poll for a Job ID to see whether a keyword SERP is ready or not and later repeat the check not more often than every 30 seconds.

Documentation:¶