📘📗From 429 Errors to 99.9% Uptime: Handling Rate Limits with SerpApi + Go

 

# From 429 Errors to 99.9% Uptime: Handling Rate Limits with SerpApi + Go

In my last post we built a Google Jobs API with SerpApi + Go.

It worked great... until I deployed it.

Within 1 hour I hit Error 429: `Too Many Requests`.

My app crashed. Users saw nothing.

If you plan to use SerpApi in production, you need 2 things:

1. Retry Logic

2. Proper Error Handling

Let me show you exactly how I did it.

### The Problem: APIs Fail in Production

Google + SerpApi are amazing. But they protect themselves with rate limits.

When you send too many requests, you get blocked.

The tricky part with the official SerpApi Go client: errors come INSIDE the JSON, not as a Go error.

### The Solution: Exponential Backoff in Go

Exponential Backoff means: "If you fail, wait. Then wait longer. Then wait even longer."

1s → 2s → 4s → 8s

Here’s a production-ready function I added to my SerpApi Go Jobs Scraper:

go

package main

import (

    "fmt"

    "log"

    "os"

    "time"

    google "github.com/serpapi/google-search-results-golang"

)

func SearchJobsWithRetry(query, apiKey string, maxRetries int) (map[string]interface{}, error) {

    parameter := map[string]string{

        "engine": "google_jobs",

        "q": query,

        "hl": "en",

    }

    for attempt := 0; attempt < maxRetries; attempt++ {

        search := google.NewGoogleSearch(parameter, apiKey)

        results, err := search.GetJson()

        if err!= nil {

            log.Printf("Network error on attempt %d: %v", attempt+1, err)

        } else {

            // Check if SerpApi returned an error inside the JSON

            if apiErr, exists := results["error"]; exists {

                log.Printf("API error on attempt %d: %v", attempt+1, apiErr)

            } else {

                log.Printf("Success on attempt %d", attempt+1)

                return results, nil // Success!

            }

        }

        // Exponential Backoff: 1s, 2s, 4s

        if attempt < maxRetries-1 {

            waitTime := time.Duration(1<<attempt) * time.Second

            log.Printf("Retrying in %v...", waitTime)

            time.Sleep(waitTime)

        }

    }

    return nil, fmt.Errorf("failed after %d retries", maxRetries)

}

func main() {

    apiKey := os.Getenv("SERPAPI_KEY") // Use environment variables

    data, err := SearchJobsWithRetry("golang developer remote", apiKey, 3)

    if err!= nil {

        log.Fatal(err)

    }

    jobs := data["jobs_results"].([]interface{})

    fmt.Printf("Jobs found: %d\n", len(jobs))

}

### 3 Key Lessons I Learned

**1. Never trust the first request**

In production, 5% to 10% of requests will fail. Plan for it.

**2. Check errors inside the JSON**

With SerpApi Go client, a 429 comes as `results["error"]`, not as `err`.

**3. Exponential > Fixed wait**

`time.Sleep(5s)` 3 times is dumb. `1s, 2s, 4s` respects the API rate limits.

### The Result

Before: App crashed on 429

After: App self-heals and gets the data on retry 2

This is what makes the difference between a demo and a product.

### Get The Full Code

I updated my open-source repo with this and more:

**GitHub**: https://github.com/Aureliopires186/-serpApi-go-jobs-scraper

It now includes:

- Retry with Exponential Backoff

- Better error handling for SerpApi

-.env for API keys

### What's Next?

Next tutorial: Caching SerpApi results in Redis to save credits.

What do you want to see next with SerpApi + Go? Drop a comment.

---

Built with SerpApi from Angola 🇦🇴

Follow me for more Go + API tutorials

By : Pires Aurélio 

https://www.linkedin.com/in/pires-aur%C3%A9lio-511330385