Go Singleflight Melts in Your Code, Not in Your DB

The original article is posted on the VictoriaMetrics blog:

Go Singleflight Melts in Your Code, Not in Your DB

So, when you've got multiple requests coming in at the same time asking for the same data, the default behavior is that each of those requests would go to the database individually to get the same information. What that means is that you'd end up executing the same query several times, which, let's be honest, is just inefficient.

How singleflight suppresses duplicate requests

So, now you've got a pretty good idea of what this post is about, right?

Singleflight

The

Demonstration of singleflight in action

Once the first call finishes, any waiting goroutines get the same result, as we can see in the output. Although we had 5 goroutines asking for the data, fetchData only ran twice, which is a massive boost.

The shared flag confirms that the result was reused across multiple goroutines.

"But why is the shared flag true for the first goroutine? I thought only the waiting ones would have shared == true?"

Yeah, this might feel a bit counterintuitive if you're thinking only the waiting goroutines should have shared == true.

The thing is, the shared variable in g.Do tells you whether the result was shared among multiple callers. It's basically saying, "Hey, this result was used by more than one caller." It's not about who ran the function, it's just a signal that the result was reused across multiple goroutines.

"I have a cache, why do I need singleflight?"

The short answer is: caches and singleflight solve different problems, and they actually work really well together.

In a setup with an external cache (like Redis or Memcached), singleflight adds an extra layer of protection, not just for your database but also for the cache itself.

Singleflight on cache miss

In this setup, we only use singleflight when a cache miss happens.

Singleflight Operations

To use singleflight, you first create a Group object, which is the core structure that tracks ongoing function calls linked to specific keys.

It has two key methods that help prevent duplicate calls:

• 
  group.Do(key, func): Runs your function while suppressing duplicate requests. When you call Do, you pass in a key and a function, if no other execution is happening for that key, the function runs. If there's already an execution in progress for the same key, your call blocks until the first one finishes and returns the same result.


• 
  group.DoChan(key, func): Similar to group.Do, but instead of blocking, it gives you a channel (<-chan Result). You'll receive the result once it's ready, making this useful if you prefer handling the result asynchronously or if you're selecting over multiple channels.

We've already seen how to use g.Do() in the demo, let's check out how to use g.DoChan() with a modified wrapper function:

CODE Kopieren

// Wrap the fetchData function with singleflight using DoChan

func fetchDataWrapper(g *singleflight.Group, id int) error {

    defer wg.Done()



    ch := g.DoChan("key-fetch-data", fetchData)



    res := <-ch

    if res.Err != nil {

        return res.Err

    }



    fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, res.Val, res.Shared)

    return nil

}

CODE Kopieren

package singleflight



type Result struct {

    Val    interface{}

    Err    error

    Shared bool

}

To be honest, using DoChan() here doesn't change much compared to Do(), since we're still waiting for the result with a channel receive operation (<-ch), which is basically blocking the same way.

Where DoChan() does shine is when you want to kick off an operation and do other stuff without blocking the goroutine. For instance, you could handle timeouts or cancellations more cleanly using channels:

CODE Kopieren

func fetchDataWrapperWithTimeout(g *singleflight.Group, id int) error {

    defer wg.Done()



    ch := g.DoChan("key-fetch-data", fetchData)

    select {

    case res := <-ch:

        if res.Err != nil {

            return res.Err

        }

        fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, res.Val, res.Shared)

    case <-time.After(50 * time.Millisecond):

        return fmt.Errorf("timeout waiting for result")

    }



  return nil

}

This example also brings up a few issues that you might run into in real-world scenarios:

• The first goroutine might take way longer than expected due to things like slow network responses, unresponsive databases, etc. In that case, all the other waiting goroutines are stuck for longer than you'd like. A timeout can help here, but any new requests will still end up waiting behind the first one.


• The data you're fetching might change frequently, so by the time the first request finishes, the result could be outdated. That means we need a way to invalidate the key and trigger a new execution.

Yes, singleflight provides a way to handle situations like these with the group.Forget(key) method, which lets you discard an ongoing execution.

The Forget() method removes a key from the internal map that tracks the ongoing function calls. It's sort of like "invalidating" the key, so if you call g.Do() again with that key, it'll execute the function as if it were a fresh request, instead of waiting on the previous execution to finish.

Let's update our example to use Forget() and see how many times the function actually gets called:

CODE Kopieren

func fetchDataWrapperWithForget(g *singleflight.Group, id int, forget bool) error {

    defer wg.Done()



    // Forget the key before fetching

    if forget {

        g.Forget("key-fetch-data")

    }



    v, err, shared := g.Do("key-fetch-data", fetchData)

    if err != nil {

        return err

    }



    fmt.Printf("Goroutine %d: result: %v, shared: %v\n", id, v, shared)

    return nil

}



func main() {

    var g singleflight.Group

  wg.Add(3)



    // 2 goroutines fetch the data

    go fetchDataWrapperWithForget(&g, 0, false)

    go fetchDataWrapperWithForget(&g, 1, false)



    // Wait a bit and launch 1 more goroutine

    // Ensures goroutines 0, 1, and 2 overlap

    time.Sleep(10 * time.Millisecond)

    go fetchDataWrapperWithForget(&g, 2, true)



    wg.Wait()

    fmt.Printf("Function was called %d times\n", callCount.Load())

}



// Output:

// Goroutine 0: result: 55, shared: true

// Goroutine 1: result: 55, shared: true

// Goroutine 2: result: 73, shared: false

// Function was called 2 times

Goroutine 0 and Goroutine 1 both call Do() with the same key ("key-fetch-data"), and their requests get combined into one execution and the result is shared between the two goroutines.

Goroutine 2, on the other hand, calls Forget() before running Do(). This clears out any previous result tied to "key-fetch-data", so it triggers a new execution of the function.

To sum up, while singleflight is useful, it can still have some edge cases, for example:

• If the first goroutine gets blocked for too long, all the others waiting on it will also be stuck. In such cases, using a timeout context or a select statement with a timeout can be a better option.


• If the first request returns an error or panics, that same error or panic will propagate to all the other goroutines waiting for the result.

If you have noticed all the issues we've discussed, let's dive into the next section to discuss how singleflight actually works under the hood.

How Singleflight Works

From using singleflight, you might already have a basic idea of how it works internally, the whole implementation of singleflight is only about 150 lines of code.

Basically, every unique key gets a struct that manages its execution. If a goroutine calls Do() and finds that the key already exists, that call will be blocked until the first execution finishes, and here is the structure:

CODE Kopieren

type Group struct {

    mu sync.Mutex       // protects the map m

    m  map[string]*call // maps keys to calls; lazily initialized

}



type call struct {

    wg    sync.WaitGroup   // waits for the function execution

    val   interface{}      // result of the function call

    err   error            // error from the function call

    dups  int              // number of duplicate callers

    chans []chan<- Result  // channels to receive the result

}

Two sync primitives are used here:

• Group mutex (g.mu): This mutex protects the entire map of keys, not one lock per key, it makes sure adding or removing keys is thread-safe.


• WaitGroup (g.call.wg): The WaitGroup is used to wait for the first goroutine associated with a specific key to finish its work.

We'll focus on the group.Do() method here since the other method, group.DoChan(), works in a similar way. The group.Forget() method is also simple as it just removes the key from the map.

When you call group.Do(), the first thing it does is lock the entire map of calls (g.mu).

"Isn't that bad for performance?"

Yeah, it might not be ideal for performance in every case (always good to benchmark first) as singleflight locks the entire keys. If you're aiming for better performance or working at a high scale, a good approach is to shard or distribute the keys. Instead of using just one singleflight group, you can spread the load across multiple groups, kind of like doing "multiflight" instead

For reference, check out this repo:

Handling of panic and runtime.Goexit() in singleflight

That's why recovered = true gets set outside the defer containing recover(), it only gets executed in two cases: when the function completes normally or when a panic is recovered, but not when runtime.Goexit() is called.

Moving forward, we'll discuss how each case is handled.

CODE Kopieren

func (g *Group) doCall(c *call, key string, fn func() (interface{}, error)) {

    ...



    defer func() {

        ...



        // Lock and remove the call from the map

        g.mu.Lock()

        defer g.mu.Unlock()

        c.wg.Done()

        if g.m[key] == c {

            delete(g.m, key)

        }



        if e, ok := c.err.(*panicError); ok {

            if len(c.chans) > 0 {

                go panic(e)

                select {} // Keep this goroutine around so that it will appear in the crash dump.

            } else {

                panic(e)

            }

        } else if c.err == errGoexit {

            // Already in the process of goexit, no need to call again

        } else {

            // Normal return

            for _, ch := range c.chans {

                ch <- Result{c.val, c.err, c.dups > 0}

            }

        }

    }()

    ...

}

If the task panics during execution, the panic is caught and saved in c.err as a panicError, which holds both the panic value and the stack trace. singleflight catches the panic to clean up gracefully, but it doesn't swallow it, it rethrows the panic after handling its state.

That means the panic will happen in the goroutine that's executing the task (the first one to kick off the operation), and all the other goroutines waiting for the result will also panic.

Since this panic happens in the developer's code, it's on us to deal with it properly.

Now, there's still a special case we need to consider: when other goroutines are using the group.DoChan() method and waiting on a result via a channel. In this case, singleflight can't panic in those goroutines. Instead, it does what's called an unrecoverable panic (go panic(e)), which makes our application crash.

Finally, if the task called runtime.Goexit(), there's no need to take any further action because the goroutine is already in the process of shutting down, and we just let that happen without interfering.

And that's pretty much it, nothing too complicated except for the special cases we've discussed.

Stay Connected

Hi, I'm Phuong Le, a software engineer at VictoriaMetrics. The writing style above focuses on clarity and simplicity, explaining concepts in a way that's easy to understand, even if it's not always perfectly aligned with academic precision.

If you spot anything that's outdated or if you have questions, don't hesitate to reach out. You can drop me a DM on

. It's a fast, open-source, and cost-saving way to keep an eye on your infrastructure.

And we're Gophers, enthusiasts who love researching, experimenting, and sharing knowledge about Go and its ecosystem.

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen