26 — Concurrency Patterns

func runServer(ctx context.Context) error {
    // Start workers:
    jobs := make(chan Job, 100)
    go startProducer(ctx, jobs)

    // Process jobs with a worker pool:
    results := runWorkerPool(ctx, jobs, 10)

    // Wait for context cancellation (SIGINT/SIGTERM):
    <-ctx.Done()
    log.Println("shutdown signal received, draining...")

    // The workers' select includes <-ctx.Done() — they exit cleanly.
    // Channels are closed by the goroutines, ranges end, results are collected.

    return nil
}

func main() {
    ctx, stop := signal.NotifyContext(context.Background(),
        syscall.SIGINT, syscall.SIGTERM)
    defer stop()

    if err := runServer(ctx); err != nil &&
        !errors.Is(err, context.Canceled) {
        log.Fatal(err)
    }
    log.Println("server stopped")
}

💡 Tips & Tricks

• Idiom: use a worker pool for bounded concurrency — unbounded go f() in a loop can create millions of goroutines, exhausting memory and overwhelming downstream resources (DB connections, API rate limits). A fixed pool of N workers processes jobs from a channel.
• Idiom: each pipeline stage closes its output channel when done — defer close(out) in the goroutine. The next stage's range exits cleanly when the input channel is closed.
• Idiom: use errgroup.WithContext for concurrent error handling — g.Go returns an error → the context is canceled → other goroutines see <-ctx.Done() and exit. g.Wait() returns the first error.
• Idiom: use a semaphore (golang.org/x/sync/semaphore) for bounded concurrency without a worker pool — each goroutine acquires a token; SetLimit on errgroup is even simpler.
• Idiom: every concurrent goroutine MUST have an exit path (<-ctx.Done()) — without it, the goroutine leaks if it blocks forever. Generators, workers, pipelines all need cancellation.
• Idiom: use signal.NotifyContext for graceful shutdown — the context is canceled on SIGINT/SIGTERM, propagating to all goroutines that select on ctx.Done().

⚠️ Edge Cases & Gotchas

• Unbounded fan-out exhausts resources: for _, item := range huge { go process(item) } creates len(huge) goroutines at once. Use a worker pool or semaphore for bounded concurrency.
• Pipeline deadlock with unbuffered channels: if stage A sends and stage B isn't ready, A blocks. Use buffered channels or ensure stages run concurrently.
• Forgetting to close the output channel: for v := range ch never returns unless ch is closed. The range goroutine leaks. Use defer close(out).
• Generator without exit path leaks: func gen() <-chan int { ch := make(chan int); go func() { for { ch <- compute() } }(); return ch } — leaks if the consumer stops reading. Pass ctx and select on ctx.Done().
• errgroup cancels on first error: g.Go returning an error cancels the context. Other goroutines see <-ctx.Done() and may exit early. This is by design (coordinated failure).
• Worker pool with unbuffered results channel: if no one reads results, workers block on results <- r → deadlock. Buffer the results channel or collect in a separate goroutine.
• Goroutine leak in fan-in: if the merged output isn't fully consumed, the fan-in goroutines block on merged <- v forever. Ensure the consumer drains the output or provide a cancellation path.
• select in worker doesn't guarantee fairness: if multiple cases are ready, select picks randomly. A always-ready case can starve others. Use separate goroutines or explicit scheduling.

🧠 Quick Quiz

func worker(jobs <-chan int, results chan<- int) {
    for j := range jobs {
        results <- j * j
    }
}

func main() {
    jobs := make(chan int, 100)
    results := make(chan int, 100)

    for w := 0; w < 3; w++ {
        go worker(jobs, results)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }
    close(jobs)

    for r := 0; r < 5; r++ {
        fmt.Println(<-results)
    }
}