TL;DR — Go’s scheduler uses a work-stealing algorithm built on the G-M-P (Goroutine, Machine, Processor) model to distribute goroutines across OS threads with minimal contention. When a thread runs out of local work, it steals half the run queue from another busy processor — a strategy that achieves near-optimal load balancing while keeping synchronization overhead low. Understanding this model is essential for writing high-performance concurrent Go applications.

Introduction

Go’s concurrency story is legendary. Millions of goroutines executing simultaneously, scheduled with sub-microsecond latency, and balanced across available CPU cores — all without a single pthread_mutex in the critical path. Behind this lies one of the most elegant work-stealing schedulers ever shipped in a production runtime.

Unlike operating system threads, which are expensive to create and context-switch, goroutines are lightweight: they start at ~2 KB of stack and can grow dynamically. But lightweight goroutines are only useful if the scheduler beneath them is equally efficient. Go’s scheduler must decide which goroutine runs next, on which OS thread, and how to keep all cores busy without introducing lock contention that would negate the benefits of concurrency.

This post dissects that scheduler — the G-M-P model, the global run queue, local run queues, the work-stealing loop, and the mechanisms that make load balancing feel invisible.

The G-M-P Model: Three Layers of Abstraction

Go’s scheduler is built on a three-tier abstraction: Goroutines, Machine threads, and Processors. Understanding how these three interact is the key to understanding everything else.

Goroutines (G)

A goroutine is the unit of work. When you write:

go func() {
    // do work
}()

The runtime allocates a g struct — typically 488 bytes on a 64-bit system — containing the stack pointer, program counter, registers, and scheduling state. Goroutines are queued into run queues and dispatched to OS threads for execution.

Machine Threads (M)

An M represents an OS thread. The runtime manages a pool of these threads. Crucially, an M is not permanently bound to a single goroutine. The scheduler can detach an M from a blocked goroutine and pick up another ready one — enabling cooperative-style multitasking on top of preemptive OS threads.

Logical Processors (P)

A P is the scheduling context that owns a local run queue. The number of Ps equals GOMAXPROCS, which defaults to the number of available CPU cores. Every M must acquire a P before executing goroutines. This design localizes run queue access: most scheduling decisions happen on the P’s local queue without touching any global lock.

┌─────────────────────────────────────────────────┐
│                   P (Processor)                  │
│  ┌──────────────────────────────────────────┐    │
│  │         Local Run Queue (LRQ)            │    │
│  │  [G1] → [G2] → [G3] → [G4] → [G5]     │    │
│  └──────────────────────────────────────────┘    │
│                                                  │
│  M ──→ executes G from LRQ                       │
└──────────────────────────────────────────────────┘

The P acts as a scheduling token. An M holding a P can pull goroutines from the local queue. An M without a P is parked in a global idle list, waiting for work.

Run Queues: Where Goroutines Wait

Go maintains three distinct run queues, each serving a different purpose in the scheduling lifecycle.

The Global Run Queue (GRQ)

The global run queue is a FIFO queue protected by a single spinlock (sched.lock). It serves two purposes: it holds goroutines that have been pushed from local queues during load balancing, and it acts as a fallback when a local queue is empty.

The global queue is deliberately underused. The scheduler only checks it periodically — roughly every 61 scheduling rounds — to avoid contention on the global lock. This design choice is critical: if every goroutine dispatch required acquiring the global lock, the scheduler would become a bottleneck on multi-core machines.

The Local Run Queue (LRQ)

Each P owns a lock-free local run queue implemented as a double-ended queue (deque). Goroutines created by a goroutine already running on that P are pushed to the tail of the LRQ. The scheduler pops from the head. This LIFO/FIFO hybrid behavior has a practical benefit: recently spawned goroutines tend to share cache locality with their parent, so the LIFO pop keeps hot data in the CPU cache.

The LRQ is lock-free because only the owning P accesses it. The P’s lock is held only during P handoff operations (when transferring the P to another M), not during normal goroutine dispatch.

The Network Poller

Network I/O is handled by a separate set of goroutines managed by the netpoller. When a goroutine performs a blocking network operation, the scheduler parks it and hands the P to another goroutine. The netpoller wakes the goroutine when the I/O completes, placing it back on a run queue. This is how Go achieves millions of concurrent network connections without an OS thread per connection.

The Work-Stealing Algorithm

The core of Go’s load balancing strategy is work stealing. When an M exhausts its local run queue, it doesn’t immediately grab a lock and search the global queue. Instead, it follows a precise sequence:

  1. Pop from the local run queue — LIFO pop for cache locality.
  2. Check the global run queue — roughly every 61 scheduling cycles.
  3. Steal from another P’s local run queue — take half the work from the tail of a randomly chosen victim’s deque.
  4. Park and wait — if no work is found anywhere, the M parks itself on the idle list and sleeps.

This sequence is implemented in the findrunnable function in the Go runtime source (src/runtime/proc.go). The work-stealing step is the critical optimization: by stealing from the tail of a victim’s deque, the thief takes the oldest, least cache-hot work, while the victim continues working on the fresher, more cache-resident goroutines at the head.

// Simplified pseudocode of the work-stealing loop
func findrunnable(_g *g) (gp *g, inheritTime bool) {
    // 1. Try local run queue
    if gp := runqget(_g); gp != nil {
        return gp, false
    }

    // 2. Check global run queue periodically
    if _g.p.ptr().schedtick%61 == 0 {
        if gp := globrunqget(_g.p, 1); gp != nil {
            return gp, false
        }
    }

    // 3. Steal from another P
    for i := 0; i < sched.npidle+int32(sched.nmspinning); i++ {
        if gp := stealWork(_g); gp != nil {
            return gp, false
        }
    }

    // 4. Park and wait for work
    park(_g)
    return findrunnable(_g)
}

Why Steal Half?

The scheduler steals approximately half the victim’s run queue, not just one goroutine. This batching reduces the frequency of stealing operations and amortizes the cost of the atomic compare-and-swap (CAS) operations required to synchronize the deque. Research by Blumofe and Leiserson (1995) showed that stealing half the work minimizes expected completion time under a wide range of task graphs — a theoretical guarantee that Go’s scheduler leverages in practice.

The Random Victim Selection

Go selects a victim P by iterating through a randomized order. The findrunnable function uses a per-P “steal generation” counter to detect when the set of runnable goroutines has changed, ensuring that a thief doesn’t repeatedly probe the same empty queue. This avoids the thundering herd problem where all idle threads simultaneously target the same busy P.

Goroutine Lifecycle and Preemption

A goroutine transitions through several states during its lifetime:

┌──────────────┐    go func()    ┌──────────────┐
│  Idle (GC)   │ ──────────────▶ │  Runnable    │
└──────────────┘                 └──────┬───────┘
                                        │ dispatched
                                        ▼
                               ┌──────────────┐
                               │  Running     │
                               └──────┬───────┘
                                      │ blocking I/O
                                      ▼
                               ┌──────────────┐
                               │  Waiting     │ ───── netpoller wake
                               └──────────────┘

Cooperative vs. Preemptive Scheduling

Early versions of Go used purely cooperative scheduling: a goroutine would run until it explicitly yielded (via a function call, channel operation, or memory allocation). This led to starvation bugs where CPU-bound goroutines would never yield, starving other goroutines on the same P.

Go 1.14 introduced async preemption. The scheduler inserts a check point at function call entries and at loop back-edges. When the preemption flag is set, the running goroutine is suspended and placed back on the run queue, allowing the scheduler to pick a different goroutine.

The preemption mechanism works by setting a flag in the goroutine’s g struct. On the next function call, the runtime checks this flag and triggers a g->m handoff. This is implemented using a signal-based approach on architectures where precise program counters are difficult to intercept.

The Scheduling Latency Goal

Go targets a 10-millisecond scheduling latency: every goroutine should get a chance to run within 10 ms. The sysmon (system monitor) thread enforces this by periodically checking if a P has been running a single goroutine for too long and triggering a forced yield.

Load Balancing in Practice

Work Stealing vs. Work Sharing

Traditional load balancing uses work sharing: a central dispatcher assigns tasks to workers. Go uses work stealing, where workers pull tasks from each other. The difference matters at scale:

  • Work sharing requires a central dispatcher, creating a single point of contention.
  • Work stealing is decentralized — each P manages its own queue, and stealing happens only when necessary.

This decentralized approach means Go’s scheduler scales nearly linearly with the number of cores. The overhead of stealing is O(1) amortized, and the probability of contention decreases as the number of Ps increases (more victims to choose from).

The Spinning M Problem

Go distinguishes between spinning and non-spinning Ms. A spinning M is one that is actively looking for work and hasn’t parked. The scheduler limits the number of spinning Ms to the number of idle Ps (sched.npidle). This prevents a scenario where multiple threads are spinning simultaneously, burning CPU cycles while no work exists.

When a goroutine becomes runnable (e.g., a channel send completes), the scheduler tries to wake a spinning M rather than creating a new OS thread. This is handled in wakep(), which sets the P’s runqwait flag and signals a parked M to resume.

// wakep attempts to wake or create an M to service a P with pending work
func wakep() {
    if !sched.npidle.empty() {
        // There's an idle M; signal it to start spinning
        notewakeup(&sched.npidle.head.ptr().waitnote)
    } else if sched.nmspinning < sched.npidle.len() {
        // Need to create a new M
        newm(nil, _g.p)
    }
}

GC and the Scheduler

The Go garbage collector is deeply integrated with the scheduler. During concurrent marking, the GC must ensure that goroutines don’t run indefinitely without being preempted — otherwise, the marking phase would never finish. The scheduler cooperates with the GC by forcing preemption at safe points, ensuring that every goroutine is paused at least once during a GC cycle.

This interaction is why Go 1.14’s async preemption was a game-changer for GC pause times: the GC no longer needed to rely solely on cooperative yield points, which could be far apart in CPU-bound code.

Architecture Patterns in Production

Pattern 1: Bounding Goroutine Sprawl

Understanding the scheduler reveals why unbounded goroutine creation is dangerous. Each goroutine consumes memory (stack + g struct), and each new goroutine adds work to the local run queue. If a single P receives a burst of millions of goroutines, the LRQ becomes a bottleneck, and stealing can’t help because there’s no work on other Ps.

The pattern: use worker pools or semaphore patterns to bound concurrency.

// Worker pool pattern: bounded concurrency with explicit control
func workerPool(jobs <-chan Job, workers int) {
    var wg sync.WaitGroup
    for i := 0; i < workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                process(job)
            }
        }()
    }
    wg.Wait()
}

Pattern 2: Channel-Based Synchronization and Scheduler Interaction

Channels are not just communication primitives — they are scheduler yield points. When a goroutine sends on a channel with no receiver, it parks and releases its P. When a receiver arrives, the scheduler wakes the sender and transfers the P. This implicit yielding is what makes channels a natural mechanism for backpressure.

Pattern 3: Avoiding runtime.Gosched() Abuse

runtime.Gosched() explicitly yields the processor, placing the current goroutine at the back of the global run queue. While occasionally useful for fairness, relying on it for load balancing is a code smell — it indicates that the scheduler’s natural mechanisms aren’t sufficient, which usually means the workload distribution is fundamentally unbalanced.

Performance Characteristics and Benchmarks

Go’s scheduler achieves remarkable throughput under load. In benchmarks with 1 million goroutines performing simple counter increments, the scheduler maintains sub-100-nanosecond dispatch latency on modern hardware. The work-stealing overhead adds roughly 20–50 nanoseconds per steal operation (a CAS on the victim’s deque), which is negligible compared to the cost of a goroutine context switch (~300 nanoseconds on modern CPUs).

The key performance metrics:

  • Dispatch latency: ~50–100 ns for local queue pops
  • Steal latency: ~20–50 ns for CAS-based deque steal
  • Context switch cost: ~300 ns (save/restore registers + TLB invalidation)
  • Maximum goroutines per P: effectively unbounded, but practical limits are set by memory
  • Scaling efficiency: ~90–95% throughput scaling up to 64+ cores

Key Takeaways

  • The G-M-P model decouples goroutines from OS threads, allowing the scheduler to manage millions of lightweight goroutines with a modest pool of OS threads.
  • Work stealing is decentralized and lock-free for the common case: most goroutine dispatch happens on the local run queue without any synchronization.
  • Stealing half the victim’s queue is not arbitrary — it’s grounded in the Blumofe-Leiserson theoretical framework and reduces amortized synchronization cost.
  • The global run queue is a fallback, not a primary path: checking it only every 61 scheduling cycles prevents lock contention from becoming the bottleneck.
  • Async preemption (Go 1.14+) was essential for both fairness and GC performance, preventing CPU-bound goroutines from monopolizing a P.
  • Understanding the scheduler helps you write better Go: bounded goroutine creation, channel-based backpressure, and avoiding forced yields are all patterns that respect the scheduler’s design.

Further Reading