Neutralization

Understanding the signal-based neutralization protocol.

State Machine

Overview

Neutralization solves the stalled thread problem: what happens when a thread stays pinned for a long time? Without intervention, it blocks reclamation and memory accumulates unboundedly.

DEBRA+ uses POSIX signals to neutralize stalled threads, allowing epoch advancement and bounded memory.

The Stalled Thread Problem

If a thread stays pinned at an old epoch:

• Reclamation is blocked indefinitely
• Memory accumulates in limbo bags
• System may run out of memory

How Neutralization Works

1. Detection: Reclamation detects thread pinned at old epoch
2. Signal: Send SIGUSR1 to stalled thread
3. Handler: Signal handler sets neutralization flag
4. Check: Thread checks flag when it unpins
5. Acknowledge: Thread acknowledges neutralization
6. Advance: Safe epoch can now advance past stalled thread

Triggering Neutralization

The simple convenience API allows scanning for and signaling stalled threads:

let signalsSent = neutralizeStalled(manager)

This returns the number of signals sent to stalled threads.

The epochsBeforeNeutralize parameter controls how far behind a thread must be before neutralization (default is 2 epochs):

let signalsSent = neutralizeStalled(manager, epochsBeforeNeutralize = 3)

For advanced use cases, the low-level typestate API provides explicit control over each step:

let complete = scanStart(addr manager)
  .loadEpoch(epochsBeforeNeutralize = 2)
  .scanAndSignal()
let count = complete.extractSignalCount()

Neutralization Handling Example

# examples/neutralization_handling.nim
## Neutralization: handling signal-based interruption of stalled threads.

import debra
import std/atomics

when isMainModule:
  var manager = initDebraManager[4](../../examples)
  setGlobalManager(addr manager)

  let handle = registerThread(manager)

  # Wrapper that handles neutralization automatically
  proc withPinnedSection(body: proc()) =
    let u = unpinned(handle)
    let pinned = u.pin()

    body()

    let unpinResult = pinned.unpin()
    case unpinResult.kind:
    of uUnpinned:
      discard
    of uNeutralized:
      # Acknowledge and the caller can retry if needed
      discard unpinResult.neutralized.acknowledge()

  # Use the wrapper for clean critical sections
  withPinnedSection(proc() =
    echo "Working in critical section..."
  )

  # Manual handling with retry logic
  block manualRetry:
    var attempts = 0
    var done = false

    while not done and attempts < 3:
      inc attempts
      let u = unpinned(handle)
      let pinned = u.pin()

      # Simulate work
      echo "Attempt ", attempts

      # Simulate neutralization on first attempt
      if attempts == 1:
        manager.threads[handle.idx].neutralized.store(true, moRelease)

      let unpinResult = pinned.unpin()
      case unpinResult.kind:
      of uUnpinned:
        done = true
        echo "Completed on attempt ", attempts
      of uNeutralized:
        echo "Neutralized on attempt ", attempts, " - will retry"
        discard unpinResult.neutralized.acknowledge()

  echo "Neutralization handling example completed successfully"

View full source

Neutralization Semantics

Neutralization is advisory, not forceful:

• Thread is not stopped mid-computation
• Thread is not killed or interrupted
• Thread continues running normally
• Flag is checked on next unpin

Safety Guarantees

• Neutralization cannot corrupt memory
• Thread completes current operation safely
• Lock-free operations remain atomic
• Memory ordering is preserved

Limitations

Neutralization cannot:

• Stop a thread in an infinite loop
• Interrupt blocking system calls
• Force a thread to check the flag
• Guarantee timely response

Configuration

Neutralization Threshold

How far behind before neutralizing? Default is 2 epochs.

Lower threshold = more aggressive neutralization, better memory bounds, more interruptions.

Higher threshold = more lenient on long operations, higher memory usage, fewer interruptions.

Signal Choice

DEBRA+ uses SIGUSR1 by default. Requirements:

• Must not be used by application
• Must not have default handler
• Must be available on platform

Platform Considerations

Neutralization works on POSIX systems (Linux, macOS, BSD). Windows does not support pthread_kill and would require alternative approaches.

Best Practices

Keep Critical Sections Short

Avoid neutralization by keeping pin/unpin sections minimal:

# GOOD - short critical section
let pinned = handle.pin()
let node = queue.dequeue()
discard pinned.unpin()
processNode(node)  # Outside critical section

# BAD - long critical section
let pinned = handle.pin()
let node = queue.dequeue()
processNode(node)  # Inside critical section!
discard pinned.unpin()

Handle Neutralization Gracefully

Don't treat neutralization as an error - retry the operation.

Monitor Neutralization Rate

Frequent neutralization indicates:

• Critical sections are too long
• Operations are blocking
• Threshold is too aggressive

Next Steps

• Learn about integration patterns
• Review API reference