Skip to content

DurableTaskGrpcWorker loop does not exit on close() or stop() #1728

@javier-aliaga

Description

@javier-aliaga

Expected Behavior

Calling close() or stop() on DurableTaskGrpcWorker should promptly terminate the startAndBlock() worker loop, regardless of whether the thread is sleeping during a retry or blocked on the gRPC getWorkItems() streaming call.

Actual Behavior

The startAndBlock() method uses a while (true) loop that never checks for shutdown signals. The loop only exits if an InterruptedException is caught during the 5-second Thread.sleep() retry window.

When close() is called while the thread is blocked on the gRPC stream, managedSidecarChannel.shutdownNow() produces a StatusRuntimeException with Status.Code.CANCELLED. The catch block logs the disconnection but falls through to the retry sleep, causing the loop to continue indefinitely — reconnecting to a channel that has already been shut down.

Additionally, the InterruptedException catch swallows the thread's interrupt status (does not re-set it), breaking the interrupt contract for upstream callers.

Steps to Reproduce the Problem

  1. Create a DurableTaskGrpcWorker and call start() (which runs startAndBlock() in a background thread)
  2. Wait for the worker to connect to the sidecar and begin processing work items via the blocking getWorkItems() stream
  3. Call close() or stop() on the worker
  4. Observe that the worker thread does not terminate — it logs "Durable Task worker has disconnected", sleeps 5 seconds, and retries the connection in a loop

Release Note

RELEASE NOTE: FIX DurableTaskGrpcWorker loop now exits promptly on close()/stop().

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions