Skip to main content

Workflow Execution and Event Flow

This sequence diagram traces the lifecycle of a workflow run from its initial creation through execution and event logging.

The flow begins with a runs.service receiving a CreateRun request. It persists the initial run state in the database and then enqueues the root action via the ActionsService. The ActionsClient (part of the Actions service) creates a TaskAction Custom Resource (CR) in Kubernetes.

The executor (specifically the TaskActionReconciler) watches for these TaskAction resources. During reconciliation, it invokes a plugin (e.g., flyteplugins.k8s) to manage the actual execution (like creating a Pod). As the execution progresses, the executor reports events back to the events.publisher (implemented as EventsProxyService), which forwards them to the runs.service to be recorded in the database.

Simultaneously, the ActionsClient watches for status updates on the TaskAction CR and synchronizes these changes back to the runs.service, ensuring the database reflects the current phase of execution.

Key Architectural Findings:

  • The runs.service acts as the primary entry point for creating runs and managing their state in the database.
  • The ActionsService and its ActionsClient bridge the gap between the gRPC API and Kubernetes Custom Resources (TaskAction).
  • The executor is a Kubernetes controller that reconciles TaskAction resources using specialized plugins like flyteplugins.k8s.
  • Event logging is handled by the EventsProxyService, which synchronously forwards execution events from the executor to the run service.
  • State synchronization is bidirectional: the executor updates the TaskAction status, and the ActionsClient watches those updates to update the central database.
Loading diagram...