Building Services with the App Framework

The Flyte application framework provides a standardized skeleton for building services. It manages the lifecycle of HTTP servers, background workers, and health checks, while providing a consistent way to initialize shared resources like Kubernetes clients and database connections.

By following this guide, you will build a service that exposes an HTTP API, runs a background reconciler, and performs custom readiness checks.

Prerequisites

To build a service with the Flyte framework, your project must include the following dependencies:

• github.com/flyteorg/flyte/v2/flytestdlib/app
• github.com/flyteorg/flyte/v2/flytestdlib/logger
• github.com/spf13/cobra

Step 1: Define the Application Entry Point

Every Flyte service starts with an app.App instance. This struct defines the service's identity and the Setup function where resource initialization occurs.

Create a main.go file:

package main

import (
	"context"
	"os"

	"github.com/flyteorg/flyte/v2/flytestdlib/app"
)

func main() {
	a := &app.App{
		Name:  "my-service",
		Short: "A custom Flyte service",
		Setup: setup,
	}

	if err := a.Run(); err != nil {
		os.Exit(1)
	}
}

The Run() method automatically handles configuration loading (via --config), logging initialization, and signal handling for graceful shutdown.

Step 2: Initialize Shared Resources

The Setup function receives a SetupContext, which acts as a registry for your service's components. You use this phase to initialize databases and Kubernetes clients.

func setup(ctx context.Context, sc *app.SetupContext) error {
	// Configure the server address
	sc.Host = "0.0.0.0"
	sc.Port = 8080
	sc.Namespace = "flyte"

	// Initialize Kubernetes client using the built-in helper
	k8sClient, restConfig, err := app.InitKubernetesClient(ctx, app.K8sConfig{
		Namespace: sc.Namespace,
		QPS:       100,
		Burst:     200,
	}, nil)
	if err != nil {
		return err
	}
	
	sc.K8sClient = k8sClient
	sc.K8sConfig = restConfig

	return nil
}

app.InitKubernetesClient automatically attempts to find credentials in-cluster before falling back to the local ~/.kube/config.

Step 3: Register HTTP Handlers

The SetupContext provides a standard *http.ServeMux via the Mux field. Flyte services typically use ConnectRPC for their APIs.

func setup(ctx context.Context, sc *app.SetupContext) error {
	// ... previous initialization ...

	// Example: Registering a ConnectRPC handler
	// path, handler := myapi.NewMyServiceHandler(implementation)
	// sc.Mux.Handle(path, handler)

	// Standard HTTP handler
	sc.Mux.HandleFunc("/api/v1/version", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("v1.0.0"))
	})

	return nil
}

The framework automatically wraps your handlers with requestGzipDecompressMiddleware to handle gzipped request bodies from various Flyte SDKs.

Step 4: Add Background Workers

If your service needs to run long-lived tasks (like a Kubernetes controller or a periodic cleanup task), register them as workers. Workers are managed by the application lifecycle and receive a context that is cancelled during shutdown.

func setup(ctx context.Context, sc *app.SetupContext) error {
	// ...

	sc.AddWorker("cleanup-task", func(ctx context.Context) error {
		ticker := time.NewTicker(1 * time.Minute)
		for {
			select {
			case <-ctx.Done():
				return nil
			case <-ticker.C:
				// Perform cleanup logic
			}
		}
	})

	return nil
}

Workers must block until the provided ctx is cancelled. If a worker returns an error, the entire application will initiate a graceful shutdown.

Step 5: Implement Readiness Checks

The framework provides /healthz (always returns 200) and /readyz endpoints. You can add custom logic to /readyz to ensure your service is only marked ready when its dependencies are available.

func setup(ctx context.Context, sc *app.SetupContext) error {
	// ...

	sc.AddReadyCheck(func(r *http.Request) error {
		if sc.DB != nil {
			if err := sc.DB.PingContext(r.Context()); err != nil {
				return err // Returns 503 Service Unavailable
			}
		}
		return nil // Returns 200 OK
	})

	return nil
}

Complete Service Example

Combining these steps, your service setup in flytestdlib/app looks like this:

func setup(ctx context.Context, sc *app.SetupContext) error {
	// 1. Resource Initialization
	k8sClient, _, err := app.InitKubernetesClient(ctx, app.K8sConfig{Namespace: "flyte"}, nil)
	if err != nil {
		return err
	}
	sc.K8sClient = k8sClient

	// 2. HTTP Handlers
	sc.Mux.HandleFunc("/api/v1/status", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("active"))
	})

	// 3. Background Workers
	sc.AddWorker("reconciler", func(ctx context.Context) error {
		// Block until context is cancelled
		<-ctx.Done()
		return nil
	})

	// 4. Health Checks
	sc.AddReadyCheck(func(r *http.Request) error {
		return nil
	})

	return nil
}

Lifecycle and Shutdown

When you call a.Run(), the Flyte framework:

1. Loads configuration from --config (defaulting to ./config.yaml).
2. Initializes the global logger.
3. Executes your Setup function.
4. Starts the HTTP server on sc.Host:sc.Port (unless sc.Port is 0).
5. Starts all registered workers in separate goroutines.
6. Waits for SIGINT or SIGTERM.

Upon receiving a signal, the framework allows 30 seconds for graceful shutdown. It cancels the worker context and shuts down the HTTP server. A second signal during this window will force an immediate exit.