Configuration Management Framework

Flyte provides a strongly-typed configuration management framework that centralizes how application settings are defined, registered, and accessed. It supports loading configurations from multiple sources—including YAML/TOML/JSON files, environment variables, and command-line flags—with a clear precedence order and support for runtime updates.

Defining Configuration Sections

In Flyte, configurations are organized into Sections. A section is a logical grouping of related settings represented by a Go struct. To define a section, you create a struct and use struct tags to map fields to configuration keys and command-line flags.

The framework uses the json tag to map configuration file keys and the pflag tag to define command-line flag descriptions.

type Config struct {
    IncludeSourceCode bool   `json:"show-source" pflag:",Includes source code location in logs."`
    Mute              bool   `json:"mute" pflag:",Mutes all logs regardless of severity."`
    Level             int    `json:"level" pflag:",Sets the minimum logging level."`
    Formatter         struct {
        Type string `json:"type" pflag:",Sets logging format type."`
    } `json:"formatter" pflag:",Sets logging format."`
}

PFlag Integration

Flyte includes a pflags code generation tool that automatically implements the PFlagProvider interface for your configuration structs. This allows the framework to automatically expose your configuration fields as command-line arguments.

You typically include a go:generate directive in your configuration file:

//go:generate pflags Config --default-var defaultConfig

Registering Sections

Once a configuration struct is defined, it must be registered with the global rootSection before the application loads its configuration. Registration is usually performed in a package's init() function or as a package-level variable.

Basic Registration

Use MustRegisterSection to register a section. The key provided is case-insensitive and must be unique.

const configSectionKey = "Logger"

var (
    defaultConfig = &Config{
        Level: 3, // WarnLevel
    }

    configSection = config.MustRegisterSection(configSectionKey, defaultConfig)
)

Registration with Update Handlers

If your component needs to react to configuration changes at runtime (e.g., when a config file is updated and reloaded), use MustRegisterSectionWithUpdates.

var configSection = config.MustRegisterSectionWithUpdates(
    configSectionKey, 
    defaultConfig, 
    func(ctx context.Context, newValue config.Config) {
        // newValue is the updated *Config pointer
        onConfigUpdated(*newValue.(*Config))
    },
)

Internally, MustRegisterSectionWithUpdates stores the SectionUpdated callback in the section struct (defined in flytestdlib/config/section.go). When the Accessor refreshes the configuration, it compares the new values using reflect.DeepEqual and invokes the handler if changes are detected.

Accessing Configuration

To load and access the registered configurations, you use an Accessor. The primary implementation in Flyte is based on Viper.

Initializing the Accessor

The viper.NewAccessor function creates an accessor configured with config.Options. You can specify SearchPaths to look for configuration files and enable StrictMode to fail if the configuration contains keys that haven't been registered.

import (
    "github.com/flyteorg/flyte/v2/flytestdlib/config"
    "github.com/flyteorg/flyte/v2/flytestdlib/config/viper"
)

func main() {
    accessor := viper.NewAccessor(config.Options{
        SearchPaths: []string{"/etc/flyte/config.yaml", "config.yaml"},
        StrictMode:  true,
    })

    // Load configuration from files and environment variables
    err := accessor.UpdateConfig(context.Background())
    if err != nil {
        panic(err)
    }
}

Retrieving Values

After UpdateConfig is called, you can retrieve the typed configuration from the registered section:

func GetConfig() *Config {
    return configSection.GetConfig().(*Config)
}

Command-Line Interface Integration

Flyte provides built-in support for integrating configuration management into Cobra-based CLIs.

Binding PFlags

To allow users to override configuration via command-line flags, you must initialize the flags on your cobra.Command:

func (v *viperAccessor) InitializePflags(cmdFlags *pflag.FlagSet) {
    // ... internal logic to add section flags to the flag set ...
    err := v.viper.BindPFlags(cmdFlags)
}

The Config Subcommand

You can add a comprehensive config subcommand to your application using viper.GetConfigCommand(). This provides three useful utilities:

• validate: Checks if the current configuration files are valid and match registered sections.
• discover: Shows which configuration files were found in the search paths.
• docs: Generates documentation for all registered configuration sections in RST format.

rootCmd := &cobra.Command{Use: "flyte"}
rootCmd.AddCommand(viper.GetConfigCommand())

Advanced Features

Nested Sections

Sections can be nested to create hierarchical configuration structures. This is useful for complex applications where different modules have their own sub-configurations.

// Register root section
parentSection := config.MustRegisterSection("parent", &ParentConfig{})

// Register nested section
childSection := parentSection.MustRegisterSection("child", &ChildConfig{})

In a YAML configuration file, this would be represented as:

parent:
  child:
    key: value

Strict Mode Validation

When StrictMode is enabled in config.Options, the viperAccessor (in flytestdlib/config/viper/viper.go) validates that every key found in the configuration files or environment variables corresponds to a registered section. If an unknown key is encountered, UpdateConfig returns config.ErrStrictModeValidation.

Case Sensitivity Workaround

Viper is natively case-insensitive for map keys. Flyte provides a sliceToMapHook in flytestdlib/config/viper/viper.go to support case-sensitive maps. If you need case sensitivity, represent the map as a slice of single-key maps in your YAML:

# Instead of:
# my-map:
#   CaseSensitiveKey: value

# Use:
my-map:
  - CaseSensitiveKey: value

The sliceToMapHook automatically converts this slice back into a Go map during decoding.