Skip to main content

Handling Complex Configuration Types

The Flyte configuration system provides specialized wrappers and patterns for handling non-primitive types such as durations, URLs, network ports, and regular expressions. These types bridge the gap between Go's internal representations and the formats used in JSON/YAML configuration files or command-line flags.

Using Built-in Type Wrappers​

Flyte defines several wrapper types in flytestdlib/config that implement json.Marshaler, json.Unmarshaler, and the pflag.Value interface.

Duration​

The config.Duration type wraps time.Duration. It can be unmarshaled from a string (e.g., "20s", "1h") or a float64 representing nanoseconds.

type MyConfig struct {
    Timeout config.Duration `json:"timeout"`
}

// In YAML:
// timeout: 30s

URL​

The config.URL type wraps net.URL. It allows you to treat URL strings as structured objects within your configuration.

type MyConfig struct {
    BaseURL config.URL `json:"base-url"`
}

// In YAML:
// base-url: https://api.flyte.org

Port​

The config.Port type handles network port numbers. It validates that the value is within the valid range (0-65535) and can be parsed from either a string or a number.

type MyConfig struct {
    ListenPort config.Port `json:"listen-port"`
}

// In YAML:
// listen-port: 8080

Regexp​

The config.Regexp type wraps regexp.Regexp, allowing you to define and compile regular expressions directly from configuration strings.

type MyConfig struct {
    Pattern config.Regexp `json:"pattern"`
}

// In YAML:
// pattern: "^[a-z]+$"

Working with Collections​

Flyte supports standard Go slices and maps for configuration. These are typically handled via JSON tags on the struct fields.

Lists and Slices​

You can use slices for ordered lists of values.

type MyConfig struct {
    Tags []string `json:"tags"`
}

// In YAML:
// tags:
//   - production
//   - internal

Maps and Nested Maps​

Maps are supported for key-value pairs. You can also nest maps for more complex structures, as seen in flytestdlib/config/tests/accessor_test.go.

type Item struct {
    ID   string `json:"id"`
    Name string `json:"name"`
}

type ItemMap struct {
    Items    map[string]Item            `json:"items"`
    ItemsMap map[string]map[string]Item `json:"itemsMap"`
}

PFlag Integration​

To expose complex types as command-line flags, implement the GetPFlagSet method from the PFlagProvider interface. The built-in wrappers like config.Duration and config.URL provide a Set(string) error method that makes them compatible with pflag.

The following example from flytestdlib/config/tests/types_test.go demonstrates how to register these flags:

func (OtherComponentConfig) GetPFlagSet(prefix string) *pflag.FlagSet {
    cmdFlags := pflag.NewFlagSet("MyComponentConfig", pflag.ExitOnError)
    
    // Standard types
    cmdFlags.String(fmt.Sprintf("%v%v", prefix, "string-value"), "hello world", "description")
    cmdFlags.Int(fmt.Sprintf("%v%v", prefix, "int-val"), 4, "description")
    
    // Complex types via string flags
    cmdFlags.String(fmt.Sprintf("%v%v", prefix, "duration-value"), "20s", "timeout duration")
    cmdFlags.String(fmt.Sprintf("%v%v", prefix, "url-value"), "http://flyte.org", "service url")
    
    // Slices
    cmdFlags.StringSlice(fmt.Sprintf("%v%v", prefix, "strings-def"), []string{"default"}, "list of strings")
    
    return cmdFlags
}

Custom Types and Enums​

You can use custom types (like enums) by defining a named type and implementing JSON marshaling if necessary. For simple enums based on integers, standard JSON marshaling often suffices.

type NamedType int

const (
    NamedTypeA NamedType = iota
    NamedTypeB
)

type MyConfig struct {
    Type NamedType `json:"namedType"`
}

Important Considerations​

  • Pointer Requirement: When calling config.RegisterSection, you must pass a pointer to your configuration struct. The system uses reflection to verify this and will return an error if a value is passed instead.
  • Case Insensitivity: Section keys are case-insensitive. Flyte converts all keys to lowercase internally during registration and lookup.
  • Port Type Identity: Note that config.Port.Type() returns "URL". This is the current behavior in flytestdlib/config/port.go and should be accounted for if you are programmatically inspecting flag types.
  • Duration Precision: When unmarshaling config.Duration from a numeric value (float64), the system treats the number as nanoseconds.