Vault

Configuration Sources

Composable configuration source system with priority chain.

The source package provides a composable configuration source system where multiple backends (environment variables, in-memory, database) are checked in priority order. The first source to contain a requested key wins.

Architecture

source.Chain (priority-ordered)
  ├── source.Memory   (in-memory, supports Watch)
  ├── source.Env      (environment variables)
  └── source.Database (reads from config.Store)

Sources are stacked in a Chain. When a key is requested, the chain queries each source in order and returns the first hit. This allows environment variables to override database values, or in-memory overrides to take precedence over everything.

Installation

import "github.com/xraph/vault/source"

Source interface

Every source implements this interface:

type Source interface {
    Name() string
    Get(ctx context.Context, key string) (*Value, error)
    List(ctx context.Context, prefix string) ([]*Value, error)
    Watch(ctx context.Context, key string, fn WatchFunc) error
    Close() error
}
MethodDescription
NameReturns a human-readable name for the source (e.g., "memory", "env", "database")
GetRetrieves a value by key. Returns ErrKeyNotFound if not present.
ListReturns all values matching a prefix. Empty prefix returns all values.
WatchRegisters a callback for changes to a key. Sources that do not support watching return nil immediately.
CloseReleases any resources held by the source.

Value

type Value struct {
    Key       string            `json:"key"`
    Raw       string            `json:"raw"`
    Source    string            `json:"source"`
    Version   int64             `json:"version,omitempty"`
    ExpiresAt *time.Time        `json:"expires_at,omitempty"`
    Metadata  map[string]string `json:"metadata,omitempty"`
}
FieldTypeDescription
KeystringThe configuration key
RawstringThe raw string value
SourcestringName of the source that provided this value
Versionint64Version number (from database source)
ExpiresAt*time.TimeOptional expiration timestamp
Metadatamap[string]stringOptional metadata from the source

WatchFunc

type WatchFunc func(ctx context.Context, key string, val *Value)

ErrKeyNotFound

var ErrKeyNotFound = errors.New("source: key not found")

Memory source

An in-memory source useful for testing, dynamic overrides, and programmatic configuration. Supports Watch -- watchers are notified synchronously when Set is called.

Creating a Memory source

func NewMemory() *Memory
mem := source.NewMemory()

Set

Sets a value and immediately notifies all watchers registered for that key.

func (m *Memory) Set(ctx context.Context, key, raw string)
mem.Set(ctx, "feature.enabled", "true")

Get

Retrieves a value by key. Returns a copy (safe for concurrent use).

val, err := mem.Get(ctx, "feature.enabled")
if err != nil {
    // source: key not found
}
fmt.Println(val.Raw) // "true"

List

Returns all values whose keys have the given prefix.

vals, err := mem.List(ctx, "feature.")
for _, v := range vals {
    fmt.Printf("%s = %s\n", v.Key, v.Raw)
}

Watch

Registers a callback for a specific key. Non-blocking -- returns immediately. The callback fires when Set is called for the key.

mem.Watch(ctx, "feature.enabled", func(ctx context.Context, key string, val *source.Value) {
    fmt.Printf("changed: %s = %s\n", key, val.Raw)
})

Properties

PropertyValue
Name()"memory"
Supports WatchYes
Supports ListYes
Thread-safeYes (sync.RWMutex)
Close()No-op

Env source

Reads configuration from environment variables with optional prefix stripping and automatic key normalization.

Creating an Env source

func NewEnv(prefix string) *Env
// Without prefix: looks up "DB_HOST" for key "db-host"
env := source.NewEnv("")

// With prefix: looks up "MYAPP_DB_HOST" for key "db-host"
env := source.NewEnv("MYAPP")

Key normalization

Keys are transformed before looking up the environment variable:

  1. Convert to uppercase.
  2. Replace dashes (-) with underscores (_).
  3. If a prefix is set, prepend PREFIX_.
KeyPrefixEnvironment variable
db-host""DB_HOST
db-host"MYAPP"MYAPP_DB_HOST
cache_ttl""CACHE_TTL
cache_ttl"app"APP_CACHE_TTL

Get

val, err := env.Get(ctx, "db-host")
if err != nil {
    // source: key not found (env var not set or empty)
}
fmt.Println(val.Raw) // value of DB_HOST

Properties

PropertyValue
Name()"env"
Supports WatchNo (returns nil)
Supports ListNo (returns nil, nil)
Thread-safeYes (reads from os.Getenv)
Close()No-op

Database source

Reads configuration from a config.Store backend, converting config entries to source Value structs. Supports polling-based Watch to detect changes.

Creating a Database source

func NewDatabase(store config.Store, appID string, opts ...DatabaseOption) *Database
db := source.NewDatabase(configStore, "myapp")

DatabaseOption

OptionSignatureDescription
WithPollIntervalWithPollInterval(d time.Duration) DatabaseOptionSets the polling interval for watch. Default is 30 seconds.
db := source.NewDatabase(configStore, "myapp",
    source.WithPollInterval(10 * time.Second),
)

Get

Retrieves a config entry from the store and wraps it as a Value. The entry's value is converted to a string representation.

val, err := db.Get(ctx, "max-connections")
if err != nil {
    // source: key not found
}
fmt.Println(val.Raw)     // "25"
fmt.Println(val.Version) // 3

Value type conversion:

Go typeString representation
stringAs-is
intstrconv.Itoa
int64strconv.FormatInt
float64strconv.FormatFloat
boolstrconv.FormatBool
OtherEmpty string

List

Returns all config entries for the app, optionally filtered by prefix.

vals, err := db.List(ctx, "db.")

Watch

Registers a callback for changes to a specific key. The database source uses a background polling goroutine to detect changes by comparing version numbers on each tick. Watch is non-blocking and returns immediately.

db.Watch(ctx, "max-connections", func(ctx context.Context, key string, val *source.Value) {
    if val == nil {
        fmt.Printf("key %s was deleted\n", key)
        return
    }
    fmt.Printf("changed: %s = %s (version %d)\n", key, val.Raw, val.Version)
})

A single goroutine is started on the first Watch call and polls all watched keys on each tick. The goroutine exits when Close() is called or the context is cancelled.

Change detection: The poller compares the Version field of each watched key against the last seen version. When the version changes, all callbacks for that key are fired. Deletions (key no longer exists) are also detected and fire the callback with a nil value.

False-positive avoidance: When Watch is called, the current version is seeded from the store so the first poll tick does not produce a spurious notification.

Close

Stops the poll goroutine and waits for it to exit. Safe to call multiple times.

db.Close()

Error mapping

The database source maps vault.ErrConfigNotFound to source.ErrKeyNotFound for consistent error handling across sources.

Properties

PropertyValue
Name()"database"
Supports WatchYes (polling-based)
Supports ListYes
Thread-safeYes (sync.RWMutex)
Close()Stops poll goroutine

Chain

The Chain composes multiple sources in priority order. The first source listed has the highest priority.

Creating a Chain

func NewChain(sources ...Source) *Chain
chain := source.NewChain(
    source.NewMemory(),                        // highest priority
    source.NewEnv("MYAPP"),                    // middle priority
    source.NewDatabase(configStore, "myapp"),   // lowest priority
)

Get (first-hit-wins)

Queries sources in order. Returns the value from the first source that contains the key. If all sources return ErrKeyNotFound, the chain returns ErrKeyNotFound. Non-ErrKeyNotFound errors are propagated immediately.

func (c *Chain) Get(ctx context.Context, key string) (*Value, error)
val, err := chain.Get(ctx, "db-host")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("value=%s source=%s\n", val.Raw, val.Source)
// e.g., "value=localhost source=env"

List (merged)

Merges values from all sources. Higher-priority sources override lower ones for the same key. Insertion order is preserved.

func (c *Chain) List(ctx context.Context, prefix string) ([]*Value, error)
vals, err := chain.List(ctx, "")
for _, v := range vals {
    fmt.Printf("%-20s = %-20s (from %s)\n", v.Key, v.Raw, v.Source)
}

Watch (all sources)

Registers the watcher on every source in the chain. Any source that supports watching will fire the callback.

func (c *Chain) Watch(ctx context.Context, key string, fn WatchFunc) error
chain.Watch(ctx, "feature.enabled", func(ctx context.Context, key string, val *source.Value) {
    fmt.Printf("changed: %s = %s (from %s)\n", key, val.Raw, val.Source)
})

Close

Closes all sources, collecting any errors.

func (c *Chain) Close() error

Returns a joined error if any source's Close() fails.

Full example

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/xraph/vault/source"
)

func main() {
    ctx := context.Background()

    // Create sources.
    mem := source.NewMemory()
    env := source.NewEnv("MYAPP")
    db := source.NewDatabase(configStore, "myapp")

    // Build a priority chain.
    chain := source.NewChain(mem, env, db)
    defer chain.Close()

    // Set a value in memory (highest priority).
    mem.Set(ctx, "feature.enabled", "true")

    // Environment variable MYAPP_DB_HOST=prod-db.example.com is set.
    // Database has db-host=localhost.

    // Memory wins for this key.
    val, err := chain.Get(ctx, "feature.enabled")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s = %s (from %s)\n", val.Key, val.Raw, val.Source)
    // feature.enabled = true (from memory)

    // Env wins over database for this key.
    val, err = chain.Get(ctx, "db-host")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s = %s (from %s)\n", val.Key, val.Raw, val.Source)
    // db-host = prod-db.example.com (from env)

    // Watch for changes (memory source supports this).
    chain.Watch(ctx, "feature.enabled", func(ctx context.Context, key string, val *source.Value) {
        fmt.Printf("CHANGED: %s = %s\n", key, val.Raw)
    })

    // Trigger the watcher.
    mem.Set(ctx, "feature.enabled", "false")
    // CHANGED: feature.enabled = false

    // List all values (merged from all sources).
    vals, err := chain.List(ctx, "")
    if err != nil {
        log.Fatal(err)
    }
    for _, v := range vals {
        fmt.Printf("  %-25s = %-20s (from %s)\n", v.Key, v.Raw, v.Source)
    }
}

On this page