sheriff.Marshal now returns unexported named type, breaking map type assertions

[lucasoares]

Previously, my code used a type assertion to convert the return value of sheriff.Marshal into a map[string]any:

var publicOptions = &sheriff.Options{
	Groups:          []string{"public"},
	IncludeEmptyTag: true,
}

func MarshalOnlyPublic(config any) (map[string]any, error) {
	data, err := sheriff.Marshal(publicOptions, &config)

	if err != nil {
		return nil, fmt.Errorf("parsing public config data: %w", err)
	}

	if data == nil {
		return nil, nil
	}

	if dataMap, ok := data.(map[string]any); ok {
		return dataMap, nil
	}

	return nil, fmt.Errorf("invalid public config data format: %w", err)
}

After updating sheriff, my tests started failing because the type assertion no longer works: sheriff.Marshal now returns the named type sheriff.kvStore.

Since kvStore is unexported and not accessible outside the sheriff package, I cannot manipulate it directly.

What is the recommended way to work with the returned value as a regular map without having to copy or convert it again to map[string]any?

For now, my workaround is to type assert the result to the KVStore interface and manually rebuild the map. However, this approach is memory-intensive and far from ideal and this workaround won't fully work for my previous code. If there are embedded structs, all nested maps remain as KVStore instances.

I had a function that performed value replacements across the map, which is now broken. I'm starting to wonder if the only viable workaround is to use json.Marshal followed by json.Unmarshal on the result to convert everything to map[string]any, but that would be much more expensive in terms of time and memory.

func ApplyToMap(mapData *map[string]any, fn ApplyFn) error {
	for k, v := range *mapData {
		if v == nil {
			continue
		}

		switch value := v.(type) {
		case string:
			result, err := fn(value)
			if err != nil {
				return err
			}

			(*mapData)[k] = result
		case map[string]any:
			err := ApplyToMap(&value, fn)
			if err != nil {
				return err
			}

			(*mapData)[k] = value
		case []any:
			err := ApplyToAnyList(&value, fn)
			if err != nil {
				return err
			}

			(*mapData)[k] = value
		}
	}

	return nil
}

[mweibel]

@alecsammon possibly also something for you to check. Can we make this easier?

@lucasoares just to gain knowledge on how people use sheriff, what's the reason for needing to have the map vs. just json marshalling it?

you can use the KVStoreFactory for supplying your own type until we figure out a better API for this. Example here: https://go.dev/play/p/GiNcbT5Hk7f

[lucasoares]

I use code that recursively walks through a generated map and manipulates all values. It assumes the map may contain nested maps, slices, and primitive values.

In my setup, I use sheriff to split a struct into two maps: one containing secret fields (stored in Vault) and another containing public fields (stored in the database). When saving the secrets, I encrypt every value in the resulting secret map before writing it to Vault.

This logic relies on being able to work with standard map[string]any types, since the recursive processing code expects regular maps rather than named or custom types. I do not want to implement this recursive function to iterate over all values myself.

I can use the KVStoreFactory to supply my own type, this is true, but it will not work in this case unfortunately since the code I'm using to walk through the map can not be changed right now.