Mapping Struct Slices Fields

[kataras]

Hello @unknwon,

I am thinking to add support for loading from .ini for Iris Configuration and custom configurations. So far we have support for json, yaml and toml and they're working fine. I have a problem though, while trying to read a config file to the iris.Configuration structure, I have defined the ini fields, I tried allowShadow with custom ini.LoaderOptions but that doesn't work either. Code speaks by itself:

type Configuration struct {
 Tunneling TunnelingConfiguration `ini:"tunneling"`
}

type TunnelingConfiguration struct {
  WebInterface string `ini:"web_interface"`
  Tunnels []Tunnel `ini:"tunnels"`
}

type Tunnel struct {
  Name string `ini:"name"`
  Addr string `ini:"addr"`
}

I tried plenty of ini formats but I would love to support something like that (if already exists, I couldn't find it):

[tunneling]
web_interface = http://127.0.0.1:5050
[tunneling.tunnels]
name = tunnel1
addr = test1
[tunneling.tunnels]
name = tunnel2
addr = test2

How I load

b, err := ioutil.ReadFile(filename)
f, err := ini.LoadSources(ini.LoadOptions{
	Insensitive:         true,
	InsensitiveKeys:     true,
	InsensitiveSections: true,
	AllowNonUniqueSections: true,
	//	AllowShadows:           true,
	DebugFunc: func(s string) {
		fmt.Printf("debug: %s\n", s)
	},
}, b)

return f.StrictMapTo(dest) // where dest is *Configuration

So even if AllowNonUniqueSections is true, the Tunnels are never binded to the dest one.

I did manage to do it by using this code,befoer StrictMapTo:

	if sections, err := f.SectionsByName("tunneling.tunnels"); err == nil {
		for _, section := range sections {
			nameKey, err := section.GetKey("name")
			if err != nil || nameKey == nil {
				continue
			}
			name := nameKey.Value()
			if name == "" {
				continue
			}

			addrKey, err := section.GetKey("addr")
			if err != nil || addrKey == nil {
				continue
			}
			addr := addrKey.Value()

			dest.Tunneling.Tunnels = append(dest.Tunneling.Tunnels, iris.Tunnel{
				Name: name,
				Addr: addr,
			})
		}

	}

Is there a way to do that mapping automatically or it's a planned feature?
I think would be trivial to do that, you already collecting multi sections of the same key under a section, so why not add support for appending them to the corresponding field?

Thanks,
Gerasimos Maropoulos.

[kataras]

I have one more proposal, support aliases in the section names.

The current NameMapper can only return a single name for keys (and not for sections, see SectionsByName). I think we can add a new field named : AliasMapper = func(section string) []string { return []string{"iris."+ section} } (PR: #265) . I need to map the keys either through root or a child if the Iris Configuration was embedded as a field in a custom end-developer's struct.

[unknwon]

Hi @kataras, thanks for investigating into this!

I manage to get it work by applying the following diff:

type TunnelingConfiguration struct {
	WebInterface string   `ini:"web_interface"`
-	Tunnels.     []Tunnel `ini:"tunnels"`
+	Tunnels      []Tunnel `ini:"tunneling.tunnels,,,nonunique"`
}

I know this is very very unintuitive 😅 and the "nonunique" part is undocumented...

Full program

package main

import (
	"fmt"
	"log"

	"github.com/davecgh/go-spew/spew"
	"gopkg.in/ini.v1"
)

type Configuration struct {
	Tunneling TunnelingConfiguration `ini:"tunneling"`
}

type TunnelingConfiguration struct {
	WebInterface string   `ini:"web_interface"`
	Tunnels      []Tunnel `ini:"tunneling.tunnels,,,nonunique"`
}

type Tunnel struct {
	Name string `ini:"name"`
	Addr string `ini:"addr"`
}

func main() {
	config := `
[tunneling]
web_interface = http://127.0.0.1:5050

[tunneling.tunnels]
name = tunnel1
addr = test1

[tunneling.tunnels]
name = tunnel2
addr = test2`

	f, err := ini.LoadSources(ini.LoadOptions{
		Insensitive:            true,
		AllowNonUniqueSections: true,
	}, []byte(config))
	if err != nil {
		log.Fatalf("Failed to load: %v", err)
	}

	var dest Configuration
	err = f.StrictMapTo(&dest)
	if err != nil {
		log.Fatalf("Failed to map: %v", err)
	}

	fmt.Println()

	spew.Dump(dest)
}

[kataras]

Hello @unknwon That's awesome, I didn't even notice that in the code.

OK, one problem solved. We have two more issues to solve and we are ready to go:

• Currently, mapping net.IP is not possible. How we can solve it? The net.IP completes the TextUnmarshaller interface. This allows us to read it as string before trying to read it as a slice of uint8. So a support of TextUnmarshaller on that kind of packages is a MUST I think.

Example Code

package main

import (
	"net"

	"github.com/davecgh/go-spew/spew"
	"github.com/go-ini/ini"
)

type Configuration struct {
	PrivateSubnets []IPRange `ini:"private_subnets,,,nonunique"`
}

type IPRange struct {
	Start net.IP `ini:"start"`
	End   net.IP `ini:"end"`
}

var testNetIP = `[private_subnets]
start = 192.168.1.1
end = 192.168.1.9

[private_subnets]
start = 192.168.1.10
end = 192.168.1.20
`

func main() {
	f, err := ini.LoadSources(ini.LoadOptions{
		Insensitive:            true,
		AllowNonUniqueSections: true,
	}, []byte(testNetIP))
	if err != nil {
		panic(err)
	}

	var dest Configuration
	if err = f.StrictMapTo(&dest); err != nil {
		panic(err)
	}

	spew.Dump(dest)
}

• The second one, is map type not supported. That leads people to add custom binders for each type of map, e.g. map[string]string, map[string]bool, map[string]interface{}.

Example Code

func bindMapStringINI(section *ini.Section, dest map[string]string, titleKeys bool) {
	if section == nil || dest == nil {
		return
	}

	for _, sectionKey := range section.Keys() {
		key := sectionKey.Name()
		value := sectionKey.Value()

		if key == "" || value == "" {
			continue
		}

		if titleKeys {
			key = strings.Title(key)
		}

		dest[key] = value
	}
}

func bindMapBoolINI(section *ini.Section, dest map[string]bool, titleKeys bool) {
	if section == nil || dest == nil {
		return
	}

	for _, sectionKey := range section.Keys() {
		key := sectionKey.Name()
		if key == "" {
			continue
		}

		value, err := sectionKey.Bool()
		if err != nil {
			continue
		}

		if titleKeys {
			key = strings.Title(key)
		}

		dest[key] = value
	}
}

// same as bindMapString but it accepts a map[string]interface{} instead.
func bindMapINI(section *ini.Section, dest map[string]interface{}, titleKeys bool) {
	if section == nil || dest == nil {
		return
	}

	for _, sectionKey := range section.Keys() {
		key := sectionKey.Name()
		if key == "" {
			continue
		}

		value := sectionKey.Value()
		if value == "" {
			continue
		}

		if titleKeys {
			key = strings.Title(key)
		}

		dest[key] = value
	}
}

Also note that the AliasMapper is still necessary because we need to match sections like [iris.tunneling.tunnels] and [tunneling.tunnels](when the end-developer didn't specified a parent of iris).

[unknwon]

@kataras Thanks for the follow up!

I suggest to file separate issues for better tracking :)

[jennes]

I stumbled upon this, too; but for my usecase the "nonunique" declaration helped. Thanks alot!