Concurreny Support

[jefree]

Hi there, i'm just trying a kinda simple code that involves go routines:

// main.go

package main

import (
  "github.com/mitchellh/go-mruby"
  "sync"
)

func main() {
  mrb := mruby.NewMrb()
  defer mrb.Close()

  var wg sync.WaitGroup

  for i := 1; i < 50000; i++ {
    wg.Add(1)

    go func() {
      mrb.LoadString("puts 'hola mundo'")
      wg.Done()
    }()
  }

  wg.Wait()

after doing go run main.go it starts to print the messages as expected but after a while it raise a fatal error:

fatal error: unexpected signal during runtime execution
fatal error: schedule: holding locks
panic during panic
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1357b900160 pc=0x55e9799579c9]

it seems like at some point something really bad is happening inside the Mrb struct, and it just starts crashing over and over again.

my go version is go1.7.4 linux/amd64 running in a Debian 9.1 (stretch)

[erikh]

Our multi-builder in box does this I think, let me look at how it works.

[erikh]

Ok, I used your code to reproduce. https://gist.githubusercontent.com/anonymous/b54f657dd38c88aef8deb0e637b785fb/raw/bbce9d7d31f400da8a3eebe90ea4c64359994720/stdin is the error, I'll refer to this later when i can fix it. Just search for `go-mruby`. This is in my box build so I can quickly re-use go-mruby, but it's just a stripped build of mruby itself is the only major differentiator. If you need more info just ask. here's the relevant trace, it's almost definitely that code I fixed the last stack corruption issue in, the one that uses that go->c trampoline to handle mruby function binding IIRC. I'll see what I can do to resolve this over the weekend or something. goroutine 703 [syscall, locked to thread]:github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby._Cfunc__go_mrb_load_string(0x127c440, 0x7f26200008c0, 0x0, 0x0) github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/_obj/_cgo_gotypes.go:906 +0x57github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.(*Mrb).LoadString.func2(0x127c440, 0x7f26200008c0, 0x0, 0x0) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/mruby.go:182 +0x80github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.(*Mrb).LoadString(0xc42000e028, 0x532937, 0x11, 0x0, 0x0, 0x0) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/mruby.go:182 +0xa2 main.main.func1(0xc42000e028, 0xc420016060) /go/src/github.com/box-builder/box/tmp/test.go:21 +0x40 created by main.main /go/src/github.com/box-builder/box/tmp/test.go:20 +0xb8 goroutine 704 [runnable, locked to thread]:github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby._Cfunc_mrb_gc_arena_restore(0x127c440, 0x2c8) github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/_obj/_cgo_gotypes.go:1263 +0x45github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.newExceptionValue.func2(0x127c440, 0x2c8) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/value.go:242 +0x68github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.newExceptionValue(0x127c440, 0xc420c40000) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/value.go:267 +0x3f4github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.checkException(0x127c440, 0x7f26280008c0, 0x0) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/mruby.go:380 +0x41github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby.(*Mrb).LoadString(0xc42000e028, 0x532937, 0x11, 0x0, 0x0, 0x0) /go/src/github.com/box-builder/box/vendor/github.com/mitchellh/go-mruby/mruby.go:183 +0xbd main.main.func1(0xc42000e028, 0xc420016060) /go/src/github.com/box-builder/box/tmp/test.go:21 +0x40 created by main.main /go/src/github.com/box-builder/box/tmp/test.go:20 +0xb8

[erikh]

did this ever get fixed?

[erikh]

ok, this actually ends up being a problem with the mrb object not being re-entrant... in mruby itself when entered externally with load_string. I'm not entirely certain this would work with mruby without golang either.

Conversely, this code executes just fine, allocating a single interpreter per goroutine:

// main.go

package main

import (
	"sync"

	"github.com/mitchellh/go-mruby"
)

func main() {

	var wg sync.WaitGroup

	for i := 1; i < 50000; i++ {
		wg.Add(1)

		go func() {
			mrb := mruby.NewMrb()
			defer mrb.Close()
			mrb.LoadString("puts 'hola mundo'")
			wg.Done()
		}()
	}

	wg.Wait()
}

since this is a resolution you can take today, I strongly suggest you take it. You can wrap mutual accesses with mutex locks of your own devising, allowing you to load code sequentially from different goroutines. Alternatively, you can feed that code through a channel and load it serially.

we can do this as well; I just need to think of the situations that can go wrong by embedding a mutex in the struct around these operations, and whether or not it's better to just say "deal with it yourself". If other maintainers have a solution they like, please chime in.