Skip to content

Add batch insert API + types #24

Open
@NeilLuno

Description

@NeilLuno

Some of our transactions insert many events, and it's inefficient for each query to do two round trips to the server to prepare the statement then execute it. Mysql docs recommend inserting using a single INSERT with multiple VALUES tuples to speed up inserts.

I think we can achieve this by introducing a manyInserter type:

type EventToInsert struct {
  ForeignID string
  Type reflex.Type
  Metadata []byte
}

type manyInserter func(ctx context.Context, tx *sql.Tx, events []EventToInsert) error

and an API to insert many events:

func (t *EventsTable) InsertMany(ctx context.Context, tx *sql.Tx, events []EventToInsert) (NotifyFunc, error) {
    // Check noops...
	err := t.manyInserter(ctx, tx, events)
	if err != nil {
		return noopFunc, err
	}

	return t.notifier.Notify, nil
}

We can provide a default implementation like we do for inserter.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions