Postgres/YugaBytes Implementation

[realazthat]

I have partially implemented a PG implementation in python as a prototype, would you be interested in me porting this back to Go for a PR?

[mkmik]

Hey! Yeah it would be great!

[mkmik]

btw it doesn't have to be in Go; if you have a solid impl in python it can stay in python. As long as it speaks the same gRPC protocol.

A rewrite in Go would make it easier for it to be integrated in a single project, possibly be tested and maintained together.

I also have some code for a python client somewhere.

[realazthat]

I have ported it to Go.

Blocks until a PR:

1. Where would you like the server to live? Right now I put it in
   pkg/pg_impl/server.go.
2. How do you want it integrated? Right now I hijacked listen.go, commented
   out masstaskerpb.RegisterMassTaskerServer(s, NewWithContext(ctx)) and put
   my own there instead. Note, my code is significantly more setup, because of
   Postgres, and I also pass in the slog.Logger. I'll post a snippet below at
   the bottom of this post.
   • Question: Do you want me to clobber your listen.go with conditionals?
   • Question: Do you want me to clobber your main.go with conditionals?
   • Question: Do we want to force everyone running main.go to have a postgres
     dependency? I am not a Go expert, so I don't know how to best set up
     multiple subprojects with different dependencies. If that is not possible,
     we might want to make it a separate project. I've had this sort of issue
     with python, and it is possible there but quite a bit of effort.
3. I am not yet sure what the best/standard way is to handle schema migrations
   over time, so for now I will just provide InitializeDB() and DropDB
   functions, expose them in main.go via an argument.

Alteration in listen.go:

func Listen(ctx context.Context, addr string, opts ...ListenOpt) (*grpc.Server, net.Listener, error) {
        ....
	logger := slog.New(slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{}))

	// DO NOT SUBMIT
	dsn := os.Getenv("DATABASE_URL")
	slog.Info("using database", "dsn", dsn)

	if dsn == "" {
		return nil, nil, fmt.Errorf("DATABASE_URL is not set")
	}

	config, err := pgxpool.ParseConfig(dsn)
	if err != nil {
		return nil, nil, err
	}
	// From https://stackoverflow.com/questions/64504879/how-to-log-queries-with-pgx
	config.ConnConfig.Tracer = &queryTracer{loggger: logger}
	pool, err := pgxpool.NewWithConfig(ctx, config)
	if err != nil {
		return nil, nil, err
	}

	p, err := pg_impl.NewWithContext(ctx, logger, pool)
	if err != nil {
		return nil, nil, err
	}

	options.grpcOptions = append(options.grpcOptions, grpc.UnaryInterceptor(loggingUnaryInterceptor(logger)))

	s := grpc.NewServer(options.grpcOptions...)
	// masstaskerpb.RegisterMassTaskerServer(s, NewWithContext(ctx))
	masstaskerpb.RegisterMassTaskerServer(s, p)
	....