Batch insertion of forwards by lazy aggregation #1934
Description
Currently, for every forward, we perform multiple database queries (one on accept and one on settle/fail). This is not very efficient and we could perform optimizations using different approaches similar to what we do in LND.
One thing that we may want to consider here is the lazy aggregation approach of lnd: https://github.com/lightningnetwork/lnd/blob/99d07fdb781ac4cd912e8054d105ff09a3cf15dd/htlcswitch/switch.go#L2535-L2562. The trade off is that for a period of time (the flush interval) the db records aren't consistent.
A middle ground (a bit more complex) would be to maintain what's effectively an on disk WAL. Forwards come in, are written to this quickly as an append only file. A background goroutine then reads this file, and updates entries in the db, relying on the primary key to prevent duplicates.
Originally posted by @Roasbeef in #1921 (comment)