Add test for "Regression in v3.8.x PopulateFeedBacklog depends on un-started InboxReader"

[KolbyML]

Fixes NIT-4014

Problem

We fixed NIT-3997 "Regression in v3.8.x PopulateFeedBacklog depends on un-started InboxReader", but want to add a regression test that the fix did work, and a similar bug doesn't happen again

The fix was done in this PR

• Correct the order TransactionStreamer and PopulateFeedBacklog #3812

Solution

I added a regression test it passes on master (i.e. this branch), but fails before the original fix was merged.

To replicate this test failing more the fix was in place you can checkout this branch https://github.com/KolbyML/nitro/tree/old-add-regresion-test

Then run make go-test, cancel it after the test setup completes aka tests start running, then run go test ./system_tests -run TestRegressionInPopulateFeedBacklog -v -count=1 -timeout=10m and it will fail with error message

    common_test.go:687:  [] error populating feed backlog on startup: error getting message 1: failed to fetch batch mentioned by batch posting report: not started 

[pmikolajczyk41]

as we recently discovered, this will turn batch poster everywhere and should be adapted to #3829; bringing @ganeshvanahalli attention in case this PR will get merged first

[pmikolajczyk41]

@KolbyML I'm afraid that the Aggregate test results fails because your fork doesn't have the codecov token secret. Probably you just have to use the original repo for PRs (though it's not a required check)

[KolbyML]

@pmikolajczyk41 ready for another look 🫡

[pmikolajczyk41]

it's really great to have steps for reproduction, but (apologies for nitpicking), it'd be more straightforward to make old-add-regression-test branch differ from the actual branch by the fix we are testing (i.e., #3812); currently these branches have much more differences (KolbyML/nitro@new-add-regression-test...old-add-regresion-test and KolbyML/nitro@old-add-regresion-test...new-add-regression-test)

I run the test with and without fix and I confirm that the test does its job :D

[KolbyML]

it's really great to have steps for reproduction, but (apologies for nitpicking), it'd be more straightforward to make old-add-regression-test branch differ from the actual branch by the fix we are testing (i.e., #3812); currently these branches have much more differences (KolbyML/[email protected] and KolbyML/[email protected])

good to know I will keep that in mind for future PR's 🫡

I run the test with and without fix and I confirm that the test does its job :D

🥳