Skip to content

Pipeline performance logging - #345

Open
chrisarridge wants to merge 4 commits into
livefrom
ca/performance-logging
Open

Pipeline performance logging#345
chrisarridge wants to merge 4 commits into
livefrom
ca/performance-logging

Conversation

@chrisarridge

Copy link
Copy Markdown
Contributor

This PR adds some performance monitoring to the pipeline run script so we can measure the run-time of different parts of the pipeline.

Note, this PR is built on #343 which should be reviewed and merged first, then this PR rebased before merging.

…run test

The multiple getter run per day test uses random dates and this appears to be
producing random test failures in GitHub Actions.  Since these tests
invariably run okay locally, this commit changes these dates to be fixed.
Add performance monitoring to blocks in the pipeline bash script
so that we can log the performance of different parts of the pipeline
to a CSV file for analysis.
@jwilson232

Copy link
Copy Markdown
Contributor

@chrisarridge does $PERFORMANCE_CSV need adding into the server config?

@chrisarridge

Copy link
Copy Markdown
Contributor Author

@chrisarridge does $PERFORMANCE_CSV need adding into the server config?

Yes it does, I have the necessary changes on a local branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants