Recently I added more checks into CircleCI configuration. It's really helpful and let you know if something wrong in the pushed plugin. But if it fails, you have to go to the CircleCI and look into the logs. The same thing with warnings. There's no possibility to show warnings with CircleCI - the build is only green or red.
Would be much better to have a kind of PR summary posted after all checks are finished. Something like codecov does:
It's possible via GitHub API, but in this case, you have to write your script and add it into CircleCI config. The problem is a github token. In order to allow script to publish a comment, you have to pass api token into the script. It may be done with CircleCI env variables, but if you want to run this script for the external PRs you need to allow passing env variables to external PRs. It's not a secure way, because external user can change a circleci config and get access to github token.
So the good alternative is to build our own github bot (with probot, for instance). Bot is listening for webhooks and can run checks and post PR summary. Another benefit from this approach - we automate issue triage and add some labels like new plugin, update, and, for example, priority based on plugin popularity.
It's related to #332 as well.
Recently I added more checks into CircleCI configuration. It's really helpful and let you know if something wrong in the pushed plugin. But if it fails, you have to go to the CircleCI and look into the logs. The same thing with warnings. There's no possibility to show warnings with CircleCI - the build is only green or red.
Would be much better to have a kind of PR summary posted after all checks are finished. Something like codecov does:
It's possible via GitHub API, but in this case, you have to write your script and add it into CircleCI config. The problem is a github token. In order to allow script to publish a comment, you have to pass api token into the script. It may be done with CircleCI env variables, but if you want to run this script for the external PRs you need to allow passing env variables to external PRs. It's not a secure way, because external user can change a circleci config and get access to github token.
So the good alternative is to build our own github bot (with probot, for instance). Bot is listening for webhooks and can run checks and post PR summary. Another benefit from this approach - we automate issue triage and add some labels like
new plugin,update, and, for example, priority based on plugin popularity.It's related to #332 as well.