Add CI test run against a Netflify staging instance

[ivelin]

Is your feature request related to a problem? Please describe.
Currently the CI runs local tests, but not live tests against a Netlify functions pre-deployment environment.

Describe the solution you'd like
Levelarge the CI best practices from ambianic-ui. We run two separate CI workflow scripts of lighthouse tests against local and live netlify staging version of the new code in a PR or commit push.

[vickywane]

@ivelin To understand this better. Are you requesting tests are run against the local Netlify instance created using netlify-dev?

Currently, tests are run against deployed functions using their API endpoints from Netlify.

[ivelin]

@vickywane good point. I overlooked that the tests are running against the actual deployed ambianic-functions on Netflify. The issue with this approach is that we deploy first (potentially buggy new code) which instantly affects users and then we run tests against it.

The tests must be focused on the new code in the specific PR.

I will correct my request. Ideally we would have two separate workflows:

1. One that runs against a local CI copy of functions via netlify dev and catches any issues before deployment.
2. Another one that runs against a preview deployment of the code changed in the given PR. See how our LH flow waits for a preview deployment and tests against it. The goal of point 2 is to catch any issues before these functions are deployed in production where all users are instantly affected.

[vickywane]

This is quite an interesting edge case. It’s is important to note that we are using newman which run the postman scripts. For the scripts to be created they have to be given a live endpoint, i dont think this might be possible with a local endpoint.

Also netlify-dev changes the PORT where the functions are run each time it’s started.

I would give this a deeper look-into

[ivelin]

Postman with Netflify dev for local testing https://blog.postman.com/serverless-functions-the-fast-way/ Deploy previews https://docs.netlify.com/configure-builds/get-started/ Split testing and staged deployment https://docs.netlify.com/site-deploys/split-testing/ Deploy front end and functions without a maintenance window : https://docs.netlify.com/configure-builds/get-started/

On Sun, Mar 7, 2021 at 4:12 PM Nwani Victory ***@***.***> wrote: This is quite an interesting edge case. It’s is important to note that we are using newman which run the postman scripts. For the scripts to be created they have to be given a live endpoint, i dont think this might be possible with a local endpoint. Also netlify-dev changes the PORT where the functions are run each time it’s started. I would give this a deeper look-into — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARBUFP23GWIHHPHKHZZP33TCP25VANCNFSM4YYBJJ3Q> .

-- Ivelin Ivanov <https://www.linkedin.com/in/ivelinatanasoffivanov/> founder, Ambianic <http://ambianic.ai> Fall Detector ® Instantly know if your loved one has fallen down™