Question: Possible to use the expect plugin to fail the test based on metrics?

[ahernandez-allegion]

Hello,

The goal is to fail the load test based on some metric, either built-in or a custom one. It seems the expect plugin can only fail based on individual responses during a scenario. Anyone have any thoughts? We're trying to use Artillery as a gate within our pipeline and as of right now there doesn't seem to be very many options outside a custom plugin or a lot of other scripting and coordination.

[hassy]

That's definitely something that would make sense and be useful to other users of the plugin. What sort of a custom metric do you have in mind / what would make sense for your use case?

[ahernandez-allegion]

That's definitely something that would make sense and be useful to other users of the plugin. What sort of a custom metric do you have in mind / what would make sense for your use case?

The built-in ensure feature only works 1) at a global level and 2) only with the basic statistics (min, max, mean, etc). What would be nice would be able to add any of the other statistics such as RPS, number of users, number of requests, or even latency for other quartiles. This would be useful for ensuring a specific load could be sustained for a specific period of time, e.g. 1000 RPS over 5 minutes or something similar. Lastly, we would want to be able to control this at the scenario level. As an example, we know that some requests take longer than others and we don't want that factored into the average latency overall.

[vegerot]

Any update on this?