Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# v10.3.0 - 2025/09/17

## Enhancements

- Add test support for Bugsnag remote config [793](https://github.com/bugsnag/maze-runner/pull/793)

# v10.2.0 - 2025/09/12

## Enhancements
Expand Down
5 changes: 5 additions & 0 deletions DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,8 @@ Internally, Maze Runner has the following modes of operation
## Test outputs

See [Test Outputs](./docs/Test_Outputs.md) for details of logging and other outputs generated by Maze Runner.

## Features

* [Commands](./docs/features/Commands.md)
* [Error Config support](./docs/features/Error_Config.md)
1 change: 1 addition & 0 deletions bin/maze-runner
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ require_relative '../lib/maze/loggers/logger'
require_relative '../lib/maze/servlets/base_servlet'
require_relative '../lib/maze/servlets/all_commands_servlet'
require_relative '../lib/maze/servlets/command_servlet'
require_relative '../lib/maze/servlets/error_config_servlet'
require_relative '../lib/maze/servlets/servlet'
require_relative '../lib/maze/servlets/log_servlet'
require_relative '../lib/maze/servlets/trace_servlet'
Expand Down
2 changes: 1 addition & 1 deletion docs/Mock_Server.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,6 @@ The mock server provides a number of endpoints for test fixture to use:
- `/sourcemap`, `react-native-source-map` - for `POST`ing Bugsnag sourcemaps to for later verification
- `/reflect` - provides a mechanism for instructing the server to behave in certain ways (e.g. responding after a specified time delay)
- `/logs` - provides a mechanism for recording and checking log messages
- `/command` and `/commands` - provides a mechanism for feeding instructions and any other information to the test fixture using only HTTP requests instigated by the test fixture. Essential for platforms that either do not support Appium, or render in such a way that elements are not accessible. See [Mock Server](./Commands.md) for more information,
- `/command` and `/commands` - provides a mechanism for feeding instructions and any other information to the test fixture using only HTTP requests instigated by the test fixture. Essential for platforms that either do not support Appium, or render in such a way that elements are not accessible. See [Mock Server](./features/Commands.md) for more information,
- `/metrics` - provides a mechanism for collecting arbitrary metrics from a test fixture, collating and writing them to a CSV file at the end of a run.
- `/docs' - a document server, just set Maze.config.document_server_root to the file location you want to server documents from.
2 changes: 1 addition & 1 deletion docs/Commands.md → docs/features/Commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Any Ruby hash can be added to the list of Commands, for example:
Maze::Server.commands.add command
```

How commands are formed is governed by the contract formed between the Cumuber scenarios and test fixture.
How commands are formed is governed by the contract formed between the Cucumber scenarios and test fixture.

Each time a command is added, two additional fields are automatically set:
- `:uuid` - a UUID for the command.
Expand Down
58 changes: 58 additions & 0 deletions docs/features/Error_Config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Error Config support

## Endpoint

The `/error-config` endpoint is provided on the mock server for test fixtures to request error configuration from. Maze Runner stores any received requests for later inspection.

## Cucumber steps

Error configs can we added to an internal queue using the following step. Error configs are removed from the queue when they are served.

```
When I prepare an error config with:
| type | name | value |
| header | Cache-Control | max-age=604800 |
| property | body | @features/support/error_config.json |
| property | status | 200 |
```

- The `type` can be either `header` or `property`.
- If the `value` provided for `body` starts with `@` then Maze Runner will treat it as a file location to read the actual body from.

Properties of received error config requests can be checked using the following steps:

```
the {word} {string} header equals {string}
the {word} {string} query parameter equals {string}
```

(where `word` is `error config request` in this case).

## Example scenario

```Cucumber
Scenario: Requesting an error config
# Prepare the error config to be served
When I prepare an error config with:
| type | name | value |
| header | Cache-Control | max-age=604800 |
| property | body | @features/support/error_config.json |
| property | status | 200 |

# Run the scenario that will request the error config
Then I run "HandledJavaSmokeScenario"
And I wait to receive an error
And I wait for 1 error config to be requested

# Check the error config request headers and parameters
Then the error config request "Bugsnag-Api-Key" header equals "12312312312312312312312312312312"
And the error config request "version" query parameter equals "1.2.3"
And the error config request "versionCode" query parameter equals "123"
And the error config request "releaseStage" query parameter equals "production"
And the error config request "osVersion" query parameter equals "11"

# Check the other outputs from the tst
Then the error payload field "events" is an array with 1 elements
And the exception "errorClass" equals "java.lang.IllegalStateException"
# And so on ...
```
16 changes: 16 additions & 0 deletions lib/features/steps/error_config_steps.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# frozen_string_literal: true

# @!group Error config steps

#
# Shortcut to waiting to receive requests for error configs
#
# @step_input count [Integer] Number of error config requests expected
Then('I wait for {int} error config(s) to be requested') do |count|
step "I wait to receive #{count} error config requests"
end

Then('I prepare an error config with:') do |table|
Maze.check.equal(%w[type name value], table.column_names, 'Error config table expects column headers "type", "name" and "value"')
ErrorConfigSupport.prepare_error_config(table.hashes)
end
2 changes: 1 addition & 1 deletion lib/features/support/cucumber_types.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ParameterType(
name: 'request_type',
regexp: /errors?|sessions?|builds?|logs?|metrics?|sampling requests?|traces?|uploads?|sourcemaps?|reflects?|reflections?|invalid requests?/,
regexp: /errors?|sessions?|error config requests?|error configs?|builds?|logs?|metrics?|sampling requests?|traces?|uploads?|sourcemaps?|reflects?|reflections?|invalid requests?/,
type: String,
transformer: ->(s) { s }
)
Expand Down
39 changes: 39 additions & 0 deletions lib/features/support/error_config_support.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
class ErrorConfigSupport
class << self
def prepare_error_config(table_hashes)

error_config = {
headers: {},
}
table_hashes.each do |hash|
type = hash['type']
name = hash['name']
value = hash['value']

case type
when 'header'
error_config[:headers][name] = value
when 'property'
case name
when 'status'
error_config[:status] = value.to_i
when 'body'
if value.start_with?('@')
body = File.read(value[1..])
else
body = value
end
error_config[:body] = body
error_config[:headers]['ETag'] = Digest::SHA1.hexdigest(body.to_s)
else
raise "Unknown property '#{name}'"
end
else
raise "Unknown type '#{type}'"
end
end

Maze::Server.error_configs.add(error_config)
end
end
end
1 change: 1 addition & 0 deletions lib/features/support/internal_hooks.rb
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@
if (scenario.failed? && Maze.config.log_requests) || Maze.config.always_log
$stdout.puts '^^^ +++' if ENV['BUILDKITE']
output_received_requests('errors')
output_received_requests('error config requests')
output_received_requests('sessions')
output_received_requests('traces')
output_received_requests('builds')
Expand Down
2 changes: 1 addition & 1 deletion lib/maze.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# providing an alternative to the proliferation of global variables or singletons.
module Maze

VERSION = '10.2.0'
VERSION = '10.3.0'

class << self
attr_accessor :check, :driver, :internal_hooks, :mode, :start_time, :dynamic_retry, :public_address,
Expand Down
1 change: 1 addition & 0 deletions lib/maze/maze_output.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ def write_requests

request_types = %w[errors sessions builds uploads logs sourcemaps traces ignored invalid reflections]
request_types << 'sampling requests'
request_types << 'error config requests'

request_types.each do |request_type|
list = Maze::Server.list_for(request_type).all
Expand Down
20 changes: 19 additions & 1 deletion lib/maze/server.rb
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ def list_for(type)
errors
when 'session', 'sessions'
sessions
when 'error config', 'error configs'
error_configs
when 'error config request', 'error config requests'
error_config_requests
when 'build', 'builds'
builds
when 'log', 'logs'
Expand Down Expand Up @@ -131,6 +135,20 @@ def sampling_requests
@sampling_requests ||= RequestList.new
end

# A list of error config requests received
#
# @return [RequestList] Received error config requests
def error_config_requests
@error_config_requests ||= RequestList.new
end

# A list of error config responses to be returned to the client
#
# @return [RequestList] Error config responses to be returned
def error_configs
@error_configs ||= RequestList.new
end

# A list of trace requests received
#
# @return [RequestList] Received trace requests
Expand Down Expand Up @@ -240,7 +258,6 @@ def start
response.status = 200
end

# When adding more endpoints, be sure to update the 'I should receive no requests' step
server.mount '/notify', Servlets::Servlet, :errors
server.mount '/sessions', Servlets::Servlet, :sessions
server.mount '/builds', Servlets::Servlet, :builds
Expand All @@ -256,6 +273,7 @@ def start
server.mount '/unity-line-mappings', Servlets::Servlet, :sourcemaps
server.mount '/command', Servlets::CommandServlet
server.mount '/commands', Servlets::AllCommandsServlet
server.mount '/error-config', Servlets::ErrorConfigServlet
server.mount '/logs', Servlets::LogServlet
server.mount '/metrics', Servlets::Servlet, :metrics
server.mount '/reflect', Servlets::ReflectiveServlet
Expand Down
2 changes: 1 addition & 1 deletion lib/maze/servlets/command_servlet.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ class CommandServlet < BaseServlet

NOOP_COMMAND = '{"action": "noop", "message": "No commands queued"}'

# Serves the next command, if these is one.
# Serves the next command, if there is one.
#
# @param request [HTTPRequest] The incoming GET request
# @param response [HTTPResponse] The response to return
Expand Down
66 changes: 66 additions & 0 deletions lib/maze/servlets/error_config_servlet.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# frozen_string_literal: true

require 'json'

module Maze
module Servlets

# Allows clients to request error configs that have been added to the queue.
class ErrorConfigServlet < BaseServlet

BAD_REQUEST_BODY = '{
"type":"about:blank",
"title":"Bad Request",
"status":400,
"detail":"Maze Runner has not been given an error config to return"
}'

# Captures the details of the request for checking and serves the next error config, if there is one.
#
# @param request [HTTPRequest] The incoming GET request
# @param response [HTTPResponse] The response to return
def do_GET(request, response)

if Server.error_configs.size_remaining > 0
# Server the next error config in the queue
error_config = Server.error_configs.current
error_config[:headers].each do |key, value|
response.header[key] = value
end
response.body = error_config[:body]
response.status = 200

Server.error_configs.next
else
# Log and return an error
$logger.error 'Error config requested but none are queued - returning 400 Bad Request'
response.body = BAD_REQUEST_BODY
response.status = 400
end
response.header['Content-Type'] = 'application/json'

# Store the query parameters in the error config request list
details = {
body: {},
query: Rack::Utils.parse_nested_query(request.query_string),
request_uri: request.request_uri,
request: request,
response: response,
method: 'GET'
}
Server.error_config_requests.add(details)
end

# Logs and returns a set of valid headers for this servlet.
#
# @param request [HTTPRequest] The incoming GET request
# @param response [HTTPResponse] The response to return
def do_OPTIONS(request, response)
super

response.header['Access-Control-Allow-Methods'] = 'GET, OPTIONS'
response.status = Server.status_code('OPTIONS')
end
end
end
end
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
Feature: Comparing JSON payloads to fixture files

Scenario: The request body matches the template text exactly
When I send a "equal"-type request
When I make a "equal"-type POST request
Then I wait to receive an error
And the error payload body matches the JSON fixture in "features/fixtures/exact_match.json"
And the error payload body matches the JSON fixture in "features/fixtures/fuzzy_match.json"

Scenario: The request body matches the template when ignoring fields
When I send an "ignore"-type request
When I make an "ignore"-type POST request
Then I wait to receive an error
And the error payload body matches the JSON fixture in "features/fixtures/ignore_apple.json"
And the error payload body does not match the JSON fixture in "features/fixtures/exact_match.json"
And the error payload body does not match the JSON fixture in "features/fixtures/fuzzy_match.json"

Scenario: The request body fuzzy matches the template
When I send an "fuzzy match"-type request
When I make an "fuzzy match"-type POST request
Then I wait to receive an error
And the error payload body does not match the JSON fixture in "features/fixtures/exact_match.json"
And the error payload body matches the JSON fixture in "features/fixtures/fuzzy_match.json"

Scenario: A subset of the request body matches a template
When I send an "subset"-type request
When I make an "subset"-type POST request
Then I wait to receive an error
And the error payload field "items.0.subset" matches the JSON fixture in "features/fixtures/exact_match.json"
And the error payload field "items.0.subset" matches the JSON fixture in "features/fixtures/fuzzy_match.json"

Scenario: The request body matches the template using "NUMBER" wildcards
When I send a "numerics"-type request
When I make a "numerics"-type POST request
And I wait to receive an error
Then the error payload body matches the JSON fixture in "features/fixtures/numerics.json"

Scenario: The request body does not match the template using "NUMBER" wildcards
When I send an "ignore"-type request
When I make an "ignore"-type POST request
And I wait to receive an error
Then the error payload body does not match the JSON fixture in "features/fixtures/numerics.json"

Scenario Outline: The request body does not match the template
When I send an "<request_type>"-type request
When I make an "<request_type>"-type POST request
Then I wait to receive an error
And the error payload body does not match the JSON fixture in "features/fixtures/exact_match.json"
And the error payload body does not match the JSON fixture in "features/fixtures/fuzzy_match.json"
Expand Down
21 changes: 21 additions & 0 deletions test/e2e/comparison/features/error_config.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Feature: Error config requests and responses

Scenario: Basic handling of error-config request
When I prepare an error config with:
| type | name | value |
| header | Cache-Control | max-age=604800 |
| property | body | @features/support/error_config.json |
| property | status | 200 |
And I request an "android" type error config and store the response
And I wait for 1 error config to be requested

Then the error config request "Bugsnag-Api-Key" header equals "12312312312312312312312312312312"
And the error config request "version" query parameter equals "1.2.3"
And the error config request "versionCode" query parameter equals "123"
And the error config request "releaseStage" query parameter equals "production"
And the error config request "osVersion" query parameter equals "11"

And the stored error config status code equals "200"
And the stored error config body matches the contents of "features/support/error_config.json"
And the stored error config "Cache-Control" header equals "max-age=604800"
And the stored error config "Etag" header equals "43ec7d2b971daba752e9546da4e1dca8094107e0"
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
#!/usr/bin/env ruby

require 'net/http'

http = Net::HTTP.new('localhost', ENV['MOCK_API_PORT'])
Expand Down
Loading