Scripted test to upload a file and check it is imported

[rufuspollock]

When developing AirCan i want a scripted test I can run to test the end to end functioning of a load of data into DataStore

Acceptance

• Create a dataset in CKAN test instance
• Create resource with a file
• Check on status of run using {ckan instance}/api/3/aircan_status
• Check datastore is updated (called data API): ... /api/3/datastore_search ...

Tasks

• Research what already exists (is it useful) @cuducos
  • Review Integration tests in ckanext-aircan https://github.com/datopian/ckanext-aircan/tree/master/cypress NOTES from eduardo: most of the tests might be repeated (comparing to integration-tests repo), maybe just focus on https://github.com/datopian/ckanext-aircan/blob/master/cypress/integration/aircan-connector.js
  • Review Integration tests for reference
• Decide whether you are running against a local instance or the cloud? If cloud what instance in the cloud? Let's run against the cloud and use the enterprise (+ versioning) https://ckan-enterprise-versioning.dev.datopian.com instance. As soon as we have an enterprise+aircan instance up and running we will use that!
• Test steps (in cypress) - we can use ckan-client-js ...
  • Create a dataset in CKAN test instance @cuducos
  • Create resource with a file @cuducos
  • Check you can download the file @cuducos
  • Check on status of run using {ckan instance}/api/3/aircan_status but we don't know run_id so how do we get it? => Maybe skip this
  • Check datastore is updated (called data API): call ... /api/3/datastore_search?resource_id={RES_ID} ... BLOCKED until we have instance deployed with AirCan
• Test steps 2
  • Explicitly submit dag by calling aircan_submit
  • Check on aircan_status using return run_id
    • Rename dag_status to aircan_status Rename action/API dag_status to aircan_status  #27

[leomrocha]

Latest discussion Q&A:

• Where do we put the “test” scripts: ckanext-aircan
• Does it get run automatically? Manually run for now
• Can you configure the ckan instance to run against? Yes! Default prob local ckan instance (e.g. localhost:5000) and put in README about the standard test ckan instance we have …
• How do I do local development/testing?
  • Just test the microservices
  • Test for integration by deploying into the cloud quickly and with automated integration tests