Skip to content

How to Test Data in Bignbit

Jump to bottom
Jacqueline Ryan edited this page Jun 9, 2025 · 2 revisions

Skip to procedure: https://github.com/podaac/bignbit/wiki/How-to-Test-Data-in-Bignbit#procedure

The Bignbit Step Function

step_function

This chart appears complicated but can be explained as essentially three steps:

  1. [Top part before the branch] Required metadata is gathered at the dataset/collection level and the granule level.
  2. [Both branches in the middle] One or more browse images are created for the input granule.
  3. [Bottom part] Metadata is created and a message is sent to GIBS notifying the delivery of the browse image(s).

Why are there two branches in the middle?

The left branch of the step function is the nominal process intended for most imagery processed through bignbit. Given some challenges with the format of Opera DSWx-HLS data, a special case was required for this dataset. In the nominal process, Harmony is called asynchronously to create one browse image per variable in the source granule.

Prerequisites

  • Fill out a GSFC NGAP CloudTamer access request on NAMS. You will need the following accounts and roles:
    • Role: APP-Developer
    • Account: podaac-application-services-sit-3404
    • Account: podaac-app-services-uat-1858
    • Account: podaac-application-cumulus-sit-1937
    • Account: podaac-application-cumulus-uat-1674
    • Contact: Sandra M. Blevins (may be different for your specific case)

Procedure

  1. Log in to the Earthdata Cloud Dashboard aka Kion.
  2. Select the podaac-application-services-* account for the venue you would like to test (sit/uat) in Kion. This will open a new tab with the AWS dashboard.
  3. Under the regions dropdown at the top of the page, select us-west-2 (Oregon). If you have not selected the correct region (us-east-1 is the default), the Step Function will not appear.
  4. Enter the Step Functions dashboard and search for podaac-sit-svc-BrowseImageWorkflow (or the equivalent for UAT).
  5. You may manually execute a test by clicking "Start Execution" and pasting in a valid Cumulus Message Adapter (CMA) message. To see examples of CMA messages, see the test data in this repository.
  6. Successful tests will of course pass all steps of the step function, and can be verified by checking the existence of the CNM.json in podaac-*-svc-internal bucket for the venue. Ideally, GIBS Ingest should be mocked on the venue and return a message as well.

Future Work

  • Automate test procedure.
  • Develop a methodology for auto-generating test CMA messages from a given granule on any DAAC.
  • Issues with the bignbit step function itself will be tracked separately.

Clone this wiki locally