Allow full GMS/z file input for sub-basin GWLF-E

[SRGDamia1]

Currently, the sub-basin GWLF-E modeling endpoint (https://staging.modelmywatershed.org/api/modeling/subbasin/run/) does NOT allow the user to input a dictionary json of prepared HUC-12 GMS/Z files as input for the run. I would like to be able to use a generated or previously-stashed subbasin/prepare output as input for subbasin/run.

Right now the subbasin/run view delegates to _parse_gwlfe_input to handle the request body, but sets the raw input flag to be false which ignores any input and looks up a job instead.

model-my-watershed/src/mmw/apps/geoprocessing_api/views.py

Lines 1504 to 1557 in f96289d

	def start_modeling_subbasin_run(request, format=None):
	"""
	Starts a job to run GWLF-E for all subbasins in given input.
	Given a `job_uuid` of a subbasin/prepare job, runs GWLF-E on every HUC-12
	subbasin whose results were prepared.
	The result includes `Loads` which have total nitrogen, phosphorus, and
	sediment loads for GWLF-E land cover types, including Hay/Pasture,
	Cropland, Wooded Areas, Open Land, Barren Areas, Low-Density Mixed, Medium-
	Density Mixed, High-Density Mixed, Low-Density Open Space, and other
	categories such as Farm Animals, Stream Bank Erosion, Subsurface Flow,
	Welands, Point Sources, and Septic Systems.
	The total loads across all these cateogires are summarized in
	`SummaryLoads`.
	`Loads` and `SummaryLoads` are also available for every HUC-12 in the
	input. Every HUC-12 also has `Catchments` level data, which include
	Loading Rate Concentrations and Total Loading Rates for nitrogen,
	phosphorus, and sediment.
	"""
	user = request.user if request.user.is_authenticated else None
	model_input, job_uuid, mods, hash = _parse_gwlfe_input(request, False)
	# Instead of using start_celery_job, we do a manual implementation here.
	# This is required because start_celery_job tries to add an error handler
	# to ever job in the chain, but this includes groups, for which that is not
	# allowed. So we manually create a job and a task_chain, wire up all the
	# error handling, and return the job response.
	job = Job.objects.create(created_at=now(), result='', error='',
	traceback='', user=user, status='started')
	task_chain = _initiate_subbasin_gwlfe_job_chain(
	model_input, job_uuid, mods, hash, job.id)
	job.uuid = task_chain.id
	job.save()
	return Response(
	{
	'job': task_chain.id,
	'job_uuid': task_chain.id,
	'status': JobStatus.STARTED,
	# TODO Remove this message when `job` is deprecated
	'messages': [
	'The `job` field will be deprecated in an upcoming release. '
	'Please switch to using `job_uuid` instead.'
	],
	},
	headers={'Location': reverse('geoprocessing_api:get_job',
	args=[task_chain.id])}
	)

The flag can't be simply reversed because _parse_gwlfe_input(request, raw_input) attempts to validate the gms by way of the validate_gwlfe_run function

model-my-watershed/src/mmw/apps/geoprocessing_api/views.py

Lines 1692 to 1733 in f96289d

	def _parse_gwlfe_input(request, raw_input=True):
	"""
	Given a request, parses the model_input from it.
	If raw_input is True, then assumes there may be a model_input key specified
	directly in the request body. If found, uses that. Otherwise uses the
	job_uuid field.
	Raises validation errors where appropriate.
	"""
	job_uuid = request.data.get('job_uuid')
	mods = request.data.get('modifications', list())
	hash = request.data.get('inputmod_hash', '')
	one_of = ['input', 'job_uuid'] if raw_input else ['job_uuid']
	check_exactly_one_provided(one_of, request.data)
	if raw_input:
	model_input = request.data.get('input')
	if model_input:
	validate_gwlfe_run(model_input)
	return model_input, job_uuid, mods, hash
	if not validate_uuid(job_uuid):
	raise ValidationError(f'Invalid `job_uuid`: {job_uuid}')
	input_job = get_object_or_404(Job, uuid=job_uuid)
	if input_job.status == JobStatus.STARTED:
	raise exceptions.JobNotReadyError(
	f'The prepare job {job_uuid} has not finished yet.')
	if input_job.status == JobStatus.FAILED:
	raise exceptions.JobFailedError(
	f'The prepare job {job_uuid} has failed.')
	model_input = json.loads(input_job.result)
	return model_input, job_uuid, mods, hash

And the validate_gwlfe_run function checks keys for a single gms file

model-my-watershed/src/mmw/apps/geoprocessing_api/validation.py

Lines 99 to 102 in f96289d

	def validate_gwlfe_run(input):
	missing_keys = [k for k in GWLFE_INPUT_KEYS if k not in input]
	if missing_keys:
	raise ValidationError(f'Provided `input` is missing: {missing_keys}')

If the GMS cannot be validated, the only option is a job UUID. So either the _parse_gwlfe_input function or the validate_gwlfe_run would need to be changed to accommodate/validate the dictionary of gms files format.

[rajadain]

Some additional context:

If you run gwlf-e/prepare and subbasin/prepare for the same HUC-12, you'll see that the output is slightly different. In subbasin, the GMS JSON is nested under the HUC ID. That is the input this card wants to be able to give subbasin/run.

The reason we didn't support that in the beginning is because when you have a lot of HUCs involved in a subbasin (e.g. Schuylkill HUC-8 with ~60 HUC-12s), the input payload is too much for our web service. Since the data is already stored in the database, it made sense to reuse that.

However, now we're allowing running HUC-12s in the subbasin endpoint, so the payload can be much smaller. Furthermore, since the input is almost the same as that generated for gwlf-e/prepare, we want to reuse those outputs for subbasin work.

So, for HUC-12s, we should allow this. In case the payload is too large, the app will return an error, and the user will be notified of the "payload too large" automatically. We don't have to handle any such use case in this card, at least I think not.