Skip to content

Sync: dev to prod #930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tech3371 merged 11 commits into from
Oct 13, 2025
Merged

Sync: dev to prod #930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bd8092d
MNT: add timeout on batch jobs
tech3371 Sep 24, 2025
bc35390
added pset deps
sdhoyt Oct 10, 2025
15cfa66
Merge branch 'dev' into pset-dep
sdhoyt Oct 10, 2025
4cc60e8
Merge pull request #928 from sdhoyt/pset-dep
sdhoyt Oct 10, 2025
f4da6f4
ENH: increase batch memory size
tech3371 Oct 13, 2025
45b8cf1
Merge branch 'dev' into batch_timeout
greglucas Oct 13, 2025
e0b641d
Merge pull request #886 from tech3371/batch_timeout
greglucas Oct 13, 2025
2a02f62
feedback
tech3371 Oct 13, 2025
aa6d57f
rebase fix
tech3371 Oct 13, 2025
632a76e
Merge branch 'dev' into batch_memory
tech3371 Oct 13, 2025
c6a0ddd
Merge pull request #929 from tech3371/batch_memory
tech3371 Oct 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions sds_data_manager/constructs/processing_construct.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ def add_job(self, job_name: str, data_access_url: str = ""):
image=ecs.ContainerImage.from_ecr_repository(
repository=container_repo, tag="latest"
),
memory=cdk.Size.mebibytes(4096),
cpu=1,
memory=cdk.Size.gibibytes(16),
cpu=4,
environment={
# Useful for switching APIs between dev / prod endpoints
"IMAP_DATA_ACCESS_URL": data_access_url,
Expand All @@ -137,4 +137,5 @@ def add_job(self, job_name: str, data_access_url: str = ""):
f"ProcessingJob-{job_name}",
job_definition_name=f"ProcessingJob-{job_name}",
container=container_definition,
timeout=cdk.Duration.hours(2),
)
9 changes: 7 additions & 2 deletions sds_data_manager/lambda_code/SDSCode/pipeline_lambdas/dependency_config.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -600,10 +600,15 @@ lo, l1b, de, lo, l1c, pset, HARD, DOWNSTREAM
lo, ancillary, good-times, lo, l1c, pset, HARD, DOWNSTREAM
lo, ancillary, hydrogen-background, lo, l1c, pset, HARD, DOWNSTREAM
lo, ancillary, oxygen-background, lo, l1c, pset, HARD, DOWNSTREAM
spin, spin, historical, lo, l1b, de, HARD, DOWNSTREAM
repoint, repoint, historical, lo, l1b, de, HARD, DOWNSTREAM
spin, spin, historical, lo, l1c, pset, HARD, DOWNSTREAM
repoint, repoint, historical, lo, l1c, pset, HARD, DOWNSTREAM
leapseconds, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
spacecraft_clock, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
attitude_history, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
imap_frames, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
planetary_ephemeris, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
pointing_attitude, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM
science_frames, spice, historical, lo, l1c, pset, HARD_NO_TRIGGER, DOWNSTREAM

# TODO: Maps for 5-7 days need to be defined. deg resolution for those maps are currently N/A in Lo mapping product sheet.

Expand Down