Incremental Sim Brainstorming & System Design

[jswope2112]

Checked for duplicates

Yes - I've already checked

Alternatives considered

Yes - and alternatives don't suffice

Related problems

No response

Describe the feature request

Investigation into the features and implementation of incremental sim for AERIE

Become up to speed on the codebase

• Onboard @jswope211 to the AERIE codebase and general workflow
• Onboard @jswope2112 to the incremental sim problem

Update previous work:

• [x ] Investigate features implemented on incremental sim created by @bradNASA
• Merge the develop branch into the incremental sim branch

Work items identified in discussion: #669 (comment)

1. database interface for storing and merging partial (incremental) simulation results
   a. How does this overlap with streaming results?
2. testing - see Sim Regression Testing Framework #1421
   a. ensure that the inc sim git branch results for a first/single simulation are the same as for the develop branch
   b. ensure that incremental simulations have the same results as if fully simulated. [This should be extensive and consider many interactions (of tasks and conditions), especially within a single time point.]
   c. test with existing complex models (e.g., from clipper)
   d. scheduling tests
   e. performance tests (simulation and scheduling)
   f. test with contrib streamline resources
3. scheduling integration -- Adrien suggested that most or all of a java class could be removed since it replicates what inc sim already does.
4. container-level management -- routing sim requests to the merlin-worker with the right SimulationEngine
5. inc sim user controls
   a. override to do full simulation (and throw out past sims)
   b. override to allow inc sim to apply across sim config changes (?)
6. inc sim core implementation (if re-doing the work in prototype/incremental-sim branch)
7. combine checkpoint restart and inc sim so that past sim can be loaded from DB and used with inc sim
8. provide user/modeler the ability to mark a task, activity, directive, cell, or resource as "always stale" so that inc sim knows to rerun/recompute it.
9. have a collapse mechanism to merge a chain of simulation engines in memory into one to avoid overhead of chaining
10. have a collapse mechanism to merge partial sim results in the database (related to bullet 1 above)