Skip to content

Conversation

@Synicix
Copy link
Contributor

@Synicix Synicix commented Sep 2, 2025

  • Remove source commit for now (Perhaps we should discuss to add a additional_label member where it is HashMap<String, String> and the user can put whatever they want in there. This would be far more flexible and allow for custom labels
  • Remove recommended_cpu and recommended_memory and combined it into RecommendSpecs struct that no longer affects the hash of the pod
  • Update GPU requirement specs as that is very code dependent and is a all or nothing when it come to reproducibility
  • Clean up a few things I caught to make the code base more robust

Depends on #114

Resolves PIPE-119

@Synicix Synicix requested review from Copilot and eywalker September 2, 2025 15:41
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors pod hashing to exclude non-replication concerns from the hash computation, focusing on reproducibility requirements. The changes consolidate execution recommendations into a separate structure and update GPU requirements to better reflect code dependencies.

  • Removed source commit URL and individual CPU/memory fields from pod hash
  • Consolidated CPU and memory recommendations into RecommendSpecs struct
  • Updated GPU requirements to use CUDA version instead of specific models
  • Added comprehensive pipeline execution infrastructure with Docker runner

Reviewed Changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/uniffi/model/pod.rs Core changes to Pod structure, removing hash-affecting fields and adding RecommendSpecs
src/uniffi/store/filestore.rs Updated to handle separate storage of recommendation specs with timestamps
tests/store.rs Updated tests to use new Pod constructor and RecommendSpecs structure
src/core/pipeline_runner.rs New comprehensive pipeline execution infrastructure
tests/fixture/mod.rs Updated fixture functions to use new Pod API and added pipeline test utilities

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@codecov
Copy link

codecov bot commented Sep 2, 2025

@Synicix Synicix linked an issue Sep 2, 2025 that may be closed by this pull request
@Synicix Synicix marked this pull request as draft September 2, 2025 15:48
@Synicix Synicix marked this pull request as ready for review September 3, 2025 11:04
@Synicix Synicix mentioned this pull request Oct 2, 2025
@eywalker
Copy link
Contributor

Indeed, we should turn your points into issues to be tracked. Namely:

  1. add additional_label so that users can add arbitrary information. Among these are Git hashes. Generally we should discuss the topic of capturing the concept of execution environment more broadly.
  2. consider placing annotation about expected reproducibiility (e.g. GPU-based code can have not perfectly reproducible execution).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should most definitely turn this into a module so we can have smaller src code dedicated to each operator, especially as we add operators in the future.

@eywalker eywalker merged commit 32ed2d6 into nauticalab:dev Oct 20, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Source commit url should be moved out of Pod spec

2 participants