[WIP] example for save to oci registry with root dir

[jonburdo]

Description

How Has This Been Tested?

Merge criteria:

• All the commits have been signed-off (To pass the DCO check)

• The commits have meaningful messages
• Automated tests are provided as part of the PR for major new functionalities; testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work.
• Code changes follow the kubeflow contribution guidelines.
• For first time contributors: Please reach out to the Reviewers to ensure all tests are being run, ensuring the label ok-to-test has been added to the PR.

If you have UI changes

• The developer has added tests or explained why testing cannot be added.
• Included any necessary screenshots or gifs if it was a UI change.
• Verify that UI/UX changes conform the UX guidelines for Kubeflow.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jonburdo. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jonburdo]

top-level dir fix: #2498

This test script here shows how save_to_oci_registry produces modelcars:

main before top-level dir fix (373689e)

One layer per file, but files in subdirs are flattened (a bug).

Model layers:
  layer 0 (b4bcc101f462)
    └── README.md  (file)
  layer 1 (d67563971e3a)
    └── config.json  (file)
  layer 2 (287ddcc05c41)
    └── vocab.txt  (file)
  layer 3 (9ff5cec2adfa)
    └── model.onnx  (file)
  layer 4 (b1a1a1298901)
    └── quantized.bin  (file)

Expected: ['README.md', 'onnx/model.onnx', 'onnx/weights/quantized.bin', 'tokenizer/config.json', 'tokenizer/vocab.txt']
Found:    ['README.md', 'config.json', 'model.onnx', 'quantized.bin', 'vocab.txt']
FAIL - missing paths: ['onnx/model.onnx', 'onnx/weights/quantized.bin', 'tokenizer/config.json', 'tokenizer/vocab.txt']

main after top-level dir fix (154f0da)

flattening bug is fixed, with caveat that we have one layer per subdir:

Model layers:
  layer 0 (db94786fa1a8)
    └── README.md  (file)
  layer 1 (0e6b4455fa50)
    ├── onnx  (dir)
    ├── onnx/model.onnx  (file)
    ├── onnx/weights  (dir)
    └── onnx/weights/quantized.bin  (file)
  layer 2 (9b7c666d9ab2)
    ├── tokenizer  (dir)
    ├── tokenizer/config.json  (file)
    └── tokenizer/vocab.txt  (file)

Expected: ['README.md', 'onnx/model.onnx', 'onnx/weights/quantized.bin', 'tokenizer/config.json', 'tokenizer/vocab.txt']
Found:    ['README.md', 'onnx', 'onnx/model.onnx', 'onnx/weights', 'onnx/weights/quantized.bin', 'tokenizer', 'tokenizer/config.json', 'tokenizer/vocab.txt']
PASS - all expected paths preserved

with root_dir (2f7ab70)

This branch is like main before the top-level dir fix except:

• code modified to with root_dir:

  oci_layers_on_top(local_image_path, files, modelcard, root_dir=model_files_path)
  

• olot installed from git repo with pip install ~/src/olot

We have the bug fixed and we also have one layer per file, which is better.

Model layers:
  layer 0 (04df0059a085)
    └── README.md  (file)
  layer 1 (339de351c9d3)
    └── tokenizer/config.json  (file)
  layer 2 (c22d54ed2085)
    └── tokenizer/vocab.txt  (file)
  layer 3 (9f1b3cb0384e)
    └── onnx/model.onnx  (file)
  layer 4 (0d9704eef200)
    └── onnx/weights/quantized.bin  (file)

Expected: ['README.md', 'onnx/model.onnx', 'onnx/weights/quantized.bin', 'tokenizer/config.json', 'tokenizer/vocab.txt']
Found:    ['README.md', 'onnx/model.onnx', 'onnx/weights/quantized.bin', 'tokenizer/config.json', 'tokenizer/vocab.txt']
PASS - all expected paths preserved

[jonburdo]

This branch allowed for testing, but I've done the actual fix in a new PR: #2552