Skip to content

Commit 97c51d1

Browse files
ramakrishnap-nvclaudesvc-nvskills-signing
authored
skills: remove redundant concept-only skills (#1487)
Remove `cuopt-routing-formulation` and `cuopt-server-common`. Their content has been merged into `cuopt-routing-api-python` and `cuopt-server-api-python` respectively. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
1 parent bc4b17d commit 97c51d1

28 files changed

Lines changed: 145 additions & 598 deletions

File tree

.claude-plugin/marketplace.json

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -56,24 +56,12 @@
5656
"skills": "./",
5757
"description": "Trace and interpret the Pareto frontier across competing objectives using repeated single-objective cuOpt solves (weighted-sum and ε-constraint)."
5858
},
59-
{
60-
"name": "cuopt-routing-formulation",
61-
"source": "./skills/cuopt-routing-formulation",
62-
"skills": "./",
63-
"description": "Vehicle routing (VRP, TSP, PDP) — problem types and data requirements. Domain concepts; no API or interface."
64-
},
6559
{
6660
"name": "cuopt-routing-api-python",
6761
"source": "./skills/cuopt-routing-api-python",
6862
"skills": "./",
6963
"description": "Vehicle routing (VRP, TSP, PDP) with cuOpt — Python API only. Use when the user is building or solving routing in Python."
7064
},
71-
{
72-
"name": "cuopt-server-common",
73-
"source": "./skills/cuopt-server-common",
74-
"skills": "./",
75-
"description": "cuOpt REST server — what it does and how requests flow. Domain concepts; no deploy or client code."
76-
},
7765
{
7866
"name": "cuopt-server-api-python",
7967
"source": "./skills/cuopt-server-api-python",

AGENTS.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@ AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/`
1414
### Common (concepts only; no API code)
1515
- `skills/cuopt-numerical-optimization-formulation/` — LP / MILP / QP: concepts + problem parsing + common formulation patterns
1616
- `skills/cuopt-multi-objective-exploration/` — Multi-objective: trace + interpret the Pareto frontier across competing objectives (ε-constraint / weighted-sum over repeated cuOpt solves)
17-
- `skills/cuopt-routing-formulation/` — Routing: VRP, TSP, PDP (problem types, data)
18-
- `skills/cuopt-server-common/` — Server: capabilities, workflow
1917

2018
### Installation
2119
- `skills/cuopt-install/` — User install for Python, C, and server (pip, conda, Docker, verification). For building cuOpt from source, see `skills/cuopt-developer/`.

skills/cuopt-multi-objective-exploration/BENCHMARK.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s
77
## Evaluation Summary
88

99
- Skill: `cuopt-multi-objective-exploration`
10-
- Evaluation date: 2026-06-26
10+
- Evaluation date: 2026-06-29
1111
- NVSkills-Eval profile: `external`
1212
- Environment: `astra-sandbox`
1313
- Dataset: 5 evaluation tasks
@@ -55,10 +55,10 @@ Task composition is derived from the evaluation dataset when possible. Entries w
5555
| Dimension | Num | `claude-code` | `codex` |
5656
|---|---:|---:|---:|
5757
| Security | 5 | 100% (+0%) | 100% (+0%) |
58-
| Correctness | 5 | 90% (+37%) | 76% (+24%) |
59-
| Discoverability | 5 | 80% (+60%) | 71% (+41%) |
60-
| Effectiveness | 5 | 91% (+20%) | 68% (+10%) |
61-
| Efficiency | 5 | 80% (+40%) | 71% (+30%) |
58+
| Correctness | 5 | 90% (+0%) | 87% (-8%) |
59+
| Discoverability | 5 | 78% (-2%) | 73% (-22%) |
60+
| Effectiveness | 5 | 85% (-6%) | 85% (+2%) |
61+
| Efficiency | 5 | 75% (-3%) | 71% (-21%) |
6262

6363
Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.
6464

skills/cuopt-multi-objective-exploration/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,4 +135,4 @@ Practical notes:
135135
This skill is solver- and interface-agnostic. The per-solve mechanics — building the objective, adding the ε constraints, passing a warm start, reading status — live in the API skills:
136136

137137
- `cuopt-numerical-optimization-api-python` / `-api-c` / `-api-cli` — LP, MILP, QP solves.
138-
- `cuopt-routing-formulation` + `cuopt-routing-api-python` — the same frontier workflow applies to routing tradeoffs (distance vs. vehicles vs. time).
138+
- `cuopt-routing-api-python` — the same frontier workflow applies to routing tradeoffs (distance vs. vehicles vs. time).

skills/cuopt-multi-objective-exploration/skill-card.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,16 @@ This skill is ready for commercial/non-commercial use. <br>
77
NVIDIA <br>
88

99
### License/Terms of Use: <br>
10-
Apache 2.0 <br>
10+
Apache-2.0 <br>
1111
## Use Case: <br>
12-
Developers and engineers exploring tradeoffs between competing objectives in optimization problems, tracing Pareto frontiers to make informed multi-objective decisions rather than accepting a single implicit weighting. <br>
12+
Developers and engineers exploring tradeoffs between competing objectives in optimization problems, using cuOpt to trace the Pareto frontier and interpret exchange rates rather than collapsing to a single weighted answer. <br>
1313

1414
### Deployment Geography for Use: <br>
1515
Global <br>
1616

1717
## Requirements / Dependencies: <br>
18-
**Requires API Key or External Credential:** [No] <br>
19-
**Credential Type(s):** [None] <br>
18+
**Requires API Key or External Credential:** [Not Specified] <br>
19+
**Credential Type(s):** [None identified] <br>
2020

2121
Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>
2222

@@ -26,23 +26,23 @@ Mitigation: Review and scan skill before deployment. <br>
2626

2727
## Reference(s): <br>
2828
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html) <br>
29-
- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples) <br>
29+
- [cuopt-examples](https://github.com/NVIDIA/cuopt-examples) <br>
3030

3131

3232
## Skill Output: <br>
3333
**Output Type(s):** [Analysis, Configuration instructions] <br>
34-
**Output Format:** [Markdown with inline code blocks] <br>
34+
**Output Format:** [Markdown with inline formulations and solver-call patterns] <br>
3535
**Output Parameters:** [1D] <br>
3636
**Other Properties Related to Output:** [None] <br>
3737

3838
## Evaluation Agents Used: <br>
39-
- Claude Code (`claude-code`) <br>
40-
- Codex (`codex`) <br>
39+
- claude-code <br>
40+
- codex <br>
4141

4242

4343

4444
## Evaluation Tasks: <br>
45-
Evaluated against 5 evaluation tasks (4 positive activation, 1 negative activation) using NVSkills-Eval external profile in astra-sandbox environment. <br>
45+
Evaluated against 5 internal evaluation tasks (4 positive skill-activation, 1 negative). <br>
4646

4747
## Evaluation Metrics Used: <br>
4848
Reported benchmark dimensions: <br>
@@ -67,10 +67,10 @@ Underlying evaluation signals used in this run: <br>
6767
| Dimension | Num | `claude-code` | `codex` |
6868
|---|---:|---:|---:|
6969
| Security | 5 | 100% (+0%) | 100% (+0%) |
70-
| Correctness | 5 | 90% (+37%) | 76% (+24%) |
71-
| Discoverability | 5 | 80% (+60%) | 71% (+41%) |
72-
| Effectiveness | 5 | 91% (+20%) | 68% (+10%) |
73-
| Efficiency | 5 | 80% (+40%) | 71% (+30%) |
70+
| Correctness | 5 | 90% (+0%) | 87% (-8%) |
71+
| Discoverability | 5 | 78% (-2%) | 73% (-22%) |
72+
| Effectiveness | 5 | 85% (-6%) | 85% (+2%) |
73+
| Efficiency | 5 | 75% (-3%) | 71% (-21%) |
7474

7575
## Skill Version(s): <br>
7676
26.08.00 (source: frontmatter) <br>
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbXVsdGktb2JqZWN0aXZlLWV4cGxvcmF0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjgwNmFlYWFjOWM3YmMxZTk1ZGJlNDdiOWVhY2FjMDk5NjBiZDczMTA4NTY0ZWE3NDk1OTdlYWI3ZDA2N2VlN2MiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiOTcyM2E4ZjZjOWQ1ZTlkZDAxYTU0N2VlOGYyYzE2MzA5ZWU0YzIzMGY0NThiYjBiYWU1ODZjMTllMTJkNWE1MyIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ0ODgwOTFlMTU0NWZmMGQxMDA0M2VmZjhiMDY2OTNkOTJiNTM5MzdkZjYxMjQyZWQzM2Q3NzZhZTM4NGZiIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJkaWdlc3QiOiAiZDk1NjE1ZWUzMjc1MTc3MWYzMDc3ZjE3MzY4M2YwNGZkMTdhZDU3Y2U4MmFlOWIxMjRjYmQ4M2ZhYWJkNjBiYiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogImM2OTJlM2Q5ZjAzMDQ1YWM2YTA5NzkwYWM4NjM4NGM4ODYxOTNhZmY2ODQxZDdjMDdhZThjMWZmYjAyYzdhMjkiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9CiAgICBdLAogICAgInNlcmlhbGl6YXRpb24iOiB7CiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRpZ25vcmUiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0sCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IgogICAgfQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMEbPeltBPGv9lD/CCSbTi12cHXvWbA2qoDfCcVLU420GcQ4kGa4r3aSmODk8E/ylNwIxALYaAi9ppzhEiqwoa7zbN1R0vq9QbUvNCcqFPcx7lLlHblGFS7nHB632atREfijbig==","keyid":""}]}}
1+
{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbXVsdGktb2JqZWN0aXZlLWV4cGxvcmF0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjcxMmY0NDMwMjNkYTc5ODNmNzliOGIwNzlmN2YzODljM2IzNDJmNGY1NGZiNWViZGRhZmM1ODZiMjZlM2ZkYzIiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0KICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICIzNzhjNmMzMDY0YzVkNDE0ZjQ5M2M2NTZjODQ2YWZlYWMyMmMyMWU4NjhmY2MzMzRlMThiNGNjMmFmYjIwMjM4IiwKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI5NWFhOTRiNGViNzQwOTE5YTI4MTFmNWY4OWRkZmVhZmVjZTkwNjcxNjJjMzE4NzRlYTBkYTM0ZGM4NWJhZDEyIiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImQ5NTYxNWVlMzI3NTE3NzFmMzA3N2YxNzM2ODNmMDRmZDE3YWQ1N2NlODJhZTliMTI0Y2JkODNmYWFiZDYwYmIiLAogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJiM2Y4NDZhMzExZDc5Zjg4ZTNhYzlmNGJmZmM3YjM4MDI4OGM5Nzk1ZTA3YzNhN2ViZWYxYjY2YWE0MGU3YTEzIiwKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfQogICAgXQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMC4Gw/I2q9Fy5WQ5dDlqOtYhZcp3y08aFKflnEzfYDWsZHEjlAIbutAhexTr80iANQIwIrWIJX73+kbJi/HXWsYHWBKF+3oqu0Yrkqgg2OFrlmdaDGzwZTKSFKqenudCk401","keyid":""}]}}

skills/cuopt-routing-api-python/BENCHMARK.md

Lines changed: 13 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,13 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s
77
## Evaluation Summary
88

99
- Skill: `cuopt-routing-api-python`
10-
- Evaluation date: 2026-05-29
10+
- Evaluation date: 2026-06-29
1111
- NVSkills-Eval profile: `external`
12-
- Environment: `local`
12+
- Environment: `astra-sandbox`
1313
- Dataset: 1 evaluation tasks
14-
- Attempts per task: 2
14+
- Attempts per task: 1
1515
- Pass threshold: 50%
16-
- Overall verdict: FAIL
16+
- Overall verdict: PASS
1717

1818
## Agents Used
1919

@@ -54,46 +54,27 @@ Task composition is derived from the evaluation dataset when possible. Entries w
5454

5555
| Dimension | Num | `claude-code` | `codex` |
5656
|---|---:|---:|---:|
57-
| Security | 2 | 100% (+0%) | 100% (+0%) |
58-
| Correctness | 2 | 100% (+0%) | 95% (+3%) |
59-
| Discoverability | 2 | 100% (+0%) | 70% (-5%) |
60-
| Effectiveness | 2 | 83% (+14%) | 83% (+12%) |
61-
| Efficiency | 2 | 93% (-0%) | 56% (-5%) |
57+
| Security | 1 | 100% (+0%) | 100% (+0%) |
58+
| Correctness | 1 | 100% (+70%) | 97% (+42%) |
59+
| Discoverability | 1 | 100% (+100%) | 82% (+57%) |
60+
| Effectiveness | 1 | 82% (+45%) | 74% (+29%) |
61+
| Efficiency | 1 | 95% (+67%) | 72% (+45%) |
6262

6363
Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.
6464

6565
## Tier 1: Static Validation Summary
6666

67-
Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings.
67+
Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 2 total findings.
6868

6969
Top findings:
7070

7171
- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-routing-api-python/SKILL.md`)
72-
- MEDIUM SECURITY/Unknown (SQP-2): Binding the cuOpt server to 0.0.0.0 exposes it on all network interfaces, making it accessible to any host that can reac (`references/server_examples.md:7`)
73-
- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-routing-api-python/SKILL.md`)
74-
- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-routing-api-python/SKILL.md`)
75-
- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-routing-api-python/SKILL.md`)
72+
- LOW SCHEMA/author_format: Author must be of the form 'Name <email@host>' (`skills/cuopt-routing-api-python/SKILL.md`)
7673

7774
## Tier 2: Deduplication Summary
7875

79-
Tier 2 validation reported findings. NVSkills-Eval ran 2 checks and found 4 total findings.
80-
81-
Top findings:
82-
83-
- HIGH DUPLICATE/duplicate: Duplicate content found within references/server_examples.md:
84-
"# Poll for solution" in references/server_examples.md (lines 45-51)
85-
vs "# Poll for solution" in references/server_examples.md (lines 156-162) (`references/server_examples.md:45`)
86-
- HIGH DUPLICATE/duplicate: Duplicate content found across SKILL.md and references/examples.md:
87-
"# Capacities" in SKILL.md (lines 30-35)
88-
vs "# Add capacity dimension (name, demand_per_order, capacity_per_vehicle)" in references/examples.md (lines 73-75)
89-
vs "# Add capacity dimension" in references/examples.md (lines 156-158) (`SKILL.md:30`)
90-
- HIGH DUPLICATE/duplicate: Duplicate content found across references/examples.md and references/server_examples.md:
91-
"## Additional References (tested in CI)" in references/examples.md (lines 237-249)
92-
vs "## Additional References (tested in CI)" in references/server_examples.md (lines 193-204) (`references/examples.md:237`)
93-
- HIGH DUPLICATE/duplicate: Duplicate content found across assets/pdp_basic/README.md and assets/pdp_basic/model.py:
94-
"# Pickup-Delivery (PDP)" in assets/pdp_basic/README.md (lines 1-7)
95-
vs "(module docstring)" in assets/pdp_basic/model.py (lines 1-2) (`assets/pdp_basic/README.md:1`)
76+
This tier was not run or did not produce findings in this report.
9677

9778
## Publication Recommendation
9879

99-
The skill should be reviewed before NVSkills-Eval publication. Skill owners should address the findings above and rerun NVSkills-Eval to refresh this benchmark.
80+
The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change.

0 commit comments

Comments
 (0)