Added missing entries in test registry#5260
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the test coverage by adding numerous new test entries to the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly expands the testregistry.textproto by adding numerous new test entries across various feature areas, including Authz, BMP, CFM, Container, enrollz, IPSec, MPLS, P4RT, Policy Forwarding, Platform, BGP, ISIS, MPLS SR, Replay, gRIBI, gNMI, gNOI, System, and Transceiver. Some existing test entries were also refactored, changing their exec paths from .go files to metadata.textproto. The review comments highlight several issues with the new entries, specifically duplicate test entries for RT-2.18 and RT-10.2, a duplicate test ID TE-3.3, and several test IDs (TestID-16.4, bootz, and System-1.1 through System-1.5) that do not conform to the required uppercase alphanumeric format.
| exec: "feature/gribi/mpls_in_udp/otg_tests/mpls_over_udp_tunnel_hashing_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "TestID-16.4" |
There was a problem hiding this comment.
The test ID TestID-16.4 does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? defined in proto/testregistry.proto. The alphabetic part of the ID should be all uppercase. Please change it to TESTID-16.4.
| id: "TestID-16.4" | |
| id: "TESTID-16.4" |
| exec: "feature/system/mount_points/tests/system_mount_points_state/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "bootz" |
There was a problem hiding this comment.
The test ID bootz does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? defined in proto/testregistry.proto. Please assign a valid ID that follows this pattern.
|
Please fix the gemini comments, they all look correct to me? |
For redundant entries , Definitely have to fix those. For test id format , can keep those as it, as these will require changes in readme's and other applicable places, which are not related to current changes. |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request significantly expands the testregistry.textproto file by adding numerous new test entries across various feature areas, including Authz, BMP, CFM, Container, enrollz, IPSec, MPLS, P4RT, Policy Forwarding, Platform, BGP/ISIS routing, Segment Routing, Replay, gRIBI, gNMI, gNOI, System, and Transceiver functionalities. It also refactors some existing test entries by updating their exec paths to reference metadata.textproto files instead of direct .go files. The review comments identify several instances where the newly added test IDs do not conform to the required naming convention [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto, necessitating corrections to ensure proper formatting and adherence to the schema.
| exec: "feature/gribi/mpls_in_udp/otg_tests/mpls_over_udp_tunnel_hashing_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "TestID-16.4" |
There was a problem hiding this comment.
The test ID TestID-16.4 appears to be a placeholder and does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Please replace it with a valid and meaningful test ID, and update the plan_id in feature/gribi/otg_tests/gribi_to_bgp_redistribution/metadata.textproto accordingly.
| exec: " " | ||
| } | ||
| test: { | ||
| id: "Authz" |
There was a problem hiding this comment.
The test ID Authz does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Please use a valid ID and update the plan_id in feature/gnsi/authz/tests/authz/metadata.textproto accordingly.
| id: "Authz" | |
| id: "AUTHZ-1.0" |
| test: { | ||
| id: "enrollz-1" | ||
| description: "enrollz test for TPM 2.0 HMAC-based Enrollment flow" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm20_hmac/README.md" | ||
| exec: "feature/security/attestz/tests/enrollz_tpm20_hmac/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "enrollz-2" | ||
| description: "enrollz test for TPM 1.2 Enrollment flow" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm12/README.md" | ||
| exec: "feature/security/attestz/tests/enrollz_tpm12/metadata.textproto" | ||
| } |
There was a problem hiding this comment.
The test IDs enrollz-1 and enrollz-2 do not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Test IDs should start with uppercase letters. Please correct these IDs and update the plan_id in the corresponding metadata.textproto files.
| test: { | |
| id: "enrollz-1" | |
| description: "enrollz test for TPM 2.0 HMAC-based Enrollment flow" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm20_hmac/README.md" | |
| exec: "feature/security/attestz/tests/enrollz_tpm20_hmac/metadata.textproto" | |
| } | |
| test: { | |
| id: "enrollz-2" | |
| description: "enrollz test for TPM 1.2 Enrollment flow" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm12/README.md" | |
| exec: "feature/security/attestz/tests/enrollz_tpm12/metadata.textproto" | |
| } | |
| test: { | |
| id: "ENROLLZ-1" | |
| description: "enrollz test for TPM 2.0 HMAC-based Enrollment flow" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm20_hmac/README.md" | |
| exec: "feature/security/attestz/tests/enrollz_tpm20_hmac/metadata.textproto" | |
| } | |
| test: { | |
| id: "ENROLLZ-2" | |
| description: "enrollz test for TPM 1.2 Enrollment flow" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/security/attestz/tests/enrollz_tpm12/README.md" | |
| exec: "feature/security/attestz/tests/enrollz_tpm12/metadata.textproto" | |
| } |
| test: { | ||
| id: "Replay-1.0" | ||
| description: "Record/replay presession test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/presession_test/README.md" | ||
| exec: "feature/replay/tests/presession_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "Replay-1.1" | ||
| description: "Record/replay diff command trees test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/diff_command_trees/README.md" | ||
| exec: "feature/replay/tests/diff_command_trees/metadata.textproto" | ||
| } |
There was a problem hiding this comment.
The test IDs Replay-1.0 and Replay-1.1 do not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. The prefix should be at least two uppercase letters. Please correct these IDs and update the plan_id in the corresponding metadata.textproto files.
| test: { | |
| id: "Replay-1.0" | |
| description: "Record/replay presession test" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/presession_test/README.md" | |
| exec: "feature/replay/tests/presession_test/metadata.textproto" | |
| } | |
| test: { | |
| id: "Replay-1.1" | |
| description: "Record/replay diff command trees test" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/diff_command_trees/README.md" | |
| exec: "feature/replay/tests/diff_command_trees/metadata.textproto" | |
| } | |
| test: { | |
| id: "REPLAY-1.0" | |
| description: "Record/replay presession test" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/presession_test/README.md" | |
| exec: "feature/replay/tests/presession_test/metadata.textproto" | |
| } | |
| test: { | |
| id: "REPLAY-1.1" | |
| description: "Record/replay diff command trees test" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/replay/tests/diff_command_trees/README.md" | |
| exec: "feature/replay/tests/diff_command_trees/metadata.textproto" | |
| } |
| test: { | ||
| id: "gNMI-1.2" | ||
| description: "Benchmarking: Full Configuration Replace" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/tests/full_configuration_replace_test/README.md" | ||
| exec: "feature/gnmi/tests/full_configuration_replace_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "gNMI-1.3" | ||
| description: "Benchmarking: Drained Configuration Convergence Time" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/otg_tests/drained_configuration_convergence_time/README.md" | ||
| exec: "feature/gnmi/otg_tests/drained_configuration_convergence_time/metadata.textproto" | ||
| } |
There was a problem hiding this comment.
The test IDs gNMI-1.2 and gNMI-1.3 do not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Test IDs should start with uppercase letters. Please correct these IDs and update the plan_id in the corresponding metadata.textproto files.
| test: { | |
| id: "gNMI-1.2" | |
| description: "Benchmarking: Full Configuration Replace" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/tests/full_configuration_replace_test/README.md" | |
| exec: "feature/gnmi/tests/full_configuration_replace_test/metadata.textproto" | |
| } | |
| test: { | |
| id: "gNMI-1.3" | |
| description: "Benchmarking: Drained Configuration Convergence Time" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/otg_tests/drained_configuration_convergence_time/README.md" | |
| exec: "feature/gnmi/otg_tests/drained_configuration_convergence_time/metadata.textproto" | |
| } | |
| test: { | |
| id: "GNMI-1.2" | |
| description: "Benchmarking: Full Configuration Replace" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/tests/full_configuration_replace_test/README.md" | |
| exec: "feature/gnmi/tests/full_configuration_replace_test/metadata.textproto" | |
| } | |
| test: { | |
| id: "GNMI-1.3" | |
| description: "Benchmarking: Drained Configuration Convergence Time" | |
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/otg_tests/drained_configuration_convergence_time/README.md" | |
| exec: "feature/gnmi/otg_tests/drained_configuration_convergence_time/metadata.textproto" | |
| } |
| exec: " " | ||
| } | ||
| test: { | ||
| id: "gNMI-1.24" |
There was a problem hiding this comment.
The test ID gNMI-1.24 does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Test IDs should start with uppercase letters. Please correct this ID and update the plan_id in feature/gnmi/tests/leaflist_update_test/metadata.textproto accordingly.
| id: "gNMI-1.24" | |
| id: "GNMI-1.24" |
| exec: " " | ||
| } | ||
| test: { | ||
| id: "gNOI-7.1" |
There was a problem hiding this comment.
The test ID gNOI-7.1 does not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto. Test IDs should start with uppercase letters. Please correct this ID and update the plan_id in feature/gnoi/secure_boot/tests/bootconfig/metadata.textproto accordingly.
| id: "gNOI-7.1" | |
| id: "GNOI-7.1" |
| id: "bootz" | ||
| description: "General bootz bootstrap tests" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/secure_boot/tests/bootz/README.md" | ||
| exec: "feature/system/secure_boot/tests/bootz/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "System-1.1" | ||
| description: "System banner test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/system_base_test/tests/system_banner_test/README.md" | ||
| exec: "feature/system/system_base_test/tests/system_banner_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "System-1.2" | ||
| description: "System g protocol test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/system_base_test/tests/system_g_protocol_test/README.md" | ||
| exec: "feature/system/system_base_test/tests/system_g_protocol_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "System-1.3" | ||
| description: "System hostname test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/system_base_test/tests/hostname_test/README.md" | ||
| exec: "feature/system/system_base_test/tests/hostname_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "System-1.4" | ||
| description: "System time test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/system_base_test/tests/system_time_test/README.md" | ||
| exec: "feature/system/system_base_test/tests/system_time_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "System-1.5" | ||
| description: "System software-version test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/system/system_base_test/tests/system_software_version/README.md" | ||
| exec: "feature/system/system_base_test/tests/system_software_version/metadata.textproto" | ||
| } |
There was a problem hiding this comment.
Several test IDs in this block do not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto:
bootz: Does not match the pattern.System-1.1throughSystem-1.5: The prefix should be at least two uppercase letters.
Please correct these IDs to follow the specified format and update the plan_id in the corresponding metadata.textproto files.
| test: { | ||
| id: "gNMI-1.27" | ||
| description: "gNMI Sample Mode Test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/subscribe/tests/gnmi_sample_mode_test/README.md" | ||
| exec: "feature/gnmi/subscribe/tests/gnmi_sample_mode_test/metadata.textproto" | ||
| } | ||
| test: { | ||
| id: "GNMI-2" | ||
| description: "gnmi_subscriptionlist_test" | ||
| readme: "https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/subscribe/tests/gnmi_subscriptionlist_test/README.md" | ||
| exec: "feature/gnmi/subscribe/tests/gnmi_subscriptionlist_test/metadata.textproto" | ||
| } |
There was a problem hiding this comment.
The test IDs in this block do not conform to the required format [A-Z][A-Z]+\-[0-9]+(\.[0-9]+)? as specified in proto/testregistry.proto:
gNMI-1.27: Should start with uppercase letters (e.g.,GNMI-1.27).GNMI-2: Is missing a hyphen and number part (e.g.,GNMI-2.0).
Please correct these IDs and update the plan_id in the corresponding metadata.textproto files.
This reverts commit f25a886.
4 participants
No description provided.