-
Notifications
You must be signed in to change notification settings - Fork 77
Add DPDK Provider (New) #2384
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rene-oromtz
wants to merge
9
commits into
main
Choose a base branch
from
add-dpdk
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add DPDK Provider (New) #2384
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
6611e5e
feat: add new provider for dpdk testing
rene-oromtz c44d465
modify runner to use snap based approach
rene-oromtz 2c0fa93
add download option for source tarball
rene-oromtz e33cd3c
use SNAP_COMMON for dpdk snap operations
rene-oromtz abfd0b2
test: add unit tests for dpdk provider
rene-oromtz 4c7bef1
add debian directory
rene-oromtz e9304b3
fix naming in test plan
rene-oromtz 93aa6d5
fix black formatting errors
rene-oromtz 63b104a
add suggestions made by reviewer
rene-oromtz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # python | ||
| *.egg-info | ||
| *.pyc | ||
| .coverage | ||
| MANIFEST | ||
| __pycache__ | ||
|
|
||
| # packaging folders | ||
| build | ||
| dist | ||
| po/*.pot | ||
|
|
||
| # tox working folder | ||
| /.tox |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| #!/usr/bin/env python3 | ||
| # This file is part of Checkbox. | ||
| # | ||
| # Copyright 2025 Canonical Ltd. | ||
| # | ||
| # Checkbox is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License version 3, | ||
| # as published by the Free Software Foundation. | ||
| # | ||
| # Checkbox is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with Checkbox. If not, see <http://www.gnu.org/licenses/>. | ||
|
|
||
| import socket | ||
|
|
||
| from collections import defaultdict | ||
| from pathlib import Path | ||
| from typing import List, Dict | ||
|
|
||
| # Dictionary with supported DPDK drivers per vendor | ||
| DPDK_SUPPORTED_DRIVERS = { | ||
| "Nvidia": ["mlx4_core", "mlx5_core"], | ||
| "Intel": [ | ||
| "cpfl", | ||
| "e1000", | ||
| "fm10k", | ||
| "i40e", | ||
| "ice", | ||
| "idpf", | ||
| "ifc", | ||
| "igc", | ||
| "ipn3ke", | ||
| "ixgbe", | ||
| ], | ||
| "Broadcom": ["bnxt"], | ||
| } | ||
|
|
||
|
|
||
| def get_dpdk_supported_drivers() -> Dict[str, List[str]]: | ||
| """Get a list of DPDK supported drivers available from network adapters. | ||
|
|
||
| :return: dpdk supported drivers and interfaces using them | ||
| """ | ||
| dpdk_drivers = defaultdict(list) | ||
|
|
||
| # Get all network interfaces | ||
| interfaces = socket.if_nameindex() | ||
|
|
||
| # Iterate over interfaces to get dpdk supported drivers if any | ||
| for _, iface in interfaces: | ||
| device_path = Path("/sys/class/net/{}/device/driver".format(iface)) | ||
| if device_path.exists() and device_path.is_symlink(): | ||
| driver_name = device_path.resolve().name | ||
| if any( | ||
| driver_name in drivers | ||
| for drivers in DPDK_SUPPORTED_DRIVERS.values() | ||
| ): | ||
| dpdk_drivers[driver_name].append(iface) | ||
|
|
||
| return dpdk_drivers | ||
|
|
||
|
|
||
| def print_drivers(drivers: Dict[str, List[str]]): | ||
| """Display DPDK supported drivers information | ||
|
|
||
| :param drivers: DPDK supported drivers and interfaces using them | ||
| """ | ||
| for driver, interfaces in drivers.items(): | ||
| print("name: {}".format(driver)) | ||
| print("interfaces: {}".format(" ".join(iface for iface in interfaces))) | ||
| print() | ||
|
|
||
|
|
||
| def main(): | ||
| supported_drivers = get_dpdk_supported_drivers() | ||
| if supported_drivers: | ||
| print("dpdk_supported: yes\n") | ||
| print_drivers(supported_drivers) | ||
| else: | ||
| print("dpdk_supported: no\n") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,215 @@ | ||
| #!/usr/bin/env python3 | ||
| """Script to execute DPDK Tests Suites on remote DTS controller. | ||
|
|
||
| Copyright 2025 Canonical Ltd. | ||
|
|
||
| This program is free software: you can redistribute it and/or modify | ||
| it under the terms of the GNU General Public License version 3, | ||
| as published by the Free Software Foundation. | ||
|
|
||
| This program is distributed in the hope that it will be useful, | ||
| but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| GNU General Public License for more details. | ||
|
|
||
| You should have received a copy of the GNU General Public License | ||
| along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| """ | ||
|
|
||
| import logging | ||
| import os | ||
| import json | ||
| import argparse | ||
| import shutil | ||
| import subprocess | ||
| from typing import Optional, Dict, Any | ||
| from pathlib import Path | ||
|
|
||
| DPDK_SNAP_BIN = "/snap/bin/dpdk-dts" | ||
| DPDK_DTS_SNAP_COMMON = Path("/var/snap/dpdk-dts/common") | ||
| DPDK_CONFIG_SNAP_PATH = DPDK_DTS_SNAP_COMMON / "dts_config.yaml" | ||
| DEFAULT_OUTPUT_DIR = DPDK_DTS_SNAP_COMMON / "dpdk_test_results" | ||
| DEFAULT_TIMEOUT = 600 | ||
|
|
||
|
|
||
| class DTSRunner: | ||
| """Class to execute snap-based DPDK Test Suite (DTS)""" | ||
|
|
||
| def __init__(self, test_suite: str, config_file: Path): | ||
| """Initialize class attributes.""" | ||
| self.test_suite = test_suite | ||
| self.config_file = config_file | ||
|
|
||
| def run_test_suite( | ||
| self, | ||
| verbose: bool, | ||
| ) -> None: | ||
| """Run specified test suite in DTS controller | ||
|
|
||
| :param verbose: verbosity level on test suite execution | ||
| """ | ||
| output_dir = DEFAULT_OUTPUT_DIR / self.test_suite | ||
| if output_dir.exists(): | ||
| shutil.rmtree(output_dir) | ||
| output_dir.mkdir(parents=True) | ||
|
|
||
| # Copy config file to SNAP_COMMON so the strict snap can access it | ||
| shutil.copy(self.config_file, DPDK_CONFIG_SNAP_PATH) | ||
|
pedro-avalos marked this conversation as resolved.
|
||
| dts_command = [ | ||
| DPDK_SNAP_BIN, | ||
| "--test-suite", | ||
| self.test_suite, | ||
| "--config-file", | ||
| str(DPDK_CONFIG_SNAP_PATH), | ||
| "--output-dir", | ||
| str(output_dir), | ||
| ] | ||
| if verbose: | ||
| dts_command.append("--verbose") | ||
|
|
||
| logging.info( | ||
| "Starting execution of %s", | ||
| self.test_suite, | ||
| ) | ||
| try: | ||
| subprocess.run( | ||
| dts_command, | ||
| check=True, | ||
| timeout=DEFAULT_TIMEOUT, | ||
| ) | ||
| except subprocess.CalledProcessError as exc: | ||
| logging.error("DPDK Test Suite execution failed: %s", exc) | ||
| raise | ||
| except subprocess.TimeoutExpired as exc: | ||
| logging.error("DPDK Test Suite execution timed out: %s", exc) | ||
| raise | ||
|
|
||
| logging.info("DTS Test Suite Run completed") | ||
|
|
||
| def get_results(self) -> Optional[Dict[str, Any]]: | ||
| """Get the results from test suite execution. | ||
|
|
||
| :return: Results in json format if any returned during test execution | ||
| """ | ||
|
|
||
| logging.info("Getting test suite results") | ||
| results_path = DEFAULT_OUTPUT_DIR / self.test_suite / "results.json" | ||
| if not results_path.is_file(): | ||
| logging.warning("No results file found at %s", results_path) | ||
| return None | ||
| with results_path.open("r") as f: | ||
| results = f.read() | ||
| if results: | ||
| try: | ||
| return json.loads(results) | ||
| except ValueError: | ||
| logging.error("Unable to parse results file as JSON") | ||
|
|
||
| return None | ||
|
|
||
| def print_results(self) -> bool: | ||
| """Print tests results from execution. | ||
|
|
||
| :return: True if results were printed, False otherwise | ||
| """ | ||
|
|
||
| test_results = self.get_results() | ||
| if not test_results: | ||
| return False | ||
|
|
||
| # Print Test Suite Results and Summary | ||
| print("\nDPDK Test Results") | ||
| print("-" * 50) | ||
|
|
||
| try: | ||
| for test_run in test_results["test_runs"]: | ||
| for test_suite in test_run["test_suites"]: | ||
| suite_name = test_suite["test_suite_name"] | ||
| print("\nTest Suite: {}".format(suite_name)) | ||
| print("{:<30} {}".format("Test Case", "Result")) | ||
| print("-" * 40) | ||
|
|
||
| # Print test case details | ||
| for test_case in test_suite["test_cases"]: | ||
| print( | ||
| "{:<30} {}".format( | ||
| test_case["test_case_name"], | ||
| test_case["result"], | ||
| ) | ||
| ) | ||
|
|
||
| # Print summary | ||
| print("\nSummary:") | ||
| print("-" * 40) | ||
| for status, count in test_suite["summary"].items(): | ||
| print("{}: {}".format(status, count)) | ||
| except KeyError: | ||
| # If unable to pretty print, dump the test results | ||
| print(json.dumps(test_results, indent=2)) | ||
|
|
||
| return True | ||
|
|
||
| def cleanup(self) -> None: | ||
| """Remove config file copied to SNAP_COMMON after test execution.""" | ||
| if DPDK_CONFIG_SNAP_PATH.is_file(): | ||
| try: | ||
| DPDK_CONFIG_SNAP_PATH.unlink() | ||
| except OSError as exc: | ||
| logging.warning( | ||
| "Failed to remove config file %s: %s", | ||
| DPDK_CONFIG_SNAP_PATH, | ||
| exc, | ||
| ) | ||
|
|
||
|
|
||
| def parse_args(): | ||
| """Parses command-line arguments.""" | ||
| parser = argparse.ArgumentParser(description="DPDK Test Suite Execution") | ||
| parser.add_argument( | ||
| "-v", | ||
| "--verbose", | ||
| action="store_true", | ||
| help="Increase logging level in test suite execution", | ||
| ) | ||
| parser.add_argument( | ||
| "-T", "--test-suite", required=True, help="Specified Test Suite to run" | ||
| ) | ||
| args = parser.parse_args() | ||
|
|
||
| return args | ||
|
|
||
|
|
||
| def main(): | ||
| """Main entrypoint to the program.""" | ||
| args = parse_args() | ||
| logging.basicConfig(level=logging.INFO) | ||
| dts_config = os.getenv("DTS_CONFIG_FILE") | ||
|
|
||
| # Print environment variables used for test suite run | ||
| logging.info("DTS_CONFIG_FILE: %s", dts_config) | ||
|
|
||
| # Validate configuration before test suite execution | ||
| if not dts_config or not Path(dts_config).is_file(): | ||
| raise SystemExit("Unable to locate config file for test execution.") | ||
|
|
||
| # Run snap-based DPDK Test Suite | ||
| dts_runner = DTSRunner( | ||
| test_suite=args.test_suite, | ||
| config_file=Path(dts_config), | ||
| ) | ||
| try: | ||
| dts_runner.run_test_suite(args.verbose) | ||
| except (subprocess.CalledProcessError, subprocess.TimeoutExpired): | ||
| # Attempt to print test results before exit | ||
| dts_runner.print_results() | ||
| raise SystemExit("Test Suite execution failed") | ||
| finally: | ||
| dts_runner.cleanup() | ||
|
|
||
| # Print test suite execution results | ||
| if not dts_runner.print_results(): | ||
| raise SystemExit("No test results found") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| checkbox-provider-dpdk (0.1.0) UNRELEASED; urgency=medium | ||
|
|
||
| * Initial release | ||
|
|
||
| -- Rene Orozco Martinez <rene.orozco@canonical.com> Tue, 10 Mar 2026 15:10:44 -0600 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| debian/tmp/usr/lib/checkbox-provider-dpdk/* | ||
| debian/tmp/usr/share/checkbox-provider-dpdk/* | ||
| debian/tmp/usr/share/plainbox-providers-1/*.provider | ||
| usr/bin/test-dpdk |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 9 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| Source: checkbox-provider-dpdk | ||
| Section: utils | ||
| Priority: optional | ||
| Maintainer: Checkbox Developers <checkbox-devel@lists.ubuntu.com> | ||
| Uploaders: Rene Orozco Martinez <rene.orozco@canonical.com> | ||
| Build-Depends: checkbox-ng, | ||
| debhelper (>= 9), | ||
| intltool, | ||
| pkg-config, | ||
| checkbox-provider-base, | ||
| python3, | ||
| python3-checkbox-ng, | ||
| python3-debian, | ||
| python3-setuptools | ||
| Standards-Version: 3.9.6 | ||
|
|
||
| Package: checkbox-provider-dpdk | ||
| Architecture: any-amd64 arm64 | ||
| Depends: checkbox-provider-base, ${plainbox:Depends}, ${misc:Depends} | ||
| X-Plainbox-Provider: yes | ||
| Description: Checkbox provider for dpdk testing | ||
| This package provides a test plan and tooling to be used by Canonical for the | ||
| testing and certification of dpdk devices in "server" computer systems. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ | ||
| Upstream-Name: checkbox-provider-dpdk | ||
| Source: https://launchpad.net/checkbox-provider-dpdk | ||
|
|
||
| Files: * | ||
| Copyright: Copyright 2020 Canonical Ltd. | ||
| License: GPL-3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| snap install dpdk-dts | ||
| snap connect dpdk-dts:ssh-keys |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| #!/usr/bin/make -f | ||
| export SRCTOP=providers/dpdk | ||
|
|
||
| %: | ||
| dh $@ --sourcedirectory=$(SRCTOP) | ||
|
|
||
| override_dh_auto_clean: | ||
| python3 $(SRCTOP)/manage.py clean | ||
|
|
||
| override_dh_auto_build: | ||
| python3 $(SRCTOP)/manage.py build | ||
| python3 $(SRCTOP)/manage.py i18n --dont-update-pot --dont-merge-po | ||
|
|
||
| override_dh_auto_test: | ||
| python3 $(SRCTOP)/manage.py validate | ||
|
|
||
| override_dh_auto_install: | ||
| python3 $(SRCTOP)/manage.py install \ | ||
| --prefix=/usr --layout=unix \ | ||
| --root=$(CURDIR)/debian/tmp/ | ||
|
|
||
| override_dh_gencontrol: | ||
| python3 $(SRCTOP)/manage.py packaging | ||
| dh_gencontrol |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 3.0 (native) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a great idea, I need to do this to the GPGPU test suite too...