Adds unit tests for serialization

[JyotinderSingh]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @JyotinderSingh, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the test coverage and reliability of the keras_remote library by introducing a suite of new unit and integration tests. The changes focus on critical components such as accelerator configuration parsing, geographical constant utilities, and the serialization/packaging mechanisms. This ensures that core functionalities behave as expected across various scenarios and environments, improving overall code quality and maintainability.

Highlights

• New Test Infrastructure: Introduced new conftest.py files at both the root and keras_remote levels to provide shared and Keras Remote-specific pytest fixtures, streamlining test setup.
• Accelerator Parsing Tests: Added comprehensive unit tests for the accelerators module, thoroughly covering GPU and TPU parsing logic, configuration validation, and registry integrity checks.
• Constants Utility Tests: Implemented unit tests for constants utilities, verifying the correctness of zone-to-region/location conversions and the retrieval of default zone settings, including environment variable handling.
• Serialization and Packaging Tests: Developed extensive unit tests for the packager module, ensuring the correct functionality of directory zipping and robust payload serialization using cloudpickle for various Python objects.
• Build Configuration Updates: Updated pyproject.toml to include necessary test dependencies (pytest, pytest-cov, pytest-mock, pytest-timeout), configure pytest for proper test discovery and markers, and refine ruff linting rules for test files.
• End-to-End Test Framework: Established a framework for end-to-end tests with conditional skipping logic based on environment variables, allowing for controlled execution of resource-intensive tests.
• Packager Integration Tests: Included integration tests to validate the full zip and extract roundtrip process for project workspaces, ensuring that packaged code can be correctly deployed and imported.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• conftest.py
  • Added sample_function fixture for serialization tests.
  • Added gcp_env fixture to set standard GCP environment variables for tests.
• keras_remote/conftest.py
  • Added mock_storage_client fixture to mock google.cloud.storage.Client.
  • Added mock_kube_config fixture to mock Kubernetes config loading.
  • Added mock_batch_v1 fixture to mock kubernetes.client.BatchV1Api.
• keras_remote/core/test_accelerators.py
  • Added unit tests for direct GPU parsing (e.g., 'l4').
  • Added unit tests for multi-count GPU parsing (e.g., 'a100x4').
  • Added unit tests for GPU alias parsing (e.g., 'nvidia-tesla-t4').
  • Added unit tests for error handling with invalid GPU counts.
  • Added unit tests for bare TPU parsing (e.g., 'v5litepod').
  • Added unit tests for TPU parsing with specific chip counts (e.g., 'v3-8').
  • Added unit tests for TPU parsing with topology specifications (e.g., 'v5litepod-2x2').
  • Added unit tests for error handling with invalid TPU configurations.
  • Added unit tests to verify full TPU configuration fields.
  • Added a unit test for CPU parsing.
  • Added unit tests for input normalization and general error handling in accelerator parsing.
  • Added unit tests for accelerator category retrieval.
  • Added unit tests to ensure the integrity of GPU and TPU registries.
• keras_remote/test_constants.py
  • Added parameterized unit tests for zone_to_region conversion, including fallback scenarios.
  • Added parameterized unit tests for zone_to_ar_location conversion.
  • Added unit tests for get_default_zone to verify environment variable precedence and default values.
  • Added a unit test to confirm the values of DEFAULT_ZONE and DEFAULT_REGION constants.
• keras_remote/utils/test_packager.py
  • Added unit tests for zip_working_dir to ensure all files are included, and .git and __pycache__ directories are excluded.
  • Added unit tests for zip_working_dir to verify preservation of nested directory structures.
  • Added a unit test for zipping an empty directory.
  • Added unit tests for save_payload to verify roundtrip serialization of simple functions, including arguments, keyword arguments, environment variables, lambdas, and closures.
  • Added unit tests for save_payload to verify roundtrip serialization of NumPy arrays and complex data structures.
• pyproject.toml
  • Added a new test dependency group with pytest, pytest-cov, pytest-mock, and pytest-timeout.
  • Updated the dev dependency group to include keras-remote[test].
  • Modified tool.ruff.lint.per-file-ignores to suppress T201 warnings in test files.
  • Added tool.pytest.ini_options to define testpaths and register an e2e marker for end-to-end tests.
• tests/e2e/conftest.py
  • Added pytest_collection_modifyitems hook to conditionally skip e2e marked tests if the E2E_TESTS environment variable is not set.
  • Added gcp_project fixture to retrieve the GCP project from an environment variable, skipping tests if it's not set.
• tests/integration/test_packager_roundtrip.py
  • Added integration tests for zip_working_dir to verify that zipping and extracting preserves file content.
  • Added integration tests to confirm that an extracted workspace can be successfully added to sys.path for module imports.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/tests.yaml

Activity

• No human activity (comments, reviews) detected in this pull request.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[JyotinderSingh]

/gemini review

[gemini-code-assist]

Code Review

The pull request introduces unit and integration tests for the packager utility, covering zip_working_dir and save_payload functionalities. The tests appear comprehensive, covering various scenarios like file inclusion/exclusion, nested structures, and different data types for serialization. The use of tempfile.TemporaryDirectory ensures proper cleanup of temporary resources. Overall, the changes improve the robustness and reliability of the packager module.