Implement Nats K/V Streamlining into OPA memory#1
Merged
Conversation
…port. Add Go module and dependencies, along with README documentation and CI workflow setup. Implement core caching logic, group management, and path routing for NATS Key-Value store integration.
…nce group management. Update logging to use OPA's logging package. Adjust tests to reflect changes in configuration structure, focusing on group regex patterns and single group settings.
Enhance GroupCache to utilize gjson for JSON document management, allowing for efficient retrieval and updates. Implement LRU caching for recently accessed paths, with automatic invalidation of affected cache entries upon updates. Introduce comprehensive tests to validate functionality, including cache eviction and document persistence. Update go.mod and go.sum to include necessary dependencies for gjson and sjson.
PER-12575 Adapt the OPA plugin to hold top K schemas from Nats K/V
The plugin should hold some sort of LRU Cache that is updated automatically from Nats K/V of the schemas used. The goal is to create an opa plugin that implements the Store and LRU caches the data from Nats K/V into the in-memory store based on path |
…andling. Update configuration files to set max_bucket_watchers to 1. Enhance EnsureBucketLoaded method to differentiate between root and non-root buckets, and adjust related methods for consistent handling of bucket data loading and OPA store interactions.
…iple files. Remove unused go-vet hook, ensure consistent formatting, and eliminate trailing whitespace in various YAML, Go, and Markdown files.
…ogy consistency. Update comments and method names to replace "group" with "bucket" for clarity. Adjust error messages and logging to reflect the new terminology. Simplify keyToPath function by removing unnecessary error handling.
Introduce new test files for additional functionality, including tests for DataTransformer, NATSClient, and MockStore. Cover various scenarios such as edge cases, error handling, and configuration validation. Refactor existing tests for clarity and consistency, ensuring robust coverage of key functionalities in the NATS store package.
…t panic Refactor the TestMockStore_Comprehensive function to ensure that the Register method does not panic by asserting the absence of errors during registration. This change enhances the robustness of the test suite for the NATS store package.
Enhance the CI workflow by adding a NATS service setup for JetStream, including health checks and configuration management. Introduce a new configuration file for JetStream and update the PluginFactory to manage bucket data more effectively. This change improves testing capabilities and ensures proper configuration of the NATS server during CI runs.
Enhance the NATS store plugin by adding a default logger instantiation in the New method of the PluginFactory. This change ensures that a logger is available for testing scenarios when none is provided, improving the robustness of the plugin's logging capabilities.
Introduce a new golangci-lint configuration file to define linting rules and settings for the project. Update the CI workflow to include a NATS service setup with health checks, ensuring proper environment for linting. Additionally, refactor the NATS store plugin to improve error handling for bucket name and key type assertions, enhancing robustness and clarity in the codebase.
Refactor the golangci-lint configuration to enhance linting rules and settings, including increased confidence for revive and added staticcheck. Update the GitHub Actions workflow to use golangci-lint version v2.3.1, ensuring compatibility and improved linting performance.
Introduce a new Dockerfile for building the OPA binary with the NATS store plugin. The Dockerfile includes multi-stage builds for dependency management and a non-root user setup. Additionally, add main.go to implement the OPA runtime with the NATS plugin, enhancing the project's capabilities. Update docker-compose.yaml to reference the new Dockerfile and adjust the command for running the OPA server.
omer9564
changed the title
Update the README.md to include a detailed project layout, highlighting the directory structure and key components. Add new configuration options for the NATS plugin, including `max_bucket_watchers` and `root_bucket`. Introduce built-in functions for interacting with NATS K/V, along with usage examples. Revise usage instructions for the custom OPA binary and Docker integration, improving clarity and completeness of documentation.
Rename example directory from `handledpaths` to `opa-nats` to better reflect its purpose. Update README.md to point to the new example location and provide clearer instructions for using the OPA with NATS plugin. Add new configuration files for Docker Compose and OPA, and include a test policy in Rego to verify functionality. Remove the old `main.go` example to streamline the project structure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enhance GroupCache to utilize gjson for JSON document management, allowing for efficient retrieval and updates. Implement LRU caching for recently accessed paths, with automatic invalidation of affected cache entries upon updates. Introduce comprehensive tests to validate functionality, including cache eviction and document persistence. Update go.mod and go.sum to include necessary dependencies for gjson and sjson.