Circleci project setup -testing

[dhairs]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @dhairs, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new CircleCI configuration file (.circleci/config.yml) to establish a continuous integration pipeline. The primary goal is to automate the build, unit testing, and integration testing processes for the project using Bazel and Docker Compose, ensuring code quality and functionality before merging. This setup includes environment preparation, service orchestration, and comprehensive test execution.

Highlights

• New CircleCI Configuration: A new .circleci/config.yml file has been added to establish a continuous integration pipeline for the project.
• Bazel Build and Test Job: A bazel-build-test job is introduced, responsible for building the project, running unit tests, and executing integration tests using Bazel.
• Docker Compose Integration: The CI pipeline now includes steps to set up and tear down Docker Compose services (Kafka and Ingest) to provide an environment for integration testing.
• Cache Management: Bazel cache restoration and saving steps are included to optimize build times during subsequent CI runs.
• Automated Testing: The configuration defines distinct steps for running unit tests (excluding integration/docker/requirements tags) and integration tests (specifically targeting integration tags) with necessary environment variables.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a CircleCI configuration to automate the build and test process. The setup is comprehensive, covering Bazel builds, caching, unit tests, and integration tests with Docker services. However, there are a few critical issues that need to be addressed. There are several instances of a typo ('telemtry' instead of 'telemetry') that will cause the pipeline to fail. Additionally, there's a security concern with writing secrets to a file, and a brittle sleep command that could lead to flaky builds. My review includes suggestions to fix these issues.

[gemini-code-assist]

Writing secrets to a file on the filesystem, even temporarily, is a security risk. The .env file containing POSTGRES_USER and POSTGRES_PASSWORD could be exposed. A more secure approach is to have docker compose use environment variables directly from the CI job's shell. This would require modifying your docker-compose.yml files to remove the env_file directive. Since those files are not part of this PR, I'm flagging this as a high-severity issue to be addressed. Once the compose files are updated, this entire run step can be removed.

[gemini-code-assist]

Using a fixed sleep 10 is brittle and can make your build flaky. If services take longer to start, subsequent steps might fail. It's better to actively poll for services to become ready. The docker compose up --wait command is a good start, but it relies on healthcheck directives in your compose files, which may not be configured. A more robust method is to loop and check if service ports are accessible.

            # Wait up to 60 seconds for Kafka and Postgres to be available
            timeout 60 bash -c 'until </dev/tcp/localhost/9092; do sleep 1; done'
            timeout 60 bash -c 'until </dev/tcp/localhost/5432; do sleep 1; done'
            echo "Services are ready."
            docker ps