Skip to content

Add an option to copy FoundationDB cluster files to a writable temporary file #19684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jon-signal
Copy link
Contributor

What does this PR do?

This pull request adds an option to FoundationDB integration instances to allow the check to make a writable copy of the cluster file before passing it to the FoundationDB client. This closes #19677.

Motivation

Please see #19677 for a detailed description of the problem, but in short, FoundationDB clients want a writable copy of the cluster file. It can be hard to provide a writable copy when running in a Kubernetes environment (the most common way to get the cluster file in that case is by mounting a ConfigMap as a file, but that will always be read-only), and so this option provides a mechanism to satisfy the FoundationDB client without jumping through Terrible Ops Hoops™.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

cluster_file = self.instance.get('cluster_file')

if self.instance.get('copy_cluster_file'):
_, cluster_file = tempfile.mkstemp(suffix=".cluster")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a "right place" to clean up this temporary file (or other long-lived resources)? I notice that this check doesn't have a mechanism for closing the FoundationDB client, so perhaps just leaving the temporary file is the right thing to do.

Comment on lines +41 to +43
@pytest.fixture
def copy_cluster_file_instance():
return COPY_CLUSTER_FILE_INSTANCE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am very new to Python unit tests and recognize that this may not be the right way to do things. I'm very open to feedback!

Copy link

codecov bot commented Feb 21, 2025

Codecov Report

Attention: Patch coverage is 66.66667% with 4 lines in your changes missing coverage. Please review.

Project coverage is 88.63%. Comparing base (f5532e4) to head (de59b5b).

Additional details and impacted files
Flag Coverage Δ
activemq ?
cassandra ?
foundationdb 81.88% <66.66%> (-0.58%) ⬇️
hive ?
hivemq ?
hudi ?
ignite ?
jboss_wildfly ?
kafka ?
presto ?
solr ?

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jon-signal jon-signal force-pushed the copy_foundationdb_cluster_file_option branch 2 times, most recently from 5509e1f to 91a010b Compare February 21, 2025 20:26
@hestonhoffman hestonhoffman added the editorial review Waiting on a more in-depth review from a docs team editor label Feb 21, 2025
@hestonhoffman
Copy link
Contributor

Hello from the docs team 👋
I'm going to hold off on reviewing the stuff in the spec file until the Agent team has taken a look, in case their review necessitates changes to that file. I'll check back in, but feel free to ping me after the Agent team's review if I haven't reviewed yet.

@jon-signal jon-signal force-pushed the copy_foundationdb_cluster_file_option branch from 91a010b to de59b5b Compare March 18, 2025 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Difficult to provide FoundationDB cluster files in Kubernetes environments
2 participants