Skip to content

Add lgalloc support#36

Merged
bobbyiliev merged 7 commits intomainfrom
add-lgalloc-support
Apr 11, 2025
Merged

Add lgalloc support#36
bobbyiliev merged 7 commits intomainfrom
add-lgalloc-support

Conversation

@bobbyiliev
Copy link
Copy Markdown
Collaborator

@bobbyiliev bobbyiliev commented Mar 28, 2025

Open to suggestions around the bootstrap script and the daemonset!

@bobbyiliev bobbyiliev marked this pull request as draft March 28, 2025 20:38
@bobbyiliev bobbyiliev force-pushed the add-lgalloc-support branch from 94ca04c to 009912f Compare March 28, 2025 20:52
Comment thread modules/gke/main.tf Outdated
@bobbyiliev bobbyiliev force-pushed the add-lgalloc-support branch 2 times, most recently from f95c785 to c93cd38 Compare April 9, 2025 16:41
@bobbyiliev bobbyiliev force-pushed the add-lgalloc-support branch from c93cd38 to f04c0eb Compare April 9, 2025 19:36
@bobbyiliev bobbyiliev changed the title [WIP] Add initial lgalloc support Add lgalloc support Apr 9, 2025
@bobbyiliev bobbyiliev marked this pull request as ready for review April 9, 2025 19:41
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread main.tf
# Disk support configuration
disk_config = {
install_openebs = var.enable_disk_support ? lookup(var.disk_support_config, "install_openebs", true) : false
run_disk_setup_script = var.enable_disk_support ? lookup(var.disk_support_config, "run_disk_setup_script", true) : false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just base this one entirely on var.enable_disk_support and not have this extra var?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the idea was that users only set enable_disk_support, and the rest use defaults. I added disk_support_config just for extra flexibility if we ever need to override things. Happy to simplify if we want to keep it more opinionated though, up to you!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part of the PR is more or less a copy and paste from the AWS implementation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.... let's just keep this

Comment thread main.tf
Comment thread modules/gke/bootstrap.sh
Comment on lines +6 to +15
# Install required tools
if command -v apt-get >/dev/null 2>&1; then
apt-get update
apt-get install -y lvm2
elif command -v yum >/dev/null 2>&1; then
yum install -y lvm2
else
echo "No package manager found. Please install required tools manually."
exit 1
fi
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely version lock this, or if there aren't dependencies, install from a binary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pinned this to a specific version.

Regarding the dependencies:

Reading package lists...
+ apt-get install -y lvm2=2.03.11-2.1
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  dmeventd dmsetup libaio1 libbsd0 libdevmapper-event1.02.1 libdevmapper1.02.1
  libedit2 libexpat1 liblvm2cmd2.03 libmd0 thin-provisioning-tools

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is extremely unlikely to work long-term. If you want to pin it, you need to bake it into an image. The upstream repos will likely not contain that specific version for long.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but I thought that we did not want to maintain our own image because of security concerns?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we can't pin the version this way, and the security concerns don't go away just because you're using someone else's image.

We can either:

  1. Unpin both the lvm2 package version and use a moving target debian image tag.
  2. Maintain our own image and use dependabot (or similar) to keep the image up to date.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just removed the version pin.

When I was originally working on that custom container, I got pushback because of concerns that a lot of vulnerability scanner noise could come from bootstrap containers.

But if we are all fine with this, I am happy to work on that bootstrap Docker image.

Comment thread modules/gke/bootstrap.sh Outdated
Comment thread modules/gke/bootstrap.sh Outdated
@bobbyiliev bobbyiliev force-pushed the add-lgalloc-support branch from cb553bc to 8f767b2 Compare April 10, 2025 11:38
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread modules/gke/bootstrap.sh Outdated
Comment thread modules/gke/bootstrap.sh
Comment thread modules/gke/bootstrap.sh Outdated
Comment thread modules/gke/main.tf
Comment on lines +250 to +259
resources {
limits = {
cpu = "200m"
memory = "256Mi"
}
requests = {
cpu = "100m"
memory = "128Mi"
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory requests and limits should always be equal.

CPU limits are generally frowned upon, as CPU can be returned extremely quickly. In the case of this bootstrap script, it will likely be idle for most of the time, so limiting CPU is likely not helpful.

Comment thread modules/gke/main.tf Outdated
Comment thread modules/gke/main.tf

init_container {
name = "disk-setup"
image = "debian:bullseye-20250407-slim"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably bake LVM into an image, or find one that comes with it.

We may be able to simplify this module quite a bit if we also bake the bootstrap script into the image.

Comment thread modules/gke/main.tf Outdated
memory = "128Mi"
}
requests = {
cpu = "100m"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still want at least some cpu request.

Comment thread variables.tf Outdated
@@ -221,10 +221,6 @@ variable "disk_support_config" {
openebs_namespace = optional(string, "openebs")
storage_class_name = optional(string, "openebs-lvm-instance-store-ext4")
storage_class_provisioner = optional(string, "local.csi.openebs.io")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The storage_class_provisioner should be constant.

Copy link
Copy Markdown
Contributor

@alex-hunt-materialize alex-hunt-materialize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably bake our own image. Some customers may block outbound access to package repos, which will prevent installation of LVM.

@bobbyiliev bobbyiliev merged commit 0991379 into main Apr 11, 2025
2 checks passed
@bobbyiliev bobbyiliev deleted the add-lgalloc-support branch April 11, 2025 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants