Skip to content

Pulp import export support#455

Open
aidenfine wants to merge 7 commits intotheforeman:masterfrom
aidenfine:SAT-44059-pulp-import-export-support
Open

Pulp import export support#455
aidenfine wants to merge 7 commits intotheforeman:masterfrom
aidenfine:SAT-44059-pulp-import-export-support

Conversation

@aidenfine
Copy link
Copy Markdown

@aidenfine aidenfine commented Apr 17, 2026

Why are you introducing these changes? (Problem description, related links)

Jira

Add pulp content import/export arguments.

What are the changes introduced in this pull request?

You can pass in multiple args of either --content-import-path or --content-export-path as many times as you want, since these will be appended as a list. These folders will be automatically created.

How to test this pull request

pytest tests/pulp_test.py

Can also manually test these steps.

# create env
./foremanctl deploy --foreman-initial-admin-password=changeme --tuning development --content-import-path /mycontentpath/import --content-export-path /mycontentpath/export --content-export-path /mycontentpath/export2

# ssh 
vagrant ssh quadlet

# verify imports (should see a list with exported/imported paths)
sudo podman exec pulp-api pulpcore-manager shell -c "from django.conf import settings; print(settings.ALLOWED_IMPORT_PATHS)"

# exports
sudo podman exec pulp-api pulpcore-manager shell -c "from django.conf import settings; print(settings.ALLOWED_EXPORT_PATHS)"

@aidenfine aidenfine changed the title Sat-44059 pulp import export support Pulp import export support Apr 17, 2026
action: append_unique
type: FQDN
parameter: --certificate-cname
pulp_extra_import_paths:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user doesn't care that it's Pulp right? How about:

Suggested change
pulp_extra_import_paths:
content_extra_import_paths:

What makes them "extra" ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I used extra because users can add extra or additional import/export paths. Would you prefer something other than extra?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to remember, I think the installer uses this "extra" terminology.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, it uses additional. So there are the "you can use these out of the box paths" and then there are "user has some setup, they want to use these paths and accept the requirements to make this work".

Comment thread src/roles/pulp/README.md
- `pulp_database_ssl_mode`: Database SSL mode (default: `disabled`)
- `pulp_database_ssl_ca`: Path to the database SSL CA certificate on the control node (default: empty)

Usage Inside foremanctl
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this detail makes sense at this level. I'd rather see a section in the deployment document around content or add a content document.

Comment thread src/roles/pulp/README.md

Variables
---------
- `pulp_container_image`: Container image to use for Pulp (default: `quay.io/foreman/pulp`)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wary of storing defaults in the documentation here. The code can show those and reduce our overhead.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping someone would comment on the docs. I tried to follow similar patterns from the roles/hammer/README.md. I also feel this is kinda an info dump but wanted to get opinions on it. I don't mind removing this doc from the PR if it makes more sense not to have this

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the having of a README. I can see now that this is following roughly the outline from the hammer role. I think as long as we are consistent in the structure then keep it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we should define that roughly in https://github.com/theforeman/foremanctl/blob/master/docs/developer/playbooks-and-roles.md#shared-metadata-fragments

You do not have to do that here in this PR. I am noting this as a reminder.

Comment thread src/roles/pulp/README.md Outdated
Pulp Role
=========

Deploys and manages [Pulp](https://pulpproject.org/) as Podman containers.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Deploys and manages [Pulp](https://pulpproject.org/) as Podman containers.
Deploys and manages [Pulp](https://pulpproject.org/) as Podman quadlets.

Comment thread src/roles/pulp/tasks/main.yaml Outdated
- imports
- exports

- name: Create extra import directories
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume these would have to be mounted into the container?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I didn't think about that, I will add this

Comment thread src/roles/pulp/defaults/main.yaml Outdated
PULP_ALLOWED_IMPORT_PATHS: >-
{{ pulp_default_import_path + pulp_extra_import_paths }}
PULP_ALLOWED_EXPORT_PATHS: >-
{{ pulp_default_export_path + pulp_extra_export_paths }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally I have this:
PULP_ALLOWED_IMPORT_PATHS: >-
['/var/lib/pulp/sync_imports', '/var/lib/pulp/imports']
PULP_ALLOWED_EXPORT_PATHS: >-
['/var/lib/pulp/exports']

These would be the default paths..

Comment thread src/roles/pulp/defaults/main.yaml Outdated

pulp_enable_analytics: false

pulp_default_import_path:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says import_path but it's an array.
I wonder can we have a single variable for import and one for export, and they are just lists that appended to?

Copy link
Copy Markdown
Author

@aidenfine aidenfine Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea. So instead of 4 variables (2 default, 2 lists to hold user passed values). We could just have 2 lists both by default containing the default path and just append user passed values to these lists.

Am I understanding this correctly?

@ianballou
Copy link
Copy Markdown

ianballou commented Apr 17, 2026

Can we make these import/export arguments also available via forge deploy-dev ?

@ehelms
Copy link
Copy Markdown
Member

ehelms commented Apr 17, 2026

Can we make these import/export arguments also available via forge deploy-dev ?

Consider this method for defining them -- https://github.com/theforeman/foremanctl/blob/master/docs/developer/playbooks-and-roles.md#shared-metadata-fragments

@ianballou
Copy link
Copy Markdown

ianballou commented Apr 17, 2026

I just tried this out, seeing the following in my pulp-api service definition:

Environment='PULP_ALLOWED_IMPORT_PATHS=['"'"'/var/lib/pulp/imports'"'"', '"'"'/var/lib/pulp/my_imports'"'"', '"'"'/var/lib/pulp/my_exports'"'"', '"'"'/var/lib/pulp/exports'"'"']'
Environment='PULP_ALLOWED_EXPORT_PATHS=['"'"'/var/lib/pulp/exports'"'"', '"'"'/var/lib/pulp/my_exports'"'"']'

So far so good, assuming the escaping is correct. And yes, I like to be able to import from my export directory.

@ianballou
Copy link
Copy Markdown

Export and import worked! I was able to generate /var/lib/pulp/exports/Default_Organization/Export-Zoo-1/1.0/2026-04-17T17-37-04-00-00/ and import it back into another repository. Perfect.

@aidenfine
Copy link
Copy Markdown
Author

@ehelms I am not too sure on how to make sure the created import and export directories are mounted into the container. Can you point me into the right direction. Maybe somewhere in the code we handle a similar workflow

@ehelms
Copy link
Copy Markdown
Member

ehelms commented Apr 17, 2026

@ehelms I am not too sure on how to make sure the created import and export directories are mounted into the container. Can you point me into the right direction. Maybe somewhere in the code we handle a similar workflow

You'll need to do something like this -- https://github.com/theforeman/foremanctl/blob/master/src/roles/pulp/tasks/main.yaml#L137

Comment thread src/roles/pulp/defaults/main.yaml
@aidenfine aidenfine requested review from ehelms and evgeni April 24, 2026 18:03

pulp_enable_analytics: false

# pulp import/export paths
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant.

pulp_volumes: >-
{{
['/var/lib/pulp:/var/lib/pulp'] +
(pulp_import_paths | map('regex_replace', '^(.+)$', '\1:\1') | list) +
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these regexs doing?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets take one item from pulp_import_paths and call it string1, so string1="/foo/bar"
this regex turns string1 into the same format as the line above. ('/var/lib/pulp:/var/lib/pulp').

So after string1 runs through regex, we get "/foo/bar:/foo:bar"

foreman_development_github_username:
help: GitHub username to add as additional remote for git checkouts
action: store
pulp_import_paths:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could define these in a file like https://github.com/theforeman/foremanctl/blob/master/src/playbooks/_database_connection/metadata.obsah.yaml and then include that file in both deploy and deploy-dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants