Add foremanctl restore test cases in robottelo

[vijaysawant]

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Problem Statement

The foremanctl restore command has been implemented upstream (theforeman/foremanctl#549) but has no test coverage in robottelo. The existing test_foremanctl_backup.py only covers the backup functionality.
https://redhat.atlassian.net/browse/SAT-44898

Solution

Added 4 new test cases and a shared helper to tests/foreman/foremanctl/test_foremanctl_backup.py

• test_positive_backup_restore (E2E): Full backup → restore with --force → health check → content verification (custom repo, RH repo, pulp artifacts)
• test_positive_restore_validate: Validates --validate flag checks backup integrity without modifying the system
• test_negative_restore_baddir: Verifies restore fails with a non-existing backup directory
• test_negative_restore_no_force: Verifies restore fails without --force on an existing deployment (safety check)
• _create_backup(): Shared helper to avoid duplicating backup logic across restore tests

Related Issues

PRT test Cases example

trigger: test-robottelo
pytest: tests/foreman/foremanctl/test_foremanctl_backup.py -k 'test_positive_backup_restore or test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force'
deployment_type: container
theforeman:
  foremanctl: 549

Summary by Sourcery

Add test coverage for foremanctl restore alongside existing backup tests.

Tests:

• Add an end-to-end foremanctl backup and restore test that verifies health, content, and pulp artifacts after restore.
• Add a test to validate that foremanctl restore --validate checks backup integrity without modifying the system.
• Add negative tests to ensure restore fails for a non-existent backup directory and when run without --force on an existing deployment.
• Introduce a shared helper for creating backups to reduce duplication across backup and restore tests.

[sourcery-ai]

Reviewer's Guide

Extends foremanctl backup test module to cover the new restore command with a shared backup helper, one end-to-end positive backup/restore test, and three focused restore validation/negative tests.

File-Level Changes

Change	Details	Files
Add shared helper for running foremanctl backup used by restore tests.	Introduce _create_backup(sat, subdir) to run foremanctl backup with --wait-for-tasks and 30m timeout Assert backup command exit status is 0 and backup directory is created before returning its path	tests/foreman/foremanctl/test_foremanctl_backup.py
Add positive E2E backup/restore test that validates content and pulp artifacts after restore.	Create backup via _create_backup and verify backup contains expected files using get_exp_files Delete /var/lib/pulp/media/artifact to ensure restore repopulates artifacts Run foremanctl restore <backup_dir> --force with 30m timeout and assert success Run foremanctl health after restore and assert success Verify both custom and RH repositories from module_synced_repos are present by searching via API and comparing IDs Assert that pulp artifact files exist again under /var/lib/pulp/media/artifact	tests/foreman/foremanctl/test_foremanctl_backup.py
Add positive restore validation test for --validate flag.	Create backup via _create_backup Run foremanctl restore <backup_dir> --validate with 5m timeout Assert command exits with status 0, implying integrity validation passes without changes	tests/foreman/foremanctl/test_foremanctl_backup.py
Add negative test for restore with non-existing backup directory.	Construct random non-existent backup directory path under BACKUP_DIR Run foremanctl restore <bad_dir> --force with 5m timeout Assert command exits with non-zero status to confirm proper failure handling	tests/foreman/foremanctl/test_foremanctl_backup.py
Add negative test for restore without --force on an existing deployment.	Create backup via _create_backup Run foremanctl restore <backup_dir> without --force with 5m timeout Assert command exits with non-zero status to verify safety check preventing restore on existing deployment	tests/foreman/foremanctl/test_foremanctl_backup.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 2 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/foreman/foremanctl/test_foremanctl_backup.py" line_range="127-129" />
<code_context>
+        f'Some required backup files are missing. Expected: {expected_files}, Found: {files}'
+    )
+
+    module_target_sat.execute('rm -rf /var/lib/pulp/media/artifact')
+
+    result = module_target_sat.execute(
</code_context>
<issue_to_address>
**suggestion (testing):** Assert that pulp artifacts are actually removed before restore to prove restore repopulates them

After the `rm -rf`, add a check that the artifact directory is actually empty (for example, by asserting `wc -l` is 0). This ensures the later `> 0` assertion clearly demonstrates that restore repopulates the artifacts, and avoids false positives if the delete fails or the path changes.

```suggestion
    result = module_target_sat.execute('rm -rf /var/lib/pulp/media/artifact')
    assert result.status == 0, (
        f'Failed to remove pulp artifacts before restore:\n{result.stdout}\n{result.stderr}'
    )

    result = module_target_sat.execute(
        'test -d /var/lib/pulp/media/artifact && '
        'find /var/lib/pulp/media/artifact -type f | wc -l || echo 0'
    )
    artifact_count = int(result.stdout.strip() or 0)
    assert artifact_count == 0, (
        f'Expected no pulp artifacts before restore, but found {artifact_count}'
    )

    result = module_target_sat.execute(
```
</issue_to_address>

### Comment 2
<location path="tests/foreman/foremanctl/test_foremanctl_backup.py" line_range="154" />
<code_context>
+    )
+
+
+def test_positive_restore_validate(module_target_sat, setup_backup_tests):
+    """Verify foremanctl restore --validate checks backup integrity without making changes
+
</code_context>
<issue_to_address>
**issue (testing):** Test docstring claims no changes on `--validate`, but the test does not verify system state

The test currently only checks `status == 0`, so it doesn’t verify the “no changes” guarantee described in the docstring. Please either assert that a representative piece of system state (e.g., config checksum, DB value, or `foremanctl health` output) is unchanged before/after `--validate`, or soften the docstring so it doesn’t claim behavior the test doesn’t cover.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion (testing): Assert that pulp artifacts are actually removed before restore to prove restore repopulates them

After the rm -rf, add a check that the artifact directory is actually empty (for example, by asserting wc -l is 0). This ensures the later > 0 assertion clearly demonstrates that restore repopulates the artifacts, and avoids false positives if the delete fails or the path changes.

Suggested change

	module_target_sat.execute('rm -rf /var/lib/pulp/media/artifact')
	result = module_target_sat.execute(
	result = module_target_sat.execute('rm -rf /var/lib/pulp/media/artifact')
	assert result.status == 0, (
	f'Failed to remove pulp artifacts before restore:\n{result.stdout}\n{result.stderr}'
	)
	result = module_target_sat.execute(
	'test -d /var/lib/pulp/media/artifact && '
	'find /var/lib/pulp/media/artifact -type f | wc -l || echo 0'
	)
	artifact_count = int(result.stdout.strip() or 0)
	assert artifact_count == 0, (
	f'Expected no pulp artifacts before restore, but found {artifact_count}'
	)
	result = module_target_sat.execute(

[vijaysawant]

trigger: test-robottelo
pytest: tests/foreman/foremanctl/test_foremanctl_backup.py -k 'test_positive_backup_restore or test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force'
deployment_type: container
theforeman:
  foremanctl: 549

[Satellite-QE]

PRT Result

Build Number: 16279
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/foremanctl/test_foremanctl_backup.py -k test_positive_backup_restore or test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force --external-logging
Test Result : ===== 2 failed, 2 passed, 1 deselected, 31 warnings in 1820.01s (0:30:20) ======

[sjha4]

Can we cover this flow:
test_positive_backup_restore (E2E): Full backup → Change content -> restore with --force → health check → content verification (custom repo, RH repo, pulp artifacts)

[vijaysawant]

trigger: test-robottelo
pytest: tests/foreman/foremanctl/test_foremanctl_backup.py -k 'test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force'
deployment_type: container
theforeman:
  foremanctl: 549

[vijaysawant]

Can we cover this flow:
test_positive_backup_restore (E2E): Full backup → Change content -> restore with --force → health check → content verification (custom repo, RH repo, pulp artifacts)

Can you explain what does mean by "Change content" here in the workflow, do you meant sync the additional repos?

[Satellite-QE]

PRT Result

Build Number: 16300
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/foremanctl/test_foremanctl_backup.py -k test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force --external-logging
Test Result : ========== 2 deselected, 12 warnings, 3 errors in 4538.48s (1:15:38) ===========

[Satellite-QE]

PRT Result

Build Number: 16301
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/foremanctl/test_foremanctl_backup.py -k test_positive_restore_validate or test_negative_restore_baddir or test_negative_restore_no_force --external-logging
Test Result : ========== 2 deselected, 12 warnings, 3 errors in 4638.91s (1:17:18) ===========

[vijaysawant]

Automated PostgreSQL upgrades during restore processes cause timeouts by failing to wait for service dependencies.

[vijaysawant]

Can we cover this flow: test_positive_backup_restore (E2E): Full backup → Change content -> restore with --force → health check → content verification (custom repo, RH repo, pulp artifacts)

@sjha4 as per the suggestion and offline discussion, I have updated test_positive_backup_restore (E2E) test case, please have a look.