Kyma-Companion: Action Workflow Improvements

[danjiawork]

Description

Several improvement possibilities are detected while working on Kyma-Companion:

• Separate API Tests into a dedicated workflow
  • Currently, API tests are run in the same job as evaluation tests within .github/workflows/evaluation-test-reusable.yaml
  • Team Proposal: Extract API tests into a separate, lightweight workflow
• Adopt k3d-action for cluster setup
  • Replace manual k3d deployment in the evaluation test workflow with the k3d-action
  • Reference: k3d-action

Reasons

• API Test Separation
  Benefits:

  • Aligns with team feedback to treat API tests as lightweight, unit-test-like checks
  • Clearer pipeline structure and labeling

  Trade-offs

  • Requires duplicating Companion setup logic across workflows/jobs (Each job uses a separate isolated runner, so the companion container from the k3d setup from another job cannot be reused)
  • API tests include /api/tools/kyma/search endpoint tests that incur LLM token costs
  • Question: Should we accept this duplication for better separation of concerns?

• k3d-action Adoption
  Benefits:

  • Support for specific k3d versions (as currently required).
  • Simplifies workflow k3d setup code (~70% reduction).
  • Includes built-in error handling, improving reliability.