feat: parallelize deploy-and-test pipeline

[decko]

Summary

• Split the monolithic pulp-functional-tests task into 3 parallel Tekton tasks (functional-tests-rpm, functional-tests-core-service, functional-tests-npm-maven)
• Decouple push-api-json-files-to-pulp from test results — its real dependency is install-bindings, not test output
• Move test dependency installation (pytest, gnupg, requirements) into install-bindings to avoid duplicating setup across parallel tasks
• Fix JUnit XML overwriting bug — each test group now writes to a unique filename
• Use unique pytest cache_dir per task to prevent concurrent cache corruption
• Reduce wait-for-api-schema retry interval from 15s to 5s (same 900s max)

Before (fully sequential)

install-bindings → pulp-functional-tests → push-api-json-files-to-pulp

After (parallel fan-out)

                 ├── functional-tests-rpm
install-bindings ├── functional-tests-core-service
                 ├── functional-tests-npm-maven
                 └── push-api-json-files-to-pulp

Estimated time savings: 15-25 minutes (pipeline currently takes ~1 hour).

Test plan

• First PR run: verify all 4 post-install tasks start simultaneously in Konflux
• All test groups pass independently
• If one test group fails, others continue and teardown still fires
• JUnit XML files have correct unique names
• Monitor pod memory usage during parallel execution for OOMKill
• Compare total pipeline wall-clock time before vs after

Risks

• Pod memory pressure: 3 concurrent pytest sessions (one with -n 8) may exceed pod memory limits sized for serial execution. Mitigation: monitor first run, increase limits or reduce -n 8 to -n 4 if OOM occurs.
• Pulpcore worker pool contention: Only 2 workers in ephemeral env. Tasks queue but could slow down. Consider scaling to 3-4 workers if timeout flakes appear.

🤖 Generated with Claude Code

Summary by Sourcery

Parallelize the Tekton deploy-and-test pipeline and adjust related test execution setup.

CI:

• Split the monolithic pulp-functional-tests Tekton task into three parallel tasks for RPM, core service, and npm/maven functional test suites.
• Introduce a shared install-test-deps step in install-bindings to provision pytest and functional test requirements once for all downstream test tasks.
• Update functional test invocations to use per-suite pytest cache directories and distinct JUnit XML output filenames to avoid conflicts during parallel runs.
• Relax the dependency of push-api-json-files-to-pulp to run after install-bindings instead of after all functional tests complete.
• Reduce the wait-for-api-schema curl retry interval from 15s to 5s while keeping the overall max timeout unchanged.

[sourcery-ai]

Reviewer's Guide

Parallelizes the Tekton deploy-and-test pipeline by splitting the monolithic pulp-functional-tests task into three focused tasks that run after install-bindings, centralizes test dependency installation into a shared install step, shortens the API schema polling interval, and hardens pytest execution with isolated cache directories and unique JUnit XML outputs.

File-Level Changes

Change	Details	Files
Shorten wait-for-api-schema polling interval without changing overall timeout.	Update curl --retry-delay from 15s to 5s Increase curl --retry count to maintain the same 900s max retry window	.tekton/pulp-deploy-and-test.yaml
Centralize and earlier-run installation of test dependencies in the install-bindings phase.	Add install-test-deps step that logs into the cluster and selects the target namespace Define cmd_prefix/cmd_stdin_prefix helpers for executing inside pulp-api Install pytest<8 and gnupg in the pulp-api container Download functest and unittest requirement files and copy them into /tmp in the container Install functional and unit test requirements and ensure pulpcore-manager is on PATH	.tekton/pulp-deploy-and-test.yaml
Replace the single pulp-functional-tests task with three specialized functional test tasks that can run in parallel.	Rename the existing test task into functional-tests-rpm, focusing it on pulp_rpm tests Create functional-tests-core-service task to run pulpcore and pulp_service functional suites Create functional-tests-npm-maven task to run pulp_maven and pulp_npm functional suites Configure each new task to depend on install-bindings and share the same BONFIRE_IMAGE/NS/secret param pattern	.tekton/pulp-deploy-and-test.yaml
Harden pytest executions to support concurrent runs and prevent data clobbering.	Introduce per-task pytest cache_dir paths (pytest_cache_rpm/core/npm) Assign unique JUnit XML output filenames per suite (e.g., junit-rpm-parallel.xml, junit-service.xml, junit-npm.xml) Reuse a shared debug_and_fail helper in each task to dump logs and routes on failure	.tekton/pulp-deploy-and-test.yaml
Decouple push-api-json-files-to-pulp from test results so it runs after bindings installation instead of functional tests.	Change push-api-json-files-to-pulp runAfter dependency from pulp-functional-tests to install-bindings so it can proceed regardless of test status	.tekton/pulp-deploy-and-test.yaml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• The three new functional test tasks duplicate a lot of setup logic (login.sh, namespace detection, PASSWORD extraction, cmd_prefix, debug_and_fail); consider moving this into a shared script or helper so future changes only need to be made in one place.
• debug_and_fail relies on oc_wrapper get pod | grep -oE 'pulp-(api|content|worker)\S*', which can match multiple pods; using label selectors or otherwise disambiguating the target pods would make log collection more robust as the deployment topology evolves.
• The hard-coded route https://env-${NS}.apps.crc-eph.r9lp.p1.openshiftapps.com/... is repeated in multiple tasks and ties the pipeline to a specific cluster domain; consider deriving the route dynamically (e.g., from oc_wrapper get route) or making it a parameter.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The three new functional test tasks duplicate a lot of setup logic (login.sh, namespace detection, PASSWORD extraction, cmd_prefix, debug_and_fail); consider moving this into a shared script or helper so future changes only need to be made in one place.
- debug_and_fail relies on `oc_wrapper get pod | grep -oE 'pulp-(api|content|worker)\S*'`, which can match multiple pods; using label selectors or otherwise disambiguating the target pods would make log collection more robust as the deployment topology evolves.
- The hard-coded route `https://env-${NS}.apps.crc-eph.r9lp.p1.openshiftapps.com/...` is repeated in multiple tasks and ties the pipeline to a specific cluster domain; consider deriving the route dynamically (e.g., from `oc_wrapper get route`) or making it a parameter.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[decko]

/retest