Push benchmark results to GitHub pages

[simonmarty]

Description

Why is this change being made?

1. Allows us to generate graphs of the agent's benchmarks.

What is changing?

1. Added a separate workflow with write permissions that runs on push to create benchmark charts.
2. Modified the workflow that runs on PRs so that it automatically fetches the latest bench baseline from GH pages.

We keep the two separate because they have different levels of permissions.

Related Links

• Issue #, if available:

---

Testing

Tested on fork https://simonmarty.github.io/aws-secretsmanager-agent/dev/bench/
Made a mock PR https://github.com/simonmarty/aws-secretsmanager-agent/actions/runs/24591788832

How was this tested?

1. Tested on fork https://simonmarty.github.io/aws-secretsmanager-agent/dev/bench/
2. Made a mock PR https://github.com/simonmarty/aws-secretsmanager-agent/actions/runs/24591788832

When testing locally, provide testing artifact(s):

---

Reviewee Checklist

Update the checklist after submitting the PR

• I have reviewed, tested and understand all changes
  If not, why:
• I have filled out the Description and Testing sections above
  If not, why:
• Build and Unit tests are passing
  If not, why:
• Unit test coverage check is passing
  If not, why:
• Integration tests pass locally
  If not, why:
• I have updated integration tests (if needed)
  If not, why: No change
• I have ensured no sensitive information is leaking (i.e., no logging of sensitive fields, or otherwise)
  If not, why:
• I have added explanatory comments for complex logic, new classes/methods and new tests
  If not, why:
• I have updated README/documentation (if needed)
  If not, why: No change
• I have clearly called out breaking changes (if any)
  If not, why: No breaking changes.

---

Reviewer Checklist

All reviewers please ensure the following are true before reviewing:

• Reviewee checklist has been accurately filled out
• Code changes align with stated purpose in description
• Test coverage adequately validates the changes

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.46%. Comparing base (c8b4c07) to head (18b422b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #193   +/-   ##
=======================================
  Coverage   92.46%   92.46%           
=======================================
  Files          15       15           
  Lines        3638     3638           
  Branches     3638     3638           
=======================================
  Hits         3364     3364           
  Misses        218      218           
  Partials       56       56           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[rstevens011]

for my edification: why is this being removed? Does benchmark-action/github-action-benchmark@v1 do this for you?

[simonmarty]

Sorry it took a minute to respond, I figured I could show you on my fork.

Here's a run from a push to main on the fork: https://github.com/simonmarty/aws-secretsmanager-agent/actions/runs/24808395936/job/72763846297

Here's a run against a no-op pull request I created on my fork, it correctly retrieves the test run results simonmarty#61 https://github.com/simonmarty/aws-secretsmanager-agent/actions/runs/24855062991/job/72765608319?pr=61

This does make me realize that there must be some variance in the performance of GitHub actions hosted runners. It's failing the regression check despite there not being any code changes. I'm going to set the workflow to be non-blocking and just post comments. Maybe up the tolerance threshold to the default of 200%.