Skip to content

ci: send a notification of scheduled failing tests#2601

Merged
zimeg merged 5 commits intomainfrom
zimeg-ci-notifications
Jul 16, 2025
Merged

ci: send a notification of scheduled failing tests#2601
zimeg merged 5 commits intomainfrom
zimeg-ci-notifications

Conversation

@zimeg
Copy link
Member

@zimeg zimeg commented Jul 9, 2025

Summary

This PR sends a notification of scheduled failing tests using the new "regression failures" workflow!

Follows: slackapi/bolt-python#1323

Requirements

@zimeg zimeg requested a review from WilliamBergamin July 9, 2025 19:34
@zimeg zimeg self-assigned this Jul 9, 2025
@zimeg zimeg added tests M-T: Testing work only github_actions Pull requests that update GitHub Actions code labels Jul 9, 2025
@zimeg
Copy link
Member Author

zimeg commented Jul 9, 2025

Checks are failing with this change - perhaps related to #2506. Planning to investigate this later!

🔗 https://github.com/slackapi/bolt-js/actions/runs/16178549521/job/45669458331?pr=2601

Error: src/Assistant.ts(398,21): error TS18048: 'payload.assistant_thread' is possibly 'undefined'.
Error: src/Assistant.ts(401,5): error TS2322: Type 'unknown' is not assignable to type 'string'.
Error: src/Assistant.ts(402,5): error TS2322: Type 'unknown' is not assignable to type 'string'.
Error: src/Assistant.ts(403,5): error TS2322: Type 'unknown' is not assignable to type 'AssistantThreadContext'.

@zimeg zimeg marked this pull request as draft July 9, 2025 19:36
@codecov
Copy link

codecov bot commented Jul 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.37%. Comparing base (1ab9b1a) to head (6687aaa).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2601   +/-   ##
=======================================
  Coverage   93.37%   93.37%           
=======================================
  Files          37       37           
  Lines        7581     7581           
  Branches      667      667           
=======================================
  Hits         7079     7079           
  Misses        497      497           
  Partials        5        5           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zimeg zimeg marked this pull request as ready for review July 15, 2025 20:46
@zimeg
Copy link
Member Author

zimeg commented Jul 15, 2025

🗣️ If this seems alright the SLACK_REGRESSION_FAILURES_WEBHOOK_URL secret will be saved before merge!

Copy link
Contributor

@WilliamBergamin WilliamBergamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good 💯 I originally hesitated on this since we may get notified when tests are being flaky, but as you brilliantly pointed out it will force us improve the stability of our tests (as they should be)

@zimeg
Copy link
Member Author

zimeg commented Jul 16, 2025

@WilliamBergamin Thanks so much 🙏 ✨

I'm optimistic that insight into tests might point out patterns that we might be missing elsewhere, but I also hope for tests to not fail without good reason! So I will merge this.

@zimeg zimeg merged commit 4982aba into main Jul 16, 2025
19 checks passed
@zimeg zimeg deleted the zimeg-ci-notifications branch July 16, 2025 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code tests M-T: Testing work only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants