Started to see a message on each PR about a new CodeQL configuration

[KristianGrafana]

Hey!

We have started to see the following message on PR's:

You have successfully added a new CodeQL configuration .github/workflows/pr-codeql-analysis-javascript.yml:analyze. As part of the setup process, we have scanned this repository and found no existing alerts. In the future, you will see all code scanning alerts on the repository Security tab.

No change has been made to our CodeQL configuration. We find this a bit disruptive and would like to turn off these messages, how can that be made?

Thanks in advance!

[sj]

Hi @KristianGrafana! I had a quick look at the situation, and indeed that message shouldn't repeat itself for every PR. It's caused by a slightly unusual configuration pattern: the Grafana repository has multiple separate CodeQL Actions workflow files:

1. one that defines the scanning for on: push (for all languages)
2. and a bunch of others that defines the configuration for scanning on PRs (one for each language, e.g. Go)

Every single time the PR analysis runs, we check whether this CodeQL configuration already exists on the target branch. In this example: main. It concludes that that this particular code scanning configuration doesn't yet exist on main, and assumes that the PR is part of the process of setting the analysis up for pushes to main. That is how the vast majority of users go through the process of setting up code scanning. But this doesn't apply to Grafana: there already is a (separate) configuration running for pushes to main.

So on every single PR, code scanning concludes "ah, you must be in the process of setting up code scanning, and congratulations 🎉 you succeeded!".

Clearly this is not how we intend this to work for you, so we're investigating whether there's any better approach to this. Thanks for your patience!

[sj]

We've temporarily switched off the "You have successfully added a ..." messages altogether while we work on a solution for this. We expect to finish work on that sometime next week, at which point new users should be seeing these messages again, but no one else. Thanks for your patience!

[KristianGrafana]

@sj Thank you, much appreciate the short term solution.

[crazy-max]

We've temporarily switched off the "You have successfully added a ..." messages altogether while we work on a solution for this. We expect to finish work on that sometime next week, at which point new users should be seeing these messages again, but no one else. Thanks for your patience!

We have the same issue in docker org where this message appears on every PR because code scanning result is not pushed to the default branch like docker/scout-cli#182 (comment). In our case we are using the upload-sarif action: https://github.com/github/codeql-action/blob/main/upload-sarif/action.yml. Would it be possible to have an action input to disable such message in PRs?