feat(lambdas): add batch SSM parameter fetching to reduce API calls

[thomasnemer]

This PR intends to reduce SSM AWS API calls by doing the following:

Add getParameters() function to aws-ssm-util that fetches multiple SSM parameters in a single API call with automatic chunking (max 10 per call per AWS API limits).

Apply batch fetching to:

• auth.ts: fetch App ID and Private Key in one call (2 calls → 1)
• ConfigLoader.ts: fetch multiple matcher config paths in one call
• ami.ts: batch resolve SSM parameter values for AMI lookups

Also remove redundant appId SSM fetch in scale-up.ts that was only used for logging.

[Copilot]

Pull request overview

This PR optimizes AWS SSM API usage by implementing batch parameter fetching to reduce the number of API calls. The changes introduce a new getParameters() function that retrieves multiple SSM parameters in a single API call (with automatic chunking for AWS's 10-parameter limit) and applies this optimization across multiple Lambda functions.

Changes:

• Added getParameters() function to aws-ssm-util for batch SSM parameter fetching with automatic chunking
• Replaced sequential SSM calls with batch fetching in authentication, configuration loading, and AMI housekeeping
• Removed redundant appId SSM fetch in scale-up.ts that was only used for logging

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
lambdas/libs/aws-ssm-util/src/index.ts	Implements new getParameters() function for batch SSM parameter retrieval
lambdas/functions/webhook/src/ConfigLoader.ts	Replaces sequential parameter fetching with batch call for matcher configs
lambdas/functions/webhook/src/ConfigLoader.test.ts	Updates tests to mock getParameters() instead of individual getParameter() calls
lambdas/functions/control-plane/src/scale-runners/scale-up.ts	Removes redundant appId SSM fetch used only for logging
lambdas/functions/control-plane/src/github/auth.ts	Batches App ID and Private Key fetching into single SSM call
lambdas/functions/control-plane/src/github/auth.test.ts	Updates tests to verify batch parameter fetching behavior
lambdas/functions/ami-housekeeper/src/ami.ts	Refactors to use batch parameter fetching for AMI SSM lookups
lambdas/functions/ami-housekeeper/src/ami.test.ts	Updates tests to use GetParametersCommand instead of GetParameterCommand

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

lambdas/functions/webhook/src/ConfigLoader.test.ts:203

• There are no tests verifying the behavior when one or more parameters are missing from the getParameters() response. The code at lines 110-117 handles this by adding an error to configLoadingErrors, but this path is not tested. Add a test case where getParameters() returns a Map with only some of the requested parameters to verify that appropriate error messages are added for the missing ones.

    it('should load config successfully from multiple paths', async () => {
      process.env.PARAMETER_RUNNER_MATCHER_CONFIG_PATH = '/path/to/matcher/config-1:/path/to/matcher/config-2';
      process.env.PARAMETER_GITHUB_APP_WEBHOOK_SECRET = '/path/to/webhook/secret';

      const partialMatcher1 =
        '[{"id":"1","arn":"arn:aws:sqs:queue1","matcherConfig":{"labelMatchers":[["a"]],"exactMatch":true}}';
      const partialMatcher2 =
        ',{"id":"2","arn":"arn:aws:sqs:queue2","matcherConfig":{"labelMatchers":[["b"]],"exactMatch":true}}]';

      const combinedMatcherConfig = [
        { id: '1', arn: 'arn:aws:sqs:queue1', matcherConfig: { labelMatchers: [['a']], exactMatch: true } },
        { id: '2', arn: 'arn:aws:sqs:queue2', matcherConfig: { labelMatchers: [['b']], exactMatch: true } },
      ];

      // Mock getParameters for batch fetching multiple paths
      vi.mocked(getParameters).mockResolvedValue(
        new Map([
          ['/path/to/matcher/config-1', partialMatcher1],
          ['/path/to/matcher/config-2', partialMatcher2],
        ]),
      );

      vi.mocked(getParameter).mockImplementation(async (paramPath: string) => {
        if (paramPath === '/path/to/webhook/secret') return 'secret';
        return '';
      });

      const config: ConfigWebhook = await ConfigWebhook.load();

      expect(config.matcherConfig).toEqual(combinedMatcherConfig);
      expect(config.webhookSecret).toBe('secret');
    });

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Brend-Smits]

@thomasnemer this is great! Thanks a lot for your contribution.
Would you mind providing a quick test plan that maintainers can refer to in order to swiftly test your changes?
Also, please have a look at the remaining copilot feedback, and if not relevant or if you disagree, comment and resolve it.

Thank you!

[Brend-Smits]

Hey there!

I was not able to complete a test deployment for this as it seems that there is some bug causing scale-up to no longer spin up instances.

Here's a snippet from the log:

Error processing batch (size: 1): User: arn:aws:sts::<account_id>:assumed-role/framework-dev-linux-x64-ubuntu-scale-up-lambda-43f7d69e/framework-dev-linux-x64-ubuntu-scale-up is not authorized to perform: ssm:GetParameters on resource: arn:aws:ssm:eu-west-1:<account_id>:parameter/github-action-runners/framework-dev/app/github_app_id because no identity-based policy allows the ssm:GetParameters action, ignoring batch",

I've deployed the examples/multi-runner and this is what I got. Can you please have a look if you can reproduce and see if you can resolve this?

Thank you!

[npalm]

@thomasnemer can you rebase the pr as well? Thanks!

[thomasnemer]

@Brend-Smits I expect the deployment to be ok now, but still couldn't give it a try on our non-prod environment. I would understand you'd prefer we put this PR on hold until I can :)

[Brend-Smits]

I have put this to the test and it seems to work reliably! Great work and thanks for the contribution. Good to merge from my side