Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,15 @@ format: ## Format the code.
poetry run black src
poetry run ruff check src --fix

.PHONY: md-fix
md-fix: ## Run markdown linting with Markdownlint and fix issues.
sh ./shell_scripts/md_fix.sh

.PHONY: lint
lint: ## Run all linters (black/ruff/pylint/mypy).
lint: ## Run all linters (black/ruff/pylint/mypy/markdownlint).
poetry run black --check src
poetry run ruff check src
make md-fix
make mypy

.PHONY: test
Expand Down
44 changes: 31 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,18 +128,26 @@ Before the doing the following, make sure your Daemon is running. If using Colim
github-repository-archive-script latest b4a1e32ce51b 12 minutes ago 840MB
```

3. Run the image.
3. Sign in with AWS SSO:

```bash
aws sso login
```

**Note:** See the Developer Onboarding Guide on the "Using AWS SSO for Local Development" page on Confluence to set up service profile selection on your local machine. This is essential as the `~/.aws` directory is mounted to the container, so it can use the SSO session for AWS authentication.

4. Run the image.

```bash
docker run --platform linux/amd64 -p 9000:8080 \
-e AWS_ACCESS_KEY_ID=<access_key_id> \
-e AWS_SECRET_ACCESS_KEY=<secret_access_key> \
-e AWS_DEFAULT_REGION=<region> \
-v ~/.aws:/root/.aws \
-e AWS_PROFILE=github-repository-archive-script \
-e AWS_DEFAULT_REGION=eu-west-2 \
-e AWS_SECRET_NAME=<secret_name> \
-e GITHUB_ORG=<org> \
-e GITHUB_APP_CLIENT_ID=<client_id> \
-e S3_BUCKET_NAME=<bucket_name>\
-e AWS_LAMBDA_FUNCTION_TIMEOUT=300
-e S3_BUCKET_NAME=<bucket_name> \
-e AWS_LAMBDA_FUNCTION_TIMEOUT=300 \
github-repository-archive-script
```

Expand All @@ -156,7 +164,7 @@ Before the doing the following, make sure your Daemon is running. If using Colim

Once the container is running, a local endpoint is created at `localhost:9000/2015-03-31/functions/function/invocations`.

4. Check the container is running (Optional).
5. Check the container is running (Optional).

```bash
docker ps
Expand All @@ -169,15 +177,15 @@ Before the doing the following, make sure your Daemon is running. If using Colim
ca890d30e24d github-repository-archive-script "/lambda-entrypoint.…" 5 seconds ago Up 4 seconds 0.0.0.0:9000->8080/tcp, :::9000->8080/tcp recursing_bartik
```

5. Post to the endpoint (`localhost:9000/2015-03-31/functions/function/invocations`).
6. Post to the endpoint (`localhost:9000/2015-03-31/functions/function/invocations`).

```bash
curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
```

This will run the Lambda function and, once complete, will return a success message.

6. After testing stop the container.
7. After testing stop the container.

```bash
docker stop <container_id>
Expand All @@ -198,11 +206,21 @@ To run the Lambda function outside of a container, we need to execute the `handl

**Please Note:** If uncommenting the above in `main.py`, make sure you re-comment the code _before_ pushing back to GitHub.

2. Export the required environment variables:
2. Sign in with AWS SSO, and export the correct profile for this service:

```bash
aws sso login

export AWS_PROFILE=github-repository-archive-script
```

This allows you to assume the AWS IAM role for the service, enabling the most secure development experience. This also means you will have limited permissions until you exit out of the profile.

**Note:** See the Developer Onboarding Guide on the "Using AWS SSO for Local Development" page on Confluence to set up service profile selection on your local machine.

3. Export the required environment variables:

```bash
export AWS_ACCESS_KEY_ID=<access_key_id>
export AWS_SECRET_ACCESS_KEY=<secret_access_key>
export AWS_DEFAULT_REGION=eu-west-2
export AWS_SECRET_NAME=<secret_name>
export S3_BUCKET_NAME=<bucket_name>
Expand All @@ -212,7 +230,7 @@ To run the Lambda function outside of a container, we need to execute the `handl

An explanation of each variable is available within the [containerised instructions](#containerised-recommended).

3. Run the script.
4. Run the script.

```bash
python3 src/main.py
Expand Down
12 changes: 6 additions & 6 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions shell_scripts/md_fix.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

docker run -v "$PWD:/workdir" ghcr.io/igorshubovych/markdownlint-cli:latest "**/*.md" --fix
70 changes: 14 additions & 56 deletions terraform/service/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,20 @@ resource "aws_iam_role" "lambda_function_role" {
Principal = {
Service = "lambda.amazonaws.com"
}
},
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
AWS = "arn:aws:iam::${var.aws_account_id}:root"
}
Condition = {
ArnLike = {
"aws:PrincipalArn" = [
"arn:aws:iam::${var.aws_account_id}:role/aws-reserved/sso.amazonaws.com/eu-west-2/AWSReservedSSO_Standard_Administrator_Access_*"
]
}
}
}
]
})
Expand Down Expand Up @@ -138,59 +152,3 @@ resource "aws_cloudwatch_log_group" "loggroup" {
name = "/aws/lambda/${aws_lambda_function.lambda_function.function_name}"
retention_in_days = var.log_retention_days
}

# IAM User Group
resource "aws_iam_group" "group" {
name = "${var.env_name}-${var.lambda_name}-user-group"
path = "/"
}

resource "aws_iam_group_policy_attachment" "group_vpc_permissions_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.vpc_permissions.arn
}

resource "aws_iam_group_policy_attachment" "group_lambda_logging_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.lambda_logging.arn
}

resource "aws_iam_group_policy_attachment" "group_lambda_s3_policy_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.lambda_s3_policy.arn
}

resource "aws_iam_group_policy_attachment" "group_lambda_secret_manager_policy_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.lambda_secret_manager_policy.arn
}

resource "aws_iam_group_policy_attachment" "group_lambda_eventbridge_policy_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.lambda_eventbridge_policy.arn
}

# IAM User
resource "aws_iam_user" "user" {
name = "${var.env_name}-${var.lambda_name}"
path = "/"
}

# Assign IAM User to group
resource "aws_iam_user_group_membership" "user_group_attach" {
user = aws_iam_user.user.name

groups = [
aws_iam_group.group.name
]
}

# IAM Key Rotation Module
module "iam_key_rotation" {
source = "git::https://github.com/ONS-Innovation/keh-aws-iam-key-rotation.git?ref=v0.1.1"

iam_username = aws_iam_user.user.name
access_key_secret_arn = aws_secretsmanager_secret.access_key.arn
secret_key_secret_arn = aws_secretsmanager_secret.secret_key.arn
rotation_in_days = 45
}
14 changes: 0 additions & 14 deletions terraform/service/secrets.tf

This file was deleted.

Loading