Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,24 @@ There are a few `TODOs` scattered throughout the README and codebase that will n
make install
```

2. Export Environment Variables
2. Sign in with AWS SSO, and export the correct profile for this service:

```bash
aws sso login

export AWS_PROFILE=keh-github-statistics-scraper
```

This allows you to assume the AWS IAM role for the service, enabling the most secure development experience. This also means you will have limited permissions until you exit out of the profile.

**Note:** See the Developer Onboarding Guide on the "Using AWS SSO for Local Development" page on Confluence to set up service profile selection on your local machine.

```bash
# AWS
export AWS_ACCESS_KEY_ID=<KEY>
export AWS_SECRET_ACCESS_KEY=<SECRET>
export AWS_DEFAULT_REGION=<REGION>
export AWS_SECRET_NAME=/<env>/github-tooling-suite/<onsdigital/ons-innovation>

3. Export Environment Variables

```bash
# GitHub
export AWS_SECRET_NAME=/<env>/github-tooling-suite/<onsdigital/ons-innovation>
export GITHUB_APP_CLIENT_ID=<CLIENT_ID>
export GITHUB_ORG=<onsdigital/ons-innovation>

Expand All @@ -73,12 +81,18 @@ There are a few `TODOs` scattered throughout the README and codebase that will n
| `batch_size` | The number of repositories that will be scraped in each batch. |
| `environment` | Determines where to save the results. `development`: locally, `production`: to S3. |

3. Run the script:
4. Run the script:

```bash
make run
```

5. To exit the profile:

```bash
unset AWS_PROFILE
```

## Deployment

### Deployments with Concourse
Expand Down
45 changes: 1 addition & 44 deletions terraform/batch/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ resource "aws_iam_policy" "s3_access" {
}

module "batch_eventbridge" {
source = "git::https://github.com/ONS-Innovation/keh-scheduled-batch-tf-module.git?ref=v1.1.1"
source = "git::https://github.com/ONS-Innovation/keh-scheduled-batch-tf-module.git?ref=v1.2.2"

aws_account_id = var.aws_account_id
aws_access_key_id = var.aws_access_key_id
Expand Down Expand Up @@ -70,46 +70,3 @@ resource "aws_iam_role_policy_attachment" "s3_policy_attachment" {
role = split("/", module.batch_eventbridge.batch_job_role_arn)[1]
policy_arn = aws_iam_policy.s3_access.arn
}

# IAM User Group
resource "aws_iam_group" "group" {
name = "${var.domain}-${var.service_subdomain}-user-group"
path = "/"
}

# Attach S3 policy to group
resource "aws_iam_group_policy_attachment" "group_s3_access_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.s3_access.arn
}

# Attach Secrets Manager policy to group
resource "aws_iam_group_policy_attachment" "group_secrets_attachment" {
group = aws_iam_group.group.name
policy_arn = aws_iam_policy.secrets_access.arn
}

# IAM User
resource "aws_iam_user" "user" {
name = "${var.domain}-${var.service_subdomain}"
path = "/"
}

# Assign IAM User to group
resource "aws_iam_user_group_membership" "user_group_attach" {
user = aws_iam_user.user.name

groups = [
aws_iam_group.group.name
]
}

# IAM Key Rotation Module
module "iam_key_rotation" {
source = "git::https://github.com/ONS-Innovation/keh-aws-iam-key-rotation.git?ref=v0.1.1"

iam_username = aws_iam_user.user.name
access_key_secret_arn = aws_secretsmanager_secret.access_key.arn
secret_key_secret_arn = aws_secretsmanager_secret.secret_key.arn
rotation_in_days = 45
}
15 changes: 0 additions & 15 deletions terraform/batch/secrets.tf

This file was deleted.

Loading