Add view cron jobs and alerting

[MEspositoE14s]

🎫 Ticket

https://jira.cms.gov/browse/DPC-5113

🛠 Changes

Adds two new sql scripts:

schedule_view_exports.sql:

• Uses pg_cron to schedule exports of the views we created for QuickSight once a day to the dpc-prod-aurora-export-20251028154132844600000001 S3 bucket.

config_cron_alerts.sql:

• Creates a stored procedure, check_cron_status that checks the status of a pg_cron job to see if it failed. If so, it uses the bcda-prod-alarm-to-slack Lambda to send a message to the #dpc_alerts Slack channel.
• Creates a trigger that calls check_cron_status whenever a row is inserted/updated to the cron.job_run_details table.

ℹ️ Context

We've been asked to export these views to the S3 bucket every six hours so they can be imported into QuickSight.

Some quick notes:

1. The S3 bucket was created by the CDAP team manually in the AWS console. They have a ticket to add it to Tofu in the future.
2. The bucket and the rest of the infra only exist in prod.
3. To make this work, I installed the aws_s3 and aws_lambda extensions in Postgres. pg_cron was already installed.
4. I also created a dpc-prod-aurora-lambda-role that has permissions (dpc-prod-aurora-lambda) to call our Slack lambda and added it to the IAM roles assigned to our RDS instance. This should be moved to OpenTofu when the infra is built there.

🧪 Validation

Currently running, you can open up the S3 bucket and see the exports:

dpc-prod DB has a new role attached (dpc-prod-aurora-lambda-role) giving it access to call our Lambda:

Alerts are being sent to Slack when an export fails. To test, go into the cron.job_run_details and update the status on a row:

[gsf]

Is there a place in dpc-app where these view and export SQL scripts can be stored? CDAP shouldn't need to own and review these.

[jscott-nava]

Is there a place in dpc-app where these view and export SQL scripts can be stored? CDAP shouldn't need to own and review these.

@gsf We decided a while back to keep all the view and export scripts in the CDAP repo as a source of truth for all the DASG data sources, so the files in this PR are alongside those of AB2D and BCDA.

[jdettmannnava]

LGTM

I will say our approach seems pretty different from the other teams', and our file naming scheme uses underscores instead of dashes, but if it doesn't bother cdap it doesn't bother me!

[MEspositoE14s]

LGTM

I will say our approach seems pretty different from the other teams', and our file naming scheme uses underscores instead of dashes, but if it doesn't bother cdap it doesn't bother me!

Yeah, this was built in a complete vacuum without knowing the other teams were doing something similar. I also don't think they had the requirement to alert on a failed export.

[MEspositoE14s]

Is there a place in dpc-app where these view and export SQL scripts can be stored? CDAP shouldn't need to own and review these.

@gsf We decided a while back to keep all the view and export scripts in the CDAP repo as a source of truth for all the DASG data sources, so the files in this PR are alongside those of AB2D and BCDA.

What's the final verdict on this? Are we going to keep these in the CDAP repo or move them back to DPC?

[gsf]

I still hold that these SQL files specific to each app should be in app repos because they are managed by app teams.

[MEspositoE14s]

Closing and moving back to DPC repo.