Skip to content

Do not backfill old DagRun.created_at#63825

Merged
vatsrahul1001 merged 1 commit intoapache:mainfrom
astronomer:do-not-backfill-dr-created-at
Mar 18, 2026
Merged

Do not backfill old DagRun.created_at#63825
vatsrahul1001 merged 1 commit intoapache:mainfrom
astronomer:do-not-backfill-dr-created-at

Conversation

@uranusjr
Copy link
Member

Filling this field in the migration is much too costly. This field is now left as NULL for rows created prior to the migration.

Close #63550.

@boring-cyborg boring-cyborg bot added the area:db-migrations PRs with DB migration label Mar 17, 2026
@uranusjr
Copy link
Member Author

Actually, I couldn’t find any usage of this field. Do we plan to use it in the future for partitions? I want to remove it outright if there’s not a planned usage.

@uranusjr
Copy link
Member Author

cc @dstandish

@uranusjr uranusjr force-pushed the do-not-backfill-dr-created-at branch from 5646bc6 to 9fc0919 Compare March 17, 2026 14:13
Filling this field in the migration is much too costly. This field is
now left as NULL for rows created prior to the migration.

I couldn't find this field used anywhere.
@uranusjr uranusjr force-pushed the do-not-backfill-dr-created-at branch from 9fc0919 to 287d856 Compare March 17, 2026 14:16
@ashb
Copy link
Member

ashb commented Mar 17, 2026

Please keep the field, it's helpful for ordering and other things I have in mind to add.

@uranusjr
Copy link
Member Author

Is it OK it’s nullable (and can contain null values)?

@vatsrahul1001 vatsrahul1001 merged commit 32f3fcc into apache:main Mar 18, 2026
140 of 141 checks passed
@vatsrahul1001 vatsrahul1001 deleted the do-not-backfill-dr-created-at branch March 18, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration 0106 upgrade takes ~6 minutes on large dag_run tables due to full-table UPDATE

4 participants