Skip to content

Conversation

@kurt-hectic
Copy link

This PR implements sub-partitioning for PostgreSQL in django-postgres-extra, and a strategy to have LIST partitions on the first and (date) RANGE partitions on the second level.

Our use-case for this was that we would like to partition a table first on an an integer column, and then use a CurrentTimePartitioning strategy to further sub-partition using a date column, and use pgpartition for partition management.

We can declare a sub-partitioned model like this:

class MyPartitionedModel(PostgresPartitionedModel):
    class PartitioningMeta:
        method = PostgresPartitioningMethod.LIST
        key = ["category"]
        sub_method = PostgresPartitioningMethod.RANGE
        sub_key = ["date"]

    my_id = models.IntegerField(primary_key=True)
    name = models.TextField()
    category = models.IntegerField()
    date = models.DateTimeField()

The PostgresCategoryCurrentTimePartitioningStrategy creates and manages first and second level partitions, and can be used like this:

manager = PostgresPartitioningManager([
    # 3 partitions ahead, each partition is one year in size,
    # delete partitions older than 10 years
    partition_by_category_and_current_time(
        model=MyPartitionedModel,
        years=1,
        count=3,
        max_age=relativedelta(years=10),
        categories=[1, 2, 100],
    ),])

Resulting in first level partitions mypartitionedmodel_1, mypartitionedmodel_2, mypartitionedmodel_100 and second level partitions mypartitionedmodel_1_2026, mypartitionedmodel_1_2027, mypartitionedmodel_1_2028, mypartitionedmodel_2_2026 ...

The implementation aimed to only make minimal changes to the basecode and implements most functions by overwriting classes as a contributing module.

If the PR is acceptable in principle, the new model classes PostgresListPartition and PostgresTimeSubPartition, as well as the add_list_partition, add_range_partition and possibly add_hash_partition in the schema editor, could be further consolidated to allow additional sub-partitioning scenarios other than LIST and RANGE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant