Skip to content

Conversation

@louiscb
Copy link
Contributor

@louiscb louiscb commented Apr 30, 2025

  • Update ex_aws_s3 to include bug fix discovered recently
  • Allows users to set a tuple of bucket and region version (since buckets are globally unique)
  • Example
          [
            fetcher_mod: @fetcher,
            fetcher_opts: [
              bucket: "mo1212net-deals",
              region: "us-east-1",
              key: "hg.html",
              failover_buckets: [
                                [bucket: "adaptmx-airflow", region: "us-east-2"]
              ]
            ],

@louiscb louiscb requested a review from sneako April 30, 2025 13:59
key: String.t(),
region: String.t(),
failover_regions: [String.t()] | nil
failover_buckets: [[bucket: String.t(), region: String.t()]] | nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's define a failover_bucket as a new type

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe even types for bucket and region since we use them more than once

failover_regions: [
type: {:list, :string},
failover_buckets: [
type: {:list, :keyword_list},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you define a a schema for the failover buckets, so that nimble options validates it has the correct key/values?

key: key,
failover_regions: failover_regions
} = state
failover_buckets: failover_buckets
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
failover_buckets: failover_buckets
failover_buckets: [_|_] = failover_buckets

just to make sure the list isn't empty. We can get rid of the guard with this change too

@louiscb louiscb merged commit 8f8a204 into main May 7, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants