Skip to content

Conversation

@vbekiaris
Copy link
Contributor

Introduces IO registry (similar to catalog registry).
Default IO implementations (S3, Azure, GSC, local, memory) are pre-registered.
Custom IO implementations can be registered with io.Register and looked up with io.Load

Copy link
Contributor

@laskoviymishka laskoviymishka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you slightly misunderstood the purpose of the registry pattern.

In Go, the main idea behind this pattern is decoupling. It allows the final application to include only the necessary dependencies. For example, with catalogs, you can import just the REST or Glue catalog, and the rest will be ignored if not imported.

There's no need to implement this pattern inside a single package. I do like the direction with using registrars, but the current implementation feels halfway done. To fully leverage this approach, you should decouple the IO logic—split it into independent packages like mem, local, gcs, azure, etc. These can then be included in the final binary via imports.

This structure will also allow for easy injection of custom IO implementations. Based on what I see in #441, that seems to be your end goal—so let’s finalize this by splitting the io package into sub-packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants