-
Notifications
You must be signed in to change notification settings - Fork 697
feat(bindings/python): Add from_uri constructors for Operators #6993
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Tests pass but I'm not clear how best to test the new functionality as it would mean parsing the config provided by the setup_config fixture and then constructing a URI out of it, which is going to be different for each service. |
|
Thank you for working on this! I will invite @chitralverma to take a review on this. |
|
@clbarnes can you run stubgen with just to get the python stubs as well? |
|
I found that running stubgen locally drops a bunch of services and deletes a lot of generated code as well - possibly all the services which have been split out of core into their own crates? --- a/bindings/python/src/services.rs
+++ b/bindings/python/src/services.rs
@@ -62,16 +62,6 @@ submit! {
#[pyo3(rename_all = "PascalCase")]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum PyScheme {
- #[cfg(feature = "services-aliyun-drive")]
- AliyunDrive,
- #[cfg(feature = "services-alluxio")]
- Alluxio,
- #[cfg(feature = "services-azblob")]
- Azblob,
- #[cfg(feature = "services-azdls")]
- Azdls,
- #[cfg(feature = "services-azfile")]
- Azfile,
#[cfg(feature = "services-b2")]
B2,
#[cfg(feature = "services-cacache")]
@@ -84,24 +74,14 @@ pub enum PyScheme {
Dropbox,
#[cfg(feature = "services-fs")]
Fs,
- #[cfg(feature = "services-ftp")]
- Ftp,
- #[cfg(feature = "services-gcs")]
- Gcs,
#[cfg(feature = "services-gdrive")]
Gdrive,
- #[cfg(feature = "services-ghac")]
- Ghac,
#[cfg(feature = "services-gridfs")]
Gridfs,
- #[cfg(feature = "services-hdfs-native")]
- HdfsNative,
#[cfg(feature = "services-http")]
Http,
#[cfg(feature = "services-huggingface")]
Huggingface,
- #[cfg(feature = "services-ipfs")]
- Ipfs,
#[cfg(feature = "services-ipmfs")]
Ipmfs,
#[cfg(feature = "services-koofr")]etc.. That comes from the The second step doesn't finish successfully - but This could possibly resolved using the devcontainer, but that has issues too: #6992 |
Which issue does this PR close?
Closes #6984 .
Rationale for this change
Instantiating Operators from URLs greatly increases flexibility for consumers of the openDAL API as an abstract backend.
What changes are included in this PR?
Add from_uri static methods to the Operator and AsyncOperator classes in the python bindings.
Are there any user-facing changes?
Two new methods are available to users of the python library.
AI Usage Statement
None
Other notes
I am having issues with my development environment which the devcontainer is not making any easier, so I'm flying a bit blind and relying on CI.