-
Notifications
You must be signed in to change notification settings - Fork 322
Feat: microgen - adds _client_helpers.py.j2 template
#2299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
b9d4a04
chore: removes old proof of concept
chalmerlowe 5b4d538
removes old __init__.py
chalmerlowe 132c571
Adds two utility files to handle basic tasks
chalmerlowe 90b224e
Adds a configuration file for the microgenerator
chalmerlowe e071eab
Removes unused comment
chalmerlowe dc72a98
chore: adds noxfile.py for the microgenerator
chalmerlowe 7318f0b
feat: microgen - adds two init file templates
chalmerlowe 07910c5
feat: adds _helpers.py.js template
chalmerlowe dc54c99
Updates with two usage examples
chalmerlowe 28de5f8
feat: adds two partial templates for creating method signatures
chalmerlowe c457754
feat: Add microgenerator __init__.py
chalmerlowe 595e59f
feat: Add AST analysis utilities
chalmerlowe 44a0777
feat: Add source file analysis capabilities
chalmerlowe 3e9ade6
feat: adds code generation logic
chalmerlowe 485b9d4
removes extraneous content
chalmerlowe a4276fe
feat: microgen - adds code generation logic
chalmerlowe 1d0d036
feat: microgen - adds main execution and post-processing logic
chalmerlowe eff7223
minor tweak to markers
chalmerlowe 0734bf8
feat: Add testing directory\n\nAdds the scripts/microgenerator/testin…
chalmerlowe 510a87b
feat: Enhance to_snake_case to handle acronyms\n\nImproves the to_sna…
chalmerlowe a3117d8
feat: Add client.py.j2 template\n\nAdds the main Jinja2 template for …
chalmerlowe ae7d3e1
feat: Add _client_helpers.j2 partial template\n\nAdds a Jinja2 partia…
chalmerlowe 0122907
Merge branch 'autogen' into feat/adds-client-helpers-template
chalmerlowe 913f521
Update scripts/microgenerator/testing/constraints-3.13.txt
chalmerlowe 3bf9f16
Update scripts/microgenerator/generate.py
chalmerlowe 898eab9
Update scripts/microgenerator/generate.py
chalmerlowe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
49 changes: 49 additions & 0 deletions
49
scripts/microgenerator/templates/partials/_client_helpers.j2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| {# | ||
| This is a partial template file intended to be included in other templates. | ||
| It contains helper methods for the BigQueryClient class. | ||
| #} | ||
|
|
||
| # --- HELPER METHODS --- | ||
| def _parse_dataset_path(self, dataset_path: str) -> Tuple[Optional[str], str]: | ||
| """ | ||
| Helper to parse project_id and/or dataset_id from a string identifier. | ||
|
|
||
| Args: | ||
| dataset_path: A string in the format 'project_id.dataset_id' or | ||
| 'dataset_id'. | ||
|
|
||
| Returns: | ||
| A tuple of (project_id, dataset_id). | ||
| """ | ||
| if "." in dataset_path: | ||
| # Use rsplit to handle legacy paths like `google.com:my-project.my_dataset`. | ||
| project_id, dataset_id = dataset_path.rsplit(".", 1) | ||
| return project_id, dataset_id | ||
| return self.project, dataset_path | ||
|
|
||
| def _parse_dataset_id_to_dict(self, dataset_id: "DatasetIdentifier") -> dict: | ||
| """ | ||
| Helper to create a dictionary from a project_id and dataset_id to pass | ||
| internally between helper functions. | ||
|
|
||
| Args: | ||
| dataset_id: A string or DatasetReference. | ||
|
|
||
| Returns: | ||
| A dict of {"project_id": project_id, "dataset_id": dataset_id_str }. | ||
| """ | ||
| if isinstance(dataset_id, str): | ||
| project_id, dataset_id_str = self._parse_dataset_path(dataset_id) | ||
| return {"project_id": project_id, "dataset_id": dataset_id_str} | ||
| elif isinstance(dataset_id, dataset_reference.DatasetReference): | ||
| return { | ||
| "project_id": dataset_id.project_id, | ||
| "dataset_id": dataset_id.dataset_id, | ||
| } | ||
| else: | ||
| raise TypeError(f"Invalid type for dataset_id: {type(dataset_id)}") | ||
|
|
||
| def _parse_project_id_to_dict(self, project_id: Optional[str] = None) -> dict: | ||
| """Helper to create a request dictionary from a project_id.""" | ||
| final_project_id = project_id or self.project | ||
| return {"project_id": final_project_id} | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these helper functions going to be included in every generated client or is the inclusion of each individual function done on a selective basis?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for checking.
There is only one generated client and these three helper functions will be included one time via jinja templating imports in the file that the client class is placed in.
For context: The purpose of this microgenerator is to create code that will support customer transition from using a strictly handwritten
BigQuery (BQ)library to using a GAPIC generatedBQlibrary. Specifically it will allow the customer to use a singleBigQueryClientinstead of using seven very disparate*ServiceClients(e.g.JobServiceClient,TableServiceClient,ModelServiceClient) AND to do so in a way that feels very similar to how they useBigQuerynow.To help facilitate this, a proxy of certain attributes associated with each
*ServiceClientwill be added to new a centralizedBigQueryClientand provide a passthrough so that users can use the attribute they want (e.g.JobServiceClient.list_jobs(),DatasetServiceClient.get_dataset(),TableServiceClient.update_table()) in theServiceClientthey want.Our helpers are present so that customer can provide the
BigQueryClientmethod calls with strings that can be parsed and passed successfully to the underlying attributes.As an example, without the centralized
BigQueryClientand the helpers:To use
TableServiceClient.list_tables(), the customer would need to instantiate aListTableRequest()object which will require the customer to pass in info such as theproject_idanddataset_id.For the past ten years, customers have not needed to do that extra step.
With the new
BigQueryClient, the passthrough methods, and the helpers, the customer will have a user experience much closer to what they are used to, for example:Our helpers take strings similar to
proj_id.dataset_idand create an appropriate*Requestobject that is passed invisibly on behalf of the customer.It may seem like a little thing, but it will hopefully ease the transition to using the new GAPIC generated library.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gkevinzheng see above.