Skip to content

Commit 49fd63f

Browse files
chore: Update discovery artifacts (#2717)
## Deleted keys were detected in the following stable discovery artifacts: displayvideo v3 https://togithub.com/googleapis/google-api-python-client/commit/8ab2268c8269d972fc28563e1e94b5972e400c97 documentai v1 https://togithub.com/googleapis/google-api-python-client/commit/1740f1e8a8ba82bd3a9700b0659d8503664416f6 fcm v1 https://togithub.com/googleapis/google-api-python-client/commit/3846cb0f285d84f13df9a5721b115f6b1134ba92 networkmanagement v1 https://togithub.com/googleapis/google-api-python-client/commit/c8976bf6a8c2156b03d534e365c108409380aabc ## Deleted keys were detected in the following pre-stable discovery artifacts: alloydb v1alpha https://togithub.com/googleapis/google-api-python-client/commit/608b32804ad097b8a479edd988a3f84749062445 alloydb v1beta https://togithub.com/googleapis/google-api-python-client/commit/608b32804ad097b8a479edd988a3f84749062445 documentai v1beta3 https://togithub.com/googleapis/google-api-python-client/commit/1740f1e8a8ba82bd3a9700b0659d8503664416f6 networkmanagement v1beta1 https://togithub.com/googleapis/google-api-python-client/commit/c8976bf6a8c2156b03d534e365c108409380aabc ## Discovery Artifact Change Summary: feat(aiplatform): update the api https://togithub.com/googleapis/google-api-python-client/commit/03cd19c43da944793730a26181730fad340bdb78 feat(alloydb): update the api https://togithub.com/googleapis/google-api-python-client/commit/608b32804ad097b8a479edd988a3f84749062445 feat(androidpublisher): update the api https://togithub.com/googleapis/google-api-python-client/commit/ac63ab7a79e7cdc76b0d8f441d91f1267406cd93 feat(apigee): update the api https://togithub.com/googleapis/google-api-python-client/commit/e5a6b07d4a8958e2f8cf38e9ae8458e5d4b26432 feat(appengine): update the api https://togithub.com/googleapis/google-api-python-client/commit/ba7f4df4f29c75d89a9ca6da872f1dd2f17668fe feat(clouddeploy): update the api https://togithub.com/googleapis/google-api-python-client/commit/693cde3277895cdf2d2cc8acd2d226a355475df8 feat(cloudkms): update the api https://togithub.com/googleapis/google-api-python-client/commit/5bc1696640a6e554dcd3514670d2f0b7e6233b0a feat(connectors): update the api https://togithub.com/googleapis/google-api-python-client/commit/668955cf54878fbc26dcbfe0b02f0a3c742252c3 feat(contactcenterinsights): update the api https://togithub.com/googleapis/google-api-python-client/commit/44dac76832a88a9c8ff164e2171c68709852d890 feat(containeranalysis): update the api https://togithub.com/googleapis/google-api-python-client/commit/817b1f34dbdda7b560d207211e77f2df864a4cf7 feat(dataflow): update the api https://togithub.com/googleapis/google-api-python-client/commit/3c9a19f785106debf72d7a084fce3f3074abe3bd feat(datafusion): update the api https://togithub.com/googleapis/google-api-python-client/commit/b7b17f87083c6855ee2e24fe89b3fc4af9c30b83 feat(datalabeling): update the api https://togithub.com/googleapis/google-api-python-client/commit/6c8635ad88bb9ebab23452eafc7f422d33cc3ea9 feat(datalineage): update the api https://togithub.com/googleapis/google-api-python-client/commit/b0d16418a27cbdf8a64cd3f11ecc223f78c213aa fix(datastore): update the api https://togithub.com/googleapis/google-api-python-client/commit/f1c9530bf6a6008125afdfaca224c28770aa0148 feat(discoveryengine): update the api https://togithub.com/googleapis/google-api-python-client/commit/aa0227ef033d52476413d94ebdb1b3f3f91fa1d8 feat(displayvideo): update the api https://togithub.com/googleapis/google-api-python-client/commit/8ab2268c8269d972fc28563e1e94b5972e400c97 feat(documentai): update the api https://togithub.com/googleapis/google-api-python-client/commit/1740f1e8a8ba82bd3a9700b0659d8503664416f6 feat(domains): update the api https://togithub.com/googleapis/google-api-python-client/commit/68046d5fb45a88614a852b1ee6a796e2a35e391e feat(eventarc): update the api https://togithub.com/googleapis/google-api-python-client/commit/4b46b8b6cb10cc16b69e55b954cb8da84fc89d0a feat(fcm): update the api https://togithub.com/googleapis/google-api-python-client/commit/3846cb0f285d84f13df9a5721b115f6b1134ba92 feat(firebasedataconnect): update the api https://togithub.com/googleapis/google-api-python-client/commit/6be57aaa9e4a0c4637329fc45865b4d496b7bb83 fix(firestore): update the api https://togithub.com/googleapis/google-api-python-client/commit/f0388b2025852aa6c16c546156a3af242ee78adb feat(gkebackup): update the api https://togithub.com/googleapis/google-api-python-client/commit/dae76d0fc170b932414277c9de14a0e29943f536 feat(kmsinventory): update the api https://togithub.com/googleapis/google-api-python-client/commit/cb6e8f88a8e55f25ac38b8027b48a6b8cac61962 feat(logging): update the api https://togithub.com/googleapis/google-api-python-client/commit/90ec37070b66fb97436cea671922bdbdb531efd8 feat(managedidentities): update the api https://togithub.com/googleapis/google-api-python-client/commit/eeb29f4108489f86cbe6573ca34fc571d7b0c978 feat(merchantapi): update the api https://togithub.com/googleapis/google-api-python-client/commit/e61b837809d0f203f4ab6e426da02185ef014704 feat(networkmanagement): update the api https://togithub.com/googleapis/google-api-python-client/commit/c8976bf6a8c2156b03d534e365c108409380aabc feat(ondemandscanning): update the api https://togithub.com/googleapis/google-api-python-client/commit/a2e057b6deaed06d63d3e2122bd7232b4d83eb00 feat(redis): update the api https://togithub.com/googleapis/google-api-python-client/commit/cc5c15667f5d08cb9a2022de821cda9c2fc7b9db fix(run): update the api https://togithub.com/googleapis/google-api-python-client/commit/72d5e943751a6051358cbaffbb345c8bc2f12ad5 feat(saasservicemgmt): update the api https://togithub.com/googleapis/google-api-python-client/commit/415b7fa81ff7b344ecbbf37c4c46e7752171dc33 feat(secretmanager): update the api https://togithub.com/googleapis/google-api-python-client/commit/9295ba11b447bbf0259205017e158e3589bd7717 feat(securityposture): update the api https://togithub.com/googleapis/google-api-python-client/commit/118d055db0b6ff99594dd0c8f049c4df5e77b128 feat(storage): update the api https://togithub.com/googleapis/google-api-python-client/commit/d17403e9c43e3e8a1ee6cfec34c5b46857b2cd9e feat(threatintelligence): update the api https://togithub.com/googleapis/google-api-python-client/commit/aa972d86579da1061e0ad37e47e7d083536789cf feat(walletobjects): update the api https://togithub.com/googleapis/google-api-python-client/commit/46937b6d2f50039c0349d0a986b98d5e70bc8f0f feat(youtube): update the api https://togithub.com/googleapis/google-api-python-client/commit/a1a7027c43593f361a56c42596dcbcc263c6929a
1 parent 9071631 commit 49fd63f

File tree

435 files changed

+99133
-6850
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

435 files changed

+99133
-6850
lines changed

.github/blunderbuss.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
# team, please update `codeowner_team` in `.repo-metadata.json`.
66

77
assign_issues:
8-
- googleapis/cloud-sdk-python-team
8+
- googleapis/python-core-client-libraries
99

1010
assign_issues_by:
1111
- labels:
@@ -14,4 +14,4 @@ assign_issues_by:
1414
- googleapis/python-samples-reviewers
1515

1616
assign_prs:
17-
- googleapis/cloud-sdk-python-team
17+
- googleapis/python-core-client-libraries

docs/dyn/adsense_v2.accounts.reports.html

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,12 @@ <h3>Method Details</h3>
167167
CUSTOM_SEARCH_STYLE_ID - Custom search style id.
168168
DOMAIN_REGISTRANT - Domain registrants.
169169
WEBSEARCH_QUERY_STRING - Query strings for web searches.
170+
OS_TYPE_NAME - Localized operating system type name (e.g. &quot;Windows&quot;, &quot;MacOS&quot;, &quot;Android&quot;). For more information, see [Operating system breakdown](https://support.google.com/adsense/answer/16853822).
171+
OS_TYPE_CODE - Operating system type code (e.g. &quot;WINDOWS&quot;, &quot;MAC&quot;, &quot;ANDROID&quot;). For more information, see [Operating system breakdown](https://support.google.com/adsense/answer/16853822).
172+
BROWSER_TYPE_NAME - Localized browser type name (e.g. &quot;Google Chrome&quot;, &quot;Firefox&quot;, &quot;Safari&quot;). For more information, see [Browser breakdown](https://support.google.com/adsense/answer/16851903).
173+
BROWSER_TYPE_CODE - Browser type code (e.g. &quot;CHROME&quot;, &quot;FIREFOX&quot;, &quot;SAFARI&quot;). For more information, see [Browser breakdown](https://support.google.com/adsense/answer/16851903).
174+
WEBVIEW_TYPE_NAME - Localized webview type name (e.g. &quot;Webview (Uncategorized)&quot;, &quot;Non-webview&quot;). For more information, see [Hosting App breakdown](https://support.google.com/adsense/answer/16853515).
175+
WEBVIEW_TYPE_CODE - Webview type code (e.g. &quot;UNCATEGORIZED&quot;, &quot;NONE&quot;). For more information, see [Hosting App breakdown](https://support.google.com/adsense/answer/16853515).
170176
endDate_day: integer, Day of a month. Must be from 1 to 31 and valid for the year and month, or 0 to specify a year by itself or a year and month where the day isn&#x27;t significant.
171177
endDate_month: integer, Month of a year. Must be from 1 to 12, or 0 to specify a year without a month and day.
172178
endDate_year: integer, Year of the date. Must be from 1 to 9999, or 0 to specify a date without a year.
@@ -347,6 +353,12 @@ <h3>Method Details</h3>
347353
CUSTOM_SEARCH_STYLE_ID - Custom search style id.
348354
DOMAIN_REGISTRANT - Domain registrants.
349355
WEBSEARCH_QUERY_STRING - Query strings for web searches.
356+
OS_TYPE_NAME - Localized operating system type name (e.g. &quot;Windows&quot;, &quot;MacOS&quot;, &quot;Android&quot;). For more information, see [Operating system breakdown](https://support.google.com/adsense/answer/16853822).
357+
OS_TYPE_CODE - Operating system type code (e.g. &quot;WINDOWS&quot;, &quot;MAC&quot;, &quot;ANDROID&quot;). For more information, see [Operating system breakdown](https://support.google.com/adsense/answer/16853822).
358+
BROWSER_TYPE_NAME - Localized browser type name (e.g. &quot;Google Chrome&quot;, &quot;Firefox&quot;, &quot;Safari&quot;). For more information, see [Browser breakdown](https://support.google.com/adsense/answer/16851903).
359+
BROWSER_TYPE_CODE - Browser type code (e.g. &quot;CHROME&quot;, &quot;FIREFOX&quot;, &quot;SAFARI&quot;). For more information, see [Browser breakdown](https://support.google.com/adsense/answer/16851903).
360+
WEBVIEW_TYPE_NAME - Localized webview type name (e.g. &quot;Webview (Uncategorized)&quot;, &quot;Non-webview&quot;). For more information, see [Hosting App breakdown](https://support.google.com/adsense/answer/16853515).
361+
WEBVIEW_TYPE_CODE - Webview type code (e.g. &quot;UNCATEGORIZED&quot;, &quot;NONE&quot;). For more information, see [Hosting App breakdown](https://support.google.com/adsense/answer/16853515).
350362
endDate_day: integer, Day of a month. Must be from 1 to 31 and valid for the year and month, or 0 to specify a year by itself or a year and month where the day isn&#x27;t significant.
351363
endDate_month: integer, Month of a year. Must be from 1 to 12, or 0 to specify a year without a month and day.
352364
endDate_year: integer, Year of the date. Must be from 1 to 9999, or 0 to specify a date without a year.

docs/dyn/aiplatform_v1.projects.locations.evaluationItems.html

Lines changed: 16 additions & 16 deletions
Large diffs are not rendered by default.

docs/dyn/aiplatform_v1.projects.locations.evaluationRuns.html

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,12 @@ <h3>Method Details</h3>
282282
},
283283
&quot;metrics&quot;: [ # Required. The metrics to be calculated in the evaluation run.
284284
{ # The metric used for evaluation runs.
285+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
286+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
287+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
288+
},
289+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
290+
},
285291
&quot;llmBasedMetricSpec&quot;: { # Specification for an LLM based metric. # Spec for an LLM based metric.
286292
&quot;additionalConfig&quot;: { # Optional. Optional additional configuration for the metric.
287293
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
@@ -543,6 +549,12 @@ <h3>Method Details</h3>
543549
&quot;bleuSpec&quot;: { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric.
544550
&quot;useEffectiveOrder&quot;: True or False, # Optional. Whether to use_effective_order to compute bleu score.
545551
},
552+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
553+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
554+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
555+
},
556+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
557+
},
546558
&quot;customCodeExecutionSpec&quot;: { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric.
547559
&quot;evaluationFunction&quot;: &quot;A String&quot;, # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -&gt; float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text=&quot;The answer is 4.&quot;), reference=EvaluationInstance.InstanceData(text=&quot;4&quot;) ) ``` Example converted input: ``` { &#x27;response&#x27;: {&#x27;text&#x27;: &#x27;The answer is 4.&#x27;}, &#x27;reference&#x27;: {&#x27;text&#x27;: &#x27;4&#x27;} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -&gt; float: if instance&#x27;response&#x27; == instance&#x27;reference&#x27;: return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
548560
},
@@ -1535,6 +1547,12 @@ <h3>Method Details</h3>
15351547
},
15361548
&quot;metrics&quot;: [ # Required. The metrics to be calculated in the evaluation run.
15371549
{ # The metric used for evaluation runs.
1550+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
1551+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
1552+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
1553+
},
1554+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
1555+
},
15381556
&quot;llmBasedMetricSpec&quot;: { # Specification for an LLM based metric. # Spec for an LLM based metric.
15391557
&quot;additionalConfig&quot;: { # Optional. Optional additional configuration for the metric.
15401558
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
@@ -1796,6 +1814,12 @@ <h3>Method Details</h3>
17961814
&quot;bleuSpec&quot;: { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric.
17971815
&quot;useEffectiveOrder&quot;: True or False, # Optional. Whether to use_effective_order to compute bleu score.
17981816
},
1817+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
1818+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
1819+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
1820+
},
1821+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
1822+
},
17991823
&quot;customCodeExecutionSpec&quot;: { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric.
18001824
&quot;evaluationFunction&quot;: &quot;A String&quot;, # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -&gt; float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text=&quot;The answer is 4.&quot;), reference=EvaluationInstance.InstanceData(text=&quot;4&quot;) ) ``` Example converted input: ``` { &#x27;response&#x27;: {&#x27;text&#x27;: &#x27;The answer is 4.&#x27;}, &#x27;reference&#x27;: {&#x27;text&#x27;: &#x27;4&#x27;} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -&gt; float: if instance&#x27;response&#x27; == instance&#x27;reference&#x27;: return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
18011825
},
@@ -2830,6 +2854,12 @@ <h3>Method Details</h3>
28302854
},
28312855
&quot;metrics&quot;: [ # Required. The metrics to be calculated in the evaluation run.
28322856
{ # The metric used for evaluation runs.
2857+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
2858+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
2859+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
2860+
},
2861+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
2862+
},
28332863
&quot;llmBasedMetricSpec&quot;: { # Specification for an LLM based metric. # Spec for an LLM based metric.
28342864
&quot;additionalConfig&quot;: { # Optional. Optional additional configuration for the metric.
28352865
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
@@ -3091,6 +3121,12 @@ <h3>Method Details</h3>
30913121
&quot;bleuSpec&quot;: { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric.
30923122
&quot;useEffectiveOrder&quot;: True or False, # Optional. Whether to use_effective_order to compute bleu score.
30933123
},
3124+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
3125+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
3126+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
3127+
},
3128+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
3129+
},
30943130
&quot;customCodeExecutionSpec&quot;: { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric.
30953131
&quot;evaluationFunction&quot;: &quot;A String&quot;, # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -&gt; float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text=&quot;The answer is 4.&quot;), reference=EvaluationInstance.InstanceData(text=&quot;4&quot;) ) ``` Example converted input: ``` { &#x27;response&#x27;: {&#x27;text&#x27;: &#x27;The answer is 4.&#x27;}, &#x27;reference&#x27;: {&#x27;text&#x27;: &#x27;4&#x27;} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -&gt; float: if instance&#x27;response&#x27; == instance&#x27;reference&#x27;: return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
30963132
},
@@ -4096,6 +4132,12 @@ <h3>Method Details</h3>
40964132
},
40974133
&quot;metrics&quot;: [ # Required. The metrics to be calculated in the evaluation run.
40984134
{ # The metric used for evaluation runs.
4135+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
4136+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
4137+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
4138+
},
4139+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
4140+
},
40994141
&quot;llmBasedMetricSpec&quot;: { # Specification for an LLM based metric. # Spec for an LLM based metric.
41004142
&quot;additionalConfig&quot;: { # Optional. Optional additional configuration for the metric.
41014143
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
@@ -4357,6 +4399,12 @@ <h3>Method Details</h3>
43574399
&quot;bleuSpec&quot;: { # Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1. # Spec for bleu metric.
43584400
&quot;useEffectiveOrder&quot;: True or False, # Optional. Whether to use_effective_order to compute bleu score.
43594401
},
4402+
&quot;computationBasedMetricSpec&quot;: { # Specification for a computation based metric. # Spec for a computation based metric.
4403+
&quot;parameters&quot;: { # Optional. A map of parameters for the metric, e.g. {&quot;rouge_type&quot;: &quot;rougeL&quot;}.
4404+
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
4405+
},
4406+
&quot;type&quot;: &quot;A String&quot;, # Required. The type of the computation based metric.
4407+
},
43604408
&quot;customCodeExecutionSpec&quot;: { # Specificies a metric that is populated by evaluating user-defined Python code. # Spec for Custom Code Execution metric.
43614409
&quot;evaluationFunction&quot;: &quot;A String&quot;, # Required. Python function. Expected user to define the following function, e.g.: def evaluate(instance: dict[str, Any]) -&gt; float: Please include this function signature in the code snippet. Instance is the evaluation instance, any fields populated in the instance are available to the function as instance[field_name]. Example: Example input: ``` instance= EvaluationInstance( response=EvaluationInstance.InstanceData(text=&quot;The answer is 4.&quot;), reference=EvaluationInstance.InstanceData(text=&quot;4&quot;) ) ``` Example converted input: ``` { &#x27;response&#x27;: {&#x27;text&#x27;: &#x27;The answer is 4.&#x27;}, &#x27;reference&#x27;: {&#x27;text&#x27;: &#x27;4&#x27;} } ``` Example python function: ``` def evaluate(instance: dict[str, Any]) -&gt; float: if instance&#x27;response&#x27; == instance&#x27;reference&#x27;: return 1.0 return 0.0 ``` CustomCodeExecutionSpec is also supported in Batch Evaluation (EvalDataset RPC) and Tuning Evaluation. Each line in the input jsonl file will be converted to dict[str, Any] and passed to the evaluation function.
43624410
},

0 commit comments

Comments
 (0)