test: run the bigtable system tests in google cloud build#8244
test: run the bigtable system tests in google cloud build#8244danieljbruce wants to merge 35 commits into
Conversation
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
…com/googleapis/google-cloud-node into migrate-bigtable-to-gcp-cloud-build
There was a problem hiding this comment.
Code Review
This pull request introduces a Cloud Build configuration for Bigtable system tests and refactors the tabular-api-surface.ts file to avoid importing from internal google-gax paths by defining a local StrictLongRunningType. It also fixes pack-n-play test issues by adding necessary node type references. Feedback includes recommendations to avoid global npm updates in CI, use standard binary paths for nyc, remove hardcoded project IDs, reduce build timeouts, and export the newly created type for external visibility.
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| npm install -g npm@latest | ||
| npm config set prefix /usr/local | ||
| npm install | ||
| npm run presystem-test |
There was a problem hiding this comment.
Updating npm globally and changing the configuration prefix in a CI environment is generally discouraged as it can lead to non-deterministic builds and potential permission issues. The gcr.io/cloud-builders/npm image should already provide a functional npm version. If a specific version of Node.js or npm is required, it is better to use a specific image tag or pin the version explicitly.
if [ -f /workspace/skip_tests ]; then exit 0; fi
npm install
npm run presystem-testThere was a problem hiding this comment.
On the other PR I committed a code change to fix the node version.
| if [ -f ./node_modules/nyc/bin/nyc.js ]; then | ||
| ./node_modules/nyc/bin/nyc.js report || true # `|| true` prevents build failure if nyc report itself exits non-zero |
There was a problem hiding this comment.
Avoid reaching into the internal directory structure of a package to execute its binary. It is more robust to use the binary located in node_modules/.bin/ or use npx.
if [ -f ./node_modules/.bin/nyc ]; then
./node_modules/.bin/nyc report || true # || true prevents build failure if nyc report itself exits non-zeroThere was a problem hiding this comment.
Maybe, but not a serious issue since the old scripts did it this way anyways.
| # Define a substitution variable for your project ID | ||
| # Replace 'long-door-651' with the actual GCP Project ID your system tests should run against. | ||
| substitutions: | ||
| _GCP_PROJECT_ID: 'long-door-651' |
There was a problem hiding this comment.
There was a problem hiding this comment.
This value was shared before therefore sharing the specific id has no downside.
| substitutions: | ||
| _GCP_PROJECT_ID: 'long-door-651' | ||
|
|
||
| timeout: '10800s' |
There was a problem hiding this comment.
There was a problem hiding this comment.
No. We need this much time.
| // StrictLongRunningType introduces a type equal to BackoffSettings so that we | ||
| // don't have to reach into the gax folder of build which is causing pack and | ||
| // play errors. | ||
| type StrictLongRunningType = NonNullable<CallOptions['longrunning']>; |
There was a problem hiding this comment.
The type StrictLongRunningType is used in the signature of the exported function getNextDelay and the exported constant DEFAULT_BACKOFF_SETTINGS. To ensure that consumers of this library can correctly reference the type, it should also be exported.
| type StrictLongRunningType = NonNullable<CallOptions['longrunning']>; | |
| export type StrictLongRunningType = NonNullable<CallOptions['longrunning']>; |
There was a problem hiding this comment.
The constant and the function are internal
|
Duplicate |
Description
This pull request adds a yaml file to instruct the Bigtable system test CI check to work with the new Cloud Build trigger thereby making the new CI check effectively run our system tests. It also includes minor code modifications to the Bigtable handwritten layer that allow the system tests to operate properly in the new Cloud Build Environment.
Impact
Leverages the strengths of running system tests in GCB rather than relying on kokoro for system tests.
Testing
This pull request tells the tests how to work with the new check to the continuous integration pipeline for the Bigtable system tests thus improves the effectiveness of that test.
Next Steps