-
Notifications
You must be signed in to change notification settings - Fork 674
test: run the bigtable system tests in google cloud build #8244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f14703f
248638f
cdb2ad1
372a03b
7caf8a0
55b0dec
7a3fadd
5f930f9
f3772ea
83f73dd
d7782b2
6be7893
1f71bd1
e6cc411
c836303
1ab42bc
8e6ff1d
051f019
20b11db
a74e61e
0aae31a
af443d1
57bbdad
f6d43fe
8dd7d70
fd18c1a
bfe5155
e51a2e8
11f861b
88a19c7
c6a65a8
83539cc
9f6587c
9916905
3ce9c3f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| steps: | ||
| # 0. Check for changes in handwritten/bigtable | ||
| - name: 'gcr.io/cloud-builders/git' | ||
| id: 'check-changes' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| git fetch origin main --quiet | ||
| # Compare with origin/main to detect changes. | ||
| if git diff --quiet origin/main... -- . ; then | ||
| echo "No changes in handwritten/bigtable. Skipping tests." | ||
| echo "skip" > /workspace/skip_tests | ||
| else | ||
| echo "Changes detected in handwritten/bigtable. Proceeding with tests." | ||
| fi | ||
| dir: 'handwritten/bigtable' | ||
|
|
||
| # 1. Set up Node.js environment | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| npm install -g npm@latest | ||
| npm config set prefix /usr/local | ||
| npm install | ||
| npm run presystem-test | ||
| dir: 'handwritten/bigtable' | ||
| id: 'install-dependencies' | ||
| waitFor: ['check-changes'] | ||
|
|
||
| # 2. Configure environment variables for the tests and run system tests | ||
| # - GOOGLE_APPLICATION_CREDENTIALS: GCB steps run as a service account | ||
| # that is typically granted permissions directly. Explicitly setting | ||
| # GOOGLE_APPLICATION_CREDENTIALS might not be needed if the GCB service | ||
| # account has the right roles (e.g., Bigtable Admin, Bigtable User). | ||
| # If you need to use specific service account key JSON, you'd store it | ||
| # in Secret Manager and mount it here. For simplicity, we'll rely on | ||
| # the GCB service account's inherent permissions. | ||
| # - GCLOUD_PROJECT: Can be passed as a build variable. | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| npm run system-test | ||
| dir: 'handwritten/bigtable' | ||
| env: | ||
| - 'GCLOUD_PROJECT=${_GCP_PROJECT_ID}' # Pass project ID via build variable | ||
| # If you need specific credentials from Secret Manager, uncomment these: | ||
| # - 'GOOGLE_APPLICATION_CREDENTIALS=/secrets/sa-key.json' | ||
| id: 'run-system-tests' | ||
| waitFor: ['install-dependencies'] | ||
| # For Secret Manager, uncomment these (adjust secret name and volume path as needed): | ||
| # secretEnv: ['SA_KEY'] | ||
| # volumes: | ||
| # - name: 'sa-keys' | ||
| # path: '/secrets' | ||
|
|
||
| # 3. (Optional) Code Coverage Reporting | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| # Check if nyc is installed and run report | ||
| if [ -f ./node_modules/nyc/bin/nyc.js ]; then | ||
| ./node_modules/nyc/bin/nyc.js report || true # `|| true` prevents build failure if nyc report itself exits non-zero | ||
|
Comment on lines
+71
to
+72
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid reaching into the internal directory structure of a package to execute its binary. It is more robust to use the binary located in if [ -f ./node_modules/.bin/nyc ]; then
./node_modules/.bin/nyc report || true # || true prevents build failure if nyc report itself exits non-zero
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe, but not a serious issue since the old scripts did it this way anyways. |
||
| else | ||
| echo "nyc not found, skipping coverage report." | ||
| fi | ||
| # The original codecov.sh script from Kokoro needs to be made available to GCB. | ||
| # Options: | ||
| # a) Commit codecov.sh into your repo (e.g., .kokoro/codecov.sh) and call it: | ||
| # if [ -f .kokoro/codecov.sh ]; then . ./.kokoro/codecov.sh; fi | ||
| # b) Replicate its functionality directly in this step. | ||
| # c) Store it in a GCS bucket and fetch it. | ||
| echo "Codecov reporting (if desired) would be integrated here." | ||
| dir: 'handwritten/bigtable' | ||
| id: 'coverage-report' | ||
| waitFor: ['run-system-tests'] | ||
|
|
||
| # If you use Secret Manager for credentials, uncomment and configure: | ||
| # availableSecrets: | ||
| # secretManager: | ||
| # - versionName: projects/${PROJECT_ID}/secrets/YOUR_SERVICE_ACCOUNT_KEY_SECRET_NAME/versions/latest | ||
| # env: 'SA_KEY' # This env var will hold the secret value. Use it as GOOGLE_APPLICATION_CREDENTIALS in step 3 if needed. | ||
|
|
||
| # Define a substitution variable for your project ID | ||
| # Replace 'long-door-651' with the actual GCP Project ID your system tests should run against. | ||
| substitutions: | ||
| _GCP_PROJECT_ID: 'long-door-651' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This value was shared before therefore sharing the specific id has no downside. |
||
|
|
||
| timeout: '10800s' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No. We need this much time. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updating
npmglobally and changing the configuration prefix in a CI environment is generally discouraged as it can lead to non-deterministic builds and potential permission issues. Thegcr.io/cloud-builders/npmimage should already provide a functionalnpmversion. If a specific version of Node.js or npm is required, it is better to use a specific image tag or pin the version explicitly.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the other PR I committed a code change to fix the node version.