-
-
Notifications
You must be signed in to change notification settings - Fork 28
[#1642] Added support for migration DB. #2269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,6 +35,9 @@ commands: | |
| cmd: | | ||
| COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-${PWD##*/}} \ | ||
| VORTEX_HOST_DB_PORT=$(docker compose port database 3306 2>/dev/null | cut -d : -f 2) \ | ||
| #;< MIGRATION | ||
| VORTEX_HOST_DB2_PORT=$(docker compose port database2 3306 2>/dev/null | cut -d : -f 2) \ | ||
| #;> MIGRATION | ||
| VORTEX_HOST_SOLR_PORT=$(docker compose port solr 8983 2>/dev/null | cut -d : -f 2) \ | ||
| VORTEX_HOST_SELENIUM_VNC_PORT=$(docker compose port chrome 7900 2>/dev/null | cut -d : -f 2) \ | ||
| VORTEX_HOST_HAS_SEQUELACE=$(uname -a | grep -i -q darwin && mdfind -name 'Sequel Ace' 2>/dev/null | grep -q "Ace" && echo 1 || true) \ | ||
|
|
@@ -48,6 +51,16 @@ commands: | |
| open "mysql://${DATABASE_USERNAME:-drupal}:${DATABASE_PASSWORD:-drupal}@127.0.0.1:${VORTEX_HOST_DB_PORT}/drupal" -a "Sequel Ace" \ | ||
| || echo "Not a supported OS or Sequel Ace is not installed." | ||
|
|
||
| #;< MIGRATION | ||
| db2: | ||
| usage: Open DB2 in Sequel Ace. | ||
| cmd: | | ||
| uname -a | grep -i -q darwin && test -d "${VORTEX_HOST_SEQUELACE_PATH:-/Applications/Sequel Ace.app}" && \ | ||
| VORTEX_HOST_DB2_PORT=$(docker compose port database2 3306 2>/dev/null | cut -d : -f 2) && \ | ||
| open "mysql://${DATABASE2_USERNAME:-drupal}:${DATABASE2_PASSWORD:-drupal}@127.0.0.1:${VORTEX_HOST_DB2_PORT}/drupal" -a "Sequel Ace" \ | ||
| || echo "Not a supported OS or Sequel Ace is not installed." | ||
| #;> MIGRATION | ||
|
|
||
| # ---------------------------------------------------------------------------- | ||
| # Container commands. | ||
| # ---------------------------------------------------------------------------- | ||
|
|
@@ -125,6 +138,15 @@ commands: | |
| cmd: | | ||
| case " $* " in *" --fresh "*) export VORTEX_DOWNLOAD_DB_FRESH=1;; esac | ||
| ./scripts/vortex/download-db.sh | ||
|
|
||
| #;< MIGRATION | ||
| download-db2: | ||
| usage: Download second database (migration). Run with "--fresh" to force. | ||
| aliases: [fetch-db2] | ||
| cmd: | | ||
| case " $* " in *" --fresh "*) export VORTEX_DOWNLOAD_DB_FORCE=1;; esac | ||
| VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2 ./scripts/vortex/download-db.sh | ||
| #;> MIGRATION | ||
|
Comment on lines
+142
to
+149
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: # Check what variables download-db.sh actually uses
rg -n "VORTEX_DOWNLOAD_DB_FORCE|VORTEX_DOWNLOAD_DB_FRESH" scripts/vortex/download-db.shRepository: drevops/vortex Length of output: 278 🏁 Script executed: # Check the download-db command to verify the variable it uses
sed -n '135,145p' .ahoy.ymlRepository: drevops/vortex Length of output: 446 🏁 Script executed: # Get more context around both commands
sed -n '130,155p' .ahoy.ymlRepository: drevops/vortex Length of output: 998 Fix variable name mismatch in The 🤖 Prompt for AI Agents |
||
| #;> !PROVISION_TYPE_PROFILE | ||
|
|
||
| reload-db: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -226,6 +226,61 @@ VORTEX_DOWNLOAD_DB_ACQUIA_DB_NAME=your_site | |
|
|
||
| #;> !PROVISION_TYPE_PROFILE | ||
|
|
||
| #;< MIGRATION | ||
| ################################################################################ | ||
| # SECOND DATABASE (MIGRATION) # | ||
| ################################################################################ | ||
|
|
||
| # Second database for migrations. | ||
| # See settings.migration.php for database credentials. | ||
|
|
||
| # Second database dump file name. | ||
| VORTEX_DOWNLOAD_DB2_FILE=db2.sql | ||
|
|
||
| # Second database download source. | ||
| VORTEX_DOWNLOAD_DB2_SOURCE=url | ||
|
|
||
| #;< MIGRATION_DB_DOWNLOAD_SOURCE_URL | ||
| # Second database dump file sourced from a URL. | ||
| # | ||
| # HTTP Basic Authentication credentials should be embedded into the value. | ||
| VORTEX_DOWNLOAD_DB2_URL= | ||
|
Comment on lines
+243
to
+247
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid duplicate The key appears in both the MIGRATION and DEMO blocks, which triggers the dotenv-linter duplication warning and makes precedence unclear. Consider commenting out one, using a distinct demo-only key, or gating so only one active value remains. Also applies to: 361-365 🤖 Prompt for AI Agents |
||
| #;> MIGRATION_DB_DOWNLOAD_SOURCE_URL | ||
|
|
||
| #;< MIGRATION_DB_DOWNLOAD_SOURCE_FTP | ||
| # Second database dump file sourced from FTP. | ||
|
|
||
| # Second database dump FTP host. | ||
| VORTEX_DOWNLOAD_DB2_FTP_HOST= | ||
|
|
||
| # Second database dump FTP port. | ||
| VORTEX_DOWNLOAD_DB2_FTP_PORT=21 | ||
|
|
||
| # Second database dump FTP file name. | ||
| VORTEX_DOWNLOAD_DB2_FTP_FILE=db2.sql | ||
| #;> MIGRATION_DB_DOWNLOAD_SOURCE_FTP | ||
|
|
||
| #;< MIGRATION_DB_DOWNLOAD_SOURCE_S3 | ||
| # Second database dump file sourced from S3. | ||
|
|
||
| # AWS S3 bucket name for second database download. | ||
| VORTEX_DOWNLOAD_DB2_S3_BUCKET= | ||
|
|
||
| # AWS S3 region. | ||
| VORTEX_DOWNLOAD_DB2_S3_REGION=ap-southeast-2 | ||
| #;> MIGRATION_DB_DOWNLOAD_SOURCE_S3 | ||
|
|
||
| # Environment to download the second database from. | ||
| # | ||
| # Applies to hosting environments. | ||
| VORTEX_DOWNLOAD_DB2_ENVIRONMENT=prod | ||
|
|
||
| #;< HOSTING_ACQUIA | ||
| # Acquia database name to download the second database from. | ||
| VORTEX_DOWNLOAD_DB2_ACQUIA_DB_NAME=your_site | ||
| #;> HOSTING_ACQUIA | ||
| #;> MIGRATION | ||
|
|
||
| ################################################################################ | ||
| # RELEASE VERSIONING # | ||
| ################################################################################ | ||
|
|
@@ -303,6 +358,13 @@ VORTEX_NOTIFY_WEBHOOK_URL= | |
| VORTEX_DOWNLOAD_DB_URL=https://github.com/drevops/vortex/releases/download/25.4.0/db_d11.demo.sql | ||
| #;> DB_DOWNLOAD_SOURCE_URL | ||
|
|
||
| #;< MIGRATION | ||
| #;< MIGRATION_DB_DOWNLOAD_SOURCE_URL | ||
| # URL of the migration source database used for demonstration. | ||
| VORTEX_DOWNLOAD_DB2_URL=https://github.com/drevops/vortex/releases/download/25.4.0/db_d11.demo_source.sql | ||
| #;> MIGRATION_DB_DOWNLOAD_SOURCE_URL | ||
| #;> MIGRATION | ||
|
|
||
| #;< DB_DOWNLOAD_SOURCE_CONTAINER_REGISTRY | ||
| # Demo database container image used for demonstration of database-in-image | ||
| # data storage. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -122,11 +122,11 @@ jobs: | |
| uses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5 | ||
| with: | ||
| path: .data | ||
| key: v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| key: v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| # Fallback to caching by default branch name only. Allows to use | ||
| # cache from the branch build on the previous day. | ||
| restore-keys: | | ||
| v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback') }}- | ||
| v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback') }}- | ||
|
|
||
| - name: Download DB | ||
| run: | | ||
|
|
@@ -139,6 +139,11 @@ jobs: | |
| VORTEX_ACQUIA_SECRET: ${{ secrets.VORTEX_ACQUIA_SECRET }} | ||
| #;> DB_DOWNLOAD_SOURCE_ACQUIA | ||
|
|
||
| #;< MIGRATION | ||
| - name: Download migration DB | ||
| run: VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2 ./scripts/vortex/download-db.sh | ||
| #;> MIGRATION | ||
|
Comment on lines
+142
to
+145
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Run the command from the review comment to find references
rg -nP "VORTEX_DOWNLOAD_DB2_(FILE|DIR)|VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2"Repository: drevops/vortex Length of output: 497 🏁 Script executed: #!/bin/bash
# First, let's look at the actual workflow file around lines 142-145 and 348-351
cat -n .github/workflows/build-test-deploy.yml | sed -n '135,155p'
echo "---"
cat -n .github/workflows/build-test-deploy.yml | sed -n '340,360p'Repository: drevops/vortex Length of output: 2275 🏁 Script executed: #!/bin/bash
# Check if the download-db.sh script exists and examine it
find . -name "download-db.sh" -type fRepository: drevops/vortex Length of output: 89 🏁 Script executed: #!/bin/bash
# Examine the download-db.sh script
wc -l ./scripts/vortex/download-db.shRepository: drevops/vortex Length of output: 92 🏁 Script executed: #!/bin/bash
# Read the download-db.sh script to understand its behavior
cat -n ./scripts/vortex/download-db.shRepository: drevops/vortex Length of output: 4519 🏁 Script executed: #!/bin/bash
# Look for how VORTEX_VAR_PREFIX is used in the codebase
rg -nA5 "VORTEX_VAR_PREFIX" ./scriptsRepository: drevops/vortex Length of output: 4298 Set When 🔧 Suggested fix - name: Download migration DB
run: VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2 ./scripts/vortex/download-db.sh
+ env:
+ VORTEX_DOWNLOAD_DB2_FILE: db2.sql
+ VORTEX_DOWNLOAD_DB2_DIR: .dataApplies to lines 142-145 and 348-351. 🤖 Prompt for AI Agents |
||
|
|
||
| - name: Export DB | ||
| run: | | ||
| if [ ! -f /tmp/download-db-success ]; then echo "==> Database download semaphore file is missing. DB export will not proceed."; exit 0; fi | ||
|
|
@@ -158,7 +163,7 @@ jobs: | |
| if: env.db_hash != hashFiles('.data') | ||
| with: | ||
| path: .data | ||
| key: v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| key: v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| #;> !PROVISION_TYPE_PROFILE | ||
|
|
||
| build: | ||
|
|
@@ -223,7 +228,7 @@ jobs: | |
| date "${VORTEX_CI_DB_CACHE_TIMESTAMP}" | tee db_cache_timestamp | ||
|
|
||
| - name: Show cache key for database caching | ||
| run: echo 'v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }}' | ||
| run: echo 'v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }}' | ||
|
|
||
| # Restore DB cache based on the cache strategy set by the cache keys below. | ||
| # Change 'v1' to 'v2', 'v3' etc., commit and push to force cache reset. | ||
|
|
@@ -235,9 +240,9 @@ jobs: | |
| path: .data | ||
| fail-on-cache-miss: true | ||
| # Use cached database from previous builds of this branch. | ||
| key: v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| key: v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}-${{ hashFiles('db_cache_timestamp') }} | ||
| restore-keys: | | ||
| v26.1.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}- | ||
| v26.2.0-db11-${{ hashFiles('db_cache_branch') }}-${{ hashFiles('db_cache_fallback_yes') }}- | ||
| #;> !PROVISION_TYPE_PROFILE | ||
|
|
||
| - name: Login to container registry | ||
|
|
@@ -340,6 +345,11 @@ jobs: | |
| if [ -f .data/db.sql ]; then | ||
| docker compose exec cli mkdir -p .data | ||
| docker compose cp -L .data/db.sql cli:/app/.data/db.sql | ||
| #;< MIGRATION | ||
| if [ -f ".data/${VORTEX_DOWNLOAD_DB2_FILE:-db2.sql}" ]; then | ||
| docker compose cp -L ".data/${VORTEX_DOWNLOAD_DB2_FILE:-db2.sql}" cli:"/app/.data/${VORTEX_DOWNLOAD_DB2_FILE:-db2.sql}" | ||
| fi | ||
| #;> MIGRATION | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| fi | ||
| docker compose exec $(env | cut -f1 -d= | sed 's/^/-e /') -T cli ./scripts/vortex/provision.sh | ||
| timeout-minutes: 30 | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -61,6 +61,16 @@ tasks: | |||||||||||||||||||||||||||||||||||||||||
| service: cli | ||||||||||||||||||||||||||||||||||||||||||
| #;> !PROVISION_TYPE_PROFILE | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| #;< MIGRATION | ||||||||||||||||||||||||||||||||||||||||||
| - run: | ||||||||||||||||||||||||||||||||||||||||||
| name: Download migration database | ||||||||||||||||||||||||||||||||||||||||||
| command: | | ||||||||||||||||||||||||||||||||||||||||||
| export VORTEX_DOWNLOAD_DB_SSH_FILE=false | ||||||||||||||||||||||||||||||||||||||||||
| export VORTEX_DB_DIR=/tmp/data | ||||||||||||||||||||||||||||||||||||||||||
| VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2 ./scripts/vortex/download-db.sh | ||||||||||||||||||||||||||||||||||||||||||
| service: cli | ||||||||||||||||||||||||||||||||||||||||||
| #;> MIGRATION | ||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+64
to
+72
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Verify intended behavior for migration database download. Two potential concerns:
If migrations should run in all environments, consider adding a comment to clarify this is intentional. Suggested fix (if non-production only) #;< MIGRATION
- run:
name: Download migration database
command: |
+ if [ "$LAGOON_ENVIRONMENT_TYPE" != "production" ] && [ "$LAGOON_GIT_BRANCH" != "${VORTEX_LAGOON_PRODUCTION_BRANCH:-main}" ]; then
export VORTEX_DOWNLOAD_DB_SSH_FILE=false
export VORTEX_DB_DIR=/tmp/data
VORTEX_VAR_PREFIX=VORTEX_DOWNLOAD_DB2 ./scripts/vortex/download-db.sh
+ fi
service: cli
#;> MIGRATION📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| - run: | ||||||||||||||||||||||||||||||||||||||||||
| name: Provision site | ||||||||||||||||||||||||||||||||||||||||||
| command: | | ||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| #!/usr/bin/env bash | ||
| ## | ||
| # Create migration source database with sample data. | ||
| # | ||
| # This script populates the migration source database with a `categories` table | ||
| # containing sample data for demonstration purposes. | ||
| # | ||
| # Usage: | ||
| # DB_URL=mysql://user:pass@host:port/dbname ./create-migrate-source-db.sh | ||
| # | ||
| # Or using drush: | ||
| # $(drush sql:connect) < <(./create-migrate-source-db.sh --sql) | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| set -eu | ||
| [ "${VORTEX_DEBUG-}" = "1" ] && set -x | ||
|
|
||
| sql=$( | ||
| cat <<'SQL' | ||
| DROP TABLE IF EXISTS `categories`; | ||
|
|
||
| CREATE TABLE `categories` ( | ||
| `id` int(11) unsigned NOT NULL AUTO_INCREMENT, | ||
| `name` varchar(255) NOT NULL, | ||
| `description` text DEFAULT NULL, | ||
| PRIMARY KEY (`id`) | ||
| ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; | ||
|
|
||
| INSERT INTO `categories` (`id`, `name`, `description`) VALUES | ||
| (1, 'Jedi Order', 'Ancient order of Force-sensitive protectors of peace and justice in the galaxy.'), | ||
| (2, 'Sith Lords', 'Dark side practitioners who seek power and dominion over the galaxy.'), | ||
| (3, 'Galactic Empire', 'Authoritarian government that ruled the galaxy through fear and military might.'), | ||
| (4, 'Rebel Alliance', 'Coalition of resistance fighters opposing the tyranny of the Galactic Empire.'), | ||
| (5, 'Bounty Hunters', 'Mercenaries and trackers who pursue targets for credits across the galaxy.'), | ||
| (6, 'Smugglers', 'Independent pilots and traders who operate outside the law for profit.'), | ||
| (7, 'Droids', 'Mechanical beings serving various roles from protocol to astromech duties.'), | ||
| (8, 'Mandalorians', 'Warrior culture known for their beskar armor and combat traditions.'), | ||
| (9, 'Wookiees', 'Tall, fur-covered species from Kashyyyk known for their strength and loyalty.'), | ||
| (10, 'Hutts', 'Powerful crime lords who control vast criminal enterprises across the Outer Rim.'); | ||
| SQL | ||
| ) | ||
|
|
||
| if [ "${1:-}" = "--sql" ]; then | ||
| printf '%s\n' "${sql}" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "${sql}" | ||
|
Comment on lines
+42
to
+47
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Redundant conditional: both branches output identical content. The
💡 Option 1: Simplify by removing the flag-if [ "${1:-}" = "--sql" ]; then
- printf '%s\n' "${sql}"
- exit 0
-fi
-
-echo "${sql}"
+printf '%s\n' "${sql}"🤖 Prompt for AI Agents |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use
DATABASE2_NAMEindb2command to respect custom DB names.Hardcoding
/drupalbreaks when DB2 name differs from the default.✏️ Suggested fix
📝 Committable suggestion
🤖 Prompt for AI Agents