-
Notifications
You must be signed in to change notification settings - Fork 5
Automate package example data on new asar releases #147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Schiano-NOAA
wants to merge
1
commit into
main
Choose a base branch
from
automate-example-data
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| # This workflow lives in Repository B (the one that runs the function) | ||
|
|
||
| name: 2. Run Function from Released Package | ||
|
|
||
| on: | ||
| repository_dispatch: | ||
| # Only run if the event type matches the one sent by Repo A | ||
| types: [release_trigger] | ||
|
|
||
| jobs: | ||
| run_released_function: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| # IMPORTANT: Grant write permission to the GITHUB_TOKEN for committing and creating the PR | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write # Explicitly needed for creating the PR | ||
|
|
||
| # Define variables extracted from the client-payload and constants for the PR | ||
| env: | ||
| RELEASE_TAG: ${{ github.event.client_payload.release_tag }} | ||
| REPO_A_NAME: ${{ github.event.client_payload.repository_a_name }} | ||
| # Define the target branch name for the new changes | ||
| PR_BRANCH_NAME: feature/update-data-from-repo-A-${{ github.event.client_payload.release_tag }} | ||
|
|
||
| steps: | ||
| # STEP 1: Checkout Repository B's code to access its data and allow committing back. | ||
| - name: Checkout Repository B (Data Source & Destination) | ||
| uses: actions/checkout@v4 | ||
| # Defaults to checking out the current repo (Repo B) into the workspace root | ||
|
|
||
| - name: Install R and dependencies (Example) | ||
| uses: r-lib/actions/setup-r@v2 | ||
|
|
||
| # Install devtools, needed for loading the source package from Repo A | ||
| - name: Install R Package Dependencies (devtools) | ||
| run: | | ||
| install.packages("devtools") | ||
|
|
||
| # STEP 2: Download Repository A's source code (Function Source) | ||
| - name: Download Repository A's source code | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: ${{ env.REPO_A_NAME }} | ||
| ref: ${{ env.RELEASE_TAG }} | ||
| path: 'repo_a_source' | ||
|
|
||
| # STEP 3: Run the local R script that loads Repo A's function and saves the data. | ||
| - name: Generate and Save Example Data | ||
| run: Rscript data-raw/update_example_data.R | ||
|
|
||
| # STEP 4: Commit the newly created data file to a new branch. | ||
| - name: Commit Data and Set PR Flag | ||
| run: | | ||
| # 1. Configure Git user for the commit | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| # 2. Check for changes in the target data file | ||
| if git status --porcelain | grep data/example_data.rda; then | ||
| # 3. Create and switch to the new feature branch | ||
| git checkout -b ${{ env.PR_BRANCH_NAME }} | ||
|
|
||
| # 4. Commit the file | ||
| git add data/example_data.rda | ||
| git commit -m "CI: Update example_data.rda from Repo A release ${{ env.RELEASE_TAG }}" | ||
|
|
||
| # 5. Set environment variable to proceed with PR | ||
| echo "COMMITTED=true" >> $GITHUB_ENV | ||
| echo "Changes committed. Proceeding to push and create PR." | ||
| else | ||
| echo "COMMITTED=false" >> $GITHUB_ENV | ||
| echo "No changes detected in data/example_data.rda. Skipping PR creation." | ||
| fi | ||
|
|
||
| # STEP 5: Push the new branch to GitHub (only if changes were committed) | ||
| - name: Push New Branch | ||
| if: env.COMMITTED == 'true' | ||
| run: git push origin ${{ env.PR_BRANCH_NAME }} | ||
|
|
||
| # STEP 6: Create the Pull Request | ||
| - name: Create Pull Request | ||
| if: env.COMMITTED == 'true' | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| # Use the default GITHUB_TOKEN which has write access due to permissions block | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| script: | | ||
| const prBranch = process.env.PR_BRANCH_NAME; | ||
| const baseBranch = context.ref.replace('refs/heads/', ''); // The branch that received the dispatch | ||
|
|
||
| const title = `[CI] Data Update: New 'example_data.rda' from Repo A \`${process.env.RELEASE_TAG}\``; | ||
| const body = `This pull request automatically updates \`data/example_data.rda\` following the release of **${process.env.RELEASE_TAG}** in Repository A (${process.env.REPO_A_NAME}). | ||
|
|
||
| The new data was generated using the \`${process.env.RELEASE_TAG}\` version of the conversion function. Please review the changes before merging.`; | ||
|
|
||
| try { | ||
| const pr = await github.rest.pulls.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| title: title, | ||
| head: prBranch, | ||
| base: baseBranch, | ||
| body: body | ||
| }); | ||
| console.log('Pull Request created successfully:', pr.data.html_url); | ||
| } catch (error) { | ||
| console.error('Failed to create pull request. This may happen if a PR for this release tag already exists.'); | ||
| } | ||
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| ## code to prepare `DATASET` dataset goes here | ||
|
|
||
| # This script runs inside the GitHub Action environment (nmfs-ost/stockplotr) | ||
|
|
||
| # Ensure devtools is installed (installed in the workflow, but good practice to check) | ||
| if (!requireNamespace("devtools", quietly = TRUE)) { | ||
| install.packages("devtools") | ||
| } | ||
| library(devtools) | ||
|
|
||
| # --- Configuration (Relative Paths inside Repo B's workspace) --- | ||
| # This path is set by the workflow step that checks out 'asar' | ||
| REPO_A_SOURCE_PATH <- "repo_a_source" | ||
|
|
||
| # VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV | ||
| # --- CUSTOMIZE THESE THREE VARIABLES --- | ||
|
|
||
| # 1. The name of the function we need to call from Repo A (nmfs-ost/asar) | ||
| FUNCTION_NAME <- "convert_output" | ||
|
|
||
| # 2. The path to the raw text data in Repo B (nmfs-ost/stockplotr) | ||
| # NOTE: The raw data is typically stored in data-raw/ to keep it out of the package build. | ||
| RAW_DATA_PATH <- "data-raw/Report.sso" | ||
|
|
||
| # 3. The name of the object (and file prefix) for the new example data | ||
| OUTPUT_OBJECT_NAME <- "example_data" | ||
|
|
||
| # ^^^^ END CUSTOMIZATION ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| # The output path for the R data file in Repo B's data/ directory | ||
| OUTPUT_FILE_PATH <- paste0("data/", OUTPUT_OBJECT_NAME, ".rda") | ||
|
|
||
|
|
||
| # --- Main Logic --- | ||
|
|
||
| # 1. Load the functions from Repository A's source package ('asar') | ||
| message("Loading functions from nmfs-ost/asar source: ", REPO_A_SOURCE_PATH) | ||
| tryCatch({ | ||
| # This makes all functions from 'asar' available in this R session | ||
| load_all(REPO_A_SOURCE_PATH, quiet = TRUE) | ||
| }, error = function(e) { | ||
| stop(paste("Failed to load Repository A source (asar):", e$message)) | ||
| }) | ||
|
|
||
|
|
||
| # 2. Check for the function existence and retrieve it | ||
| if (!exists(FUNCTION_NAME)) { | ||
| stop(paste0("Function '", FUNCTION_NAME, "' not found in nmfs-ost/asar source.")) | ||
| } | ||
| conversion_function <- get(FUNCTION_NAME) | ||
|
|
||
|
|
||
| # 3. Read the data from Repository B (stockplotr) | ||
| message("Reading raw data from nmfs-ost/stockplotr: ", RAW_DATA_PATH) | ||
| # NOTE: Adjust the read function (read.csv, read.table, etc.) based on your raw file | ||
| raw_data <- RAW_DATA_PATH | ||
|
|
||
|
|
||
| # 4. Run the function from Repo A on the data from Repo B | ||
| message("Running conversion function ", FUNCTION_NAME, "...") | ||
| # The resulting object is the data frame for the package | ||
| converted_data_frame <- conversion_function(raw_data) | ||
|
|
||
|
|
||
| # 5. Save the resulting data frame into the data/ directory of Repo B | ||
| message("Saving generated data frame as R package data: ", OUTPUT_FILE_PATH) | ||
| if (!dir.exists("data")) { dir.create("data") } | ||
|
|
||
| # Assign the resulting data frame to the specified object name | ||
| # This object name will be what the user loads via `data(asar_catch_data)` | ||
| assign(OUTPUT_OBJECT_NAME, converted_data_frame) | ||
|
|
||
| # save() is used to create the .rda file for the package | ||
| save(list = OUTPUT_OBJECT_NAME, file = OUTPUT_FILE_PATH, compress = "gzip") | ||
|
|
||
| message("Data generation complete. File saved to ", OUTPUT_FILE_PATH) | ||
|
|
||
| # Exit successfully | ||
| q(save = "no") | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium