Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
766ac46
Merge pull request #117 from FRRouting/stable
RodrigoMNardi Jun 24, 2025
8c623e7
[GBA-9] Allow multiple plans
RodrigoMNardi Aug 19, 2025
de15d51
[GBA-9] Allow multiple plans
RodrigoMNardi Aug 19, 2025
90ba07e
Merge remote-tracking branch 'origin/master' into feature/gba-9/mulit…
RodrigoMNardi Aug 19, 2025
bb7be2f
[GBA-9] Allow multiple plans
RodrigoMNardi Aug 22, 2025
ea5e767
[GBA-9] Allow multiple plans
RodrigoMNardi Aug 29, 2025
ae3d373
[GBA-9] Allow multiple plans
RodrigoMNardi Aug 29, 2025
c62f90e
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 1, 2025
2ccf9e4
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 1, 2025
89d2764
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 1, 2025
4ddfa19
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 2, 2025
918e1cc
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 3, 2025
784b38f
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 9, 2025
47b7448
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 10, 2025
6f5a728
[GBA-9] Allow multiple plans
RodrigoMNardi Sep 23, 2025
ec5322c
[GBA-9] Allow multiple plans
RodrigoMNardi Oct 28, 2025
291f52b
[GBA-9] Allow multiple plans
RodrigoMNardi Oct 28, 2025
6eeb448
[GBA-9] Allow multiple plans
RodrigoMNardi Oct 28, 2025
faee8e5
Merge pull request #119 from RodrigoMNardi/feature/gba-9/mulitple-plans
RodrigoMNardi Oct 28, 2025
26442b2
Add Prometheus metrics integration and middleware
RodrigoMNardi May 9, 2026
77c2fbc
Add Prometheus metrics endpoint and update configuration
RodrigoMNardi May 12, 2026
c4c27a7
Refactor enum declarations for consistency and update Prometheus metr…
RodrigoMNardi May 12, 2026
e46b9a1
Add detailed metrics for scheduled delayed jobs and refactor refresh …
RodrigoMNardi May 13, 2026
3753cf0
Enhance Prometheus metrics to include table names for SQL queries and…
RodrigoMNardi May 14, 2026
4167f0b
Add GitHub Actions sync script for Bamboo CI validation
RodrigoMNardi May 19, 2026
d8f25d0
Refactor PR processing to improve clarity and consistency
RodrigoMNardi May 22, 2026
0af2899
Refactor PR processing to improve clarity and consistency
RodrigoMNardi May 22, 2026
02d30aa
Add authentication for Prometheus metrics endpoint
RodrigoMNardi May 22, 2026
5d65b4c
Add authentication for Prometheus metrics endpoint
RodrigoMNardi May 22, 2026
c03892b
Refactor GAUGE_COUNT_MAP to use a hash for improved readability and a…
RodrigoMNardi May 22, 2026
1cb29e9
Refactor GAUGE_COUNT_MAP to use a hash for improved readability and a…
RodrigoMNardi May 22, 2026
141e32c
Refactor job status queries for improved readability
RodrigoMNardi May 22, 2026
9af6ac6
Refactor job status query for improved readability
RodrigoMNardi May 22, 2026
5b5a570
Merge pull request #121 from RodrigoMNardi/feature/prometheus-exporter
RodrigoMNardi May 22, 2026
9eeef35
Merge branch 'master' into feature/github-action-sync
RodrigoMNardi May 22, 2026
07abcb7
Add authentication metrics method with comprehensive tests
RodrigoMNardi May 22, 2026
7f26124
Update GitHub Actions to use latest action versions for improved func…
RodrigoMNardi May 22, 2026
1718e57
Update GitHub Actions to use latest action versions for improved func…
RodrigoMNardi May 22, 2026
d4eb70b
Update GitHub Actions permissions to allow write access for pull requ…
RodrigoMNardi May 22, 2026
180cb8f
Update GitHub Actions permissions to allow write access for pull requ…
RodrigoMNardi May 22, 2026
cff0242
Update GitHub Actions permissions to allow write access for pull requ…
RodrigoMNardi May 22, 2026
9f6f6c9
Fix URL generation in api_spec.rb to remove spaces from bamboo_ci_pla…
RodrigoMNardi May 22, 2026
9a9e3c4
Merge pull request #122 from RodrigoMNardi/feature/github-action-sync
RodrigoMNardi May 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 8 additions & 14 deletions .github/workflows/ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,23 @@ on:

permissions:
contents: read
pull-requests: write
issues: write

jobs:
rubocop:
name: Rubocop
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
uses: actions/checkout@v6

- uses: ruby/setup-ruby@v1
with:
ruby-version: 3.1.2

- name: rubocop
uses: reviewdog/action-rubocop@v2.6.0
uses: reviewdog/action-rubocop@v2.22.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
rubocop_version: gemfile
Expand All @@ -59,7 +61,7 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0

Expand Down Expand Up @@ -94,21 +96,13 @@ jobs:
run: |
cp config_template.yml config.yml
bundle exec rspec \
-f j -o tmp/rspec_results.json \
-f progress \
-f h -o tmp/rspec_results.html

- name: Save coverage report
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: always()
with:
name: coverage-report
path: /home/runner/work/netdef-ci-github-app/netdef-ci-github-app/coverage
if-no-files-found: warn

- name: RSpec Report
uses: SonicGarden/rspec-report-action@v2
with:
token: ${{ secrets.GITHUB_TOKEN }}
title: 'Unit tests'
json-path: tmp/rspec_results.json
if: always()
if-no-files-found: warn
5 changes: 4 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

source 'https://rubygems.org'

ruby '3.1.2'
ruby '>= 3.1.2'

# Token
gem 'jwt'
Expand Down Expand Up @@ -39,6 +39,9 @@ gem 'rake'
# Delayed Job
gem 'delayed_job_active_record'

# Prometheus metrics
gem 'prometheus-client'

# Code lint
gem 'rubocop', '1.56.1', group: %i[development test]
gem 'rubocop-performance', group: %i[development test]
Expand Down
3 changes: 3 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ GEM
ast (~> 2.4.1)
racc
pg (1.5.7)
prometheus-client (4.2.5)
base64
public_suffix (6.0.1)
puma (6.4.2)
nio4r (~> 2.0)
Expand Down Expand Up @@ -181,6 +183,7 @@ DEPENDENCIES
octokit (~> 9.1)
otr-activerecord (~> 2.3)
pg (~> 1.5, >= 1.5.3)
prometheus-client
puma
rack (= 3.1.7)
rack-test
Expand Down
9 changes: 9 additions & 0 deletions app/github_app.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
require 'json'
require 'sinatra'
require 'octokit'
require 'prometheus/client/formats/text'
require 'netrc'
require 'date'
require 'yaml'
Expand All @@ -38,6 +39,13 @@
attr_writer :sinatra_logger_level
end

get '/metrics' do
authenticate_metrics
PrometheusMetrics.refresh!
content_type 'text/plain; version=0.0.4; charset=utf-8'
Prometheus::Client::Formats::Text.marshal(PrometheusMetrics::REGISTRY)
end

get '/telemetry' do
content_type :json

Expand Down Expand Up @@ -121,6 +129,7 @@

halt 200, 'PONG!'
when 'pull_request'
logger.info 'Creating new action'
build_plan = Github::BuildPlan.new(payload, logger_level: GithubApp.sinatra_logger_level)
resp = build_plan.create

Expand Down Expand Up @@ -168,7 +177,7 @@
logger.debug '======= POST DONE ========'
end

def slack_authentication

Check warning on line 180 in app/github_app.rb

View workflow job for this annotation

GitHub Actions / Rubocop

[rubocop] reported by reviewdog 🐶 Predicate method names should end with `?`. Raw Output: app/github_app.rb:180:9: C: Naming/PredicateMethod: Predicate method names should end with `?`.
netrc = Netrc.read
user, passwd = netrc['slack_bot.netdef.org']

Expand Down
16 changes: 16 additions & 0 deletions bin/pr_bamboo_sync.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
#
# SPDX-License-Identifier: BSD-2-Clause
#
# pr_bamboo_sync.sh
# Part of NetDEF CI System
#
# Validates Bamboo CI execution status for PRs whose GitHub Actions
# check suites were active between 24h and 2h ago.
#
# Copyright (c) 2026 by
# Network Device Education Foundation, Inc. ("NetDEF")

# shellcheck disable=SC2164
cd /home/githubchecks/server
RACK_ENV=production ruby workers/pr_bamboo_sync.rb
8 changes: 8 additions & 0 deletions config.ru
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,20 @@ require 'rackup'
require 'rack/handler/puma'
require 'rack/session/cookie'
require 'securerandom'
require 'prometheus/middleware/collector'
require 'prometheus/middleware/exporter'

require_relative 'app/github_app'
require_relative 'config/delayed_job'
require_relative 'lib/helpers/prometheus_metrics'

PrometheusMetrics.subscribe_query_notifications!

File.write('.session.key', SecureRandom.hex(32))

# Refresh delayed_job + CI domain gauges on every Prometheus scrape
use Prometheus::Middleware::Collector, metrics_prefix: 'http'

pids = []
pids << spawn("RACK_ENV=#{ENV.fetch('RACK_ENV', 'development')} rake jobs:work QUEUES=0,1,2,3")
pids << spawn("RACK_ENV=#{ENV.fetch('RACK_ENV', 'development')} rake jobs:work QUEUES=4,5,6")
Expand Down
1 change: 1 addition & 0 deletions config/puma.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
Thread.new do
loop do
Telemetry.instance.update_stats Puma.stats
File.write('tmp/puma_stats.json', Puma.stats_hash.to_json)
sleep 30
end
end
Expand Down
16 changes: 16 additions & 0 deletions db/migrate/20250812101554_add_pull_requests_plans.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# SPDX-License-Identifier: BSD-2-Clause
#
# 20250812101554_add_pull_requests_plans.rb
# Part of NetDEF CI System
#
# Copyright (c) 2025 by
# Network Device Education Foundation, Inc. ("NetDEF")
#
# frozen_string_literal: true

class AddPullRequestsPlans < ActiveRecord::Migration[6.0]
def change
add_reference :plans, :pull_request, foreign_key: true
add_column :plans, :name, :string, null: false, default: ''
end
end
15 changes: 15 additions & 0 deletions db/migrate/20250822071834_add_check_suite_plan.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-License-Identifier: BSD-2-Clause
#
# 20250822071834_add_check_suite_plan.rb
# Part of NetDEF CI System
#
# Copyright (c) 2025 by
# Network Device Education Foundation, Inc. ("NetDEF")
#
# frozen_string_literal: true

class AddCheckSuitePlan < ActiveRecord::Migration[6.0]
def change
add_reference :check_suites, :plan, foreign_key: true
end
end
9 changes: 8 additions & 1 deletion db/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema[7.2].define(version: 2025_04_16_153222) do
ActiveRecord::Schema[7.2].define(version: 2025_08_22_071834) do
# These are extensions that must be enabled in order to support this database
enable_extension "plpgsql"

Expand Down Expand Up @@ -60,8 +60,10 @@
t.bigint "github_user_id"
t.bigint "stopped_in_stage_id"
t.bigint "cancelled_previous_check_suite_id"
t.bigint "plan_id"
t.index ["cancelled_previous_check_suite_id"], name: "index_check_suites_on_cancelled_previous_check_suite_id"
t.index ["github_user_id"], name: "index_check_suites_on_github_user_id"
t.index ["plan_id"], name: "index_check_suites_on_plan_id"
t.index ["pull_request_id"], name: "index_check_suites_on_pull_request_id"
t.index ["stopped_in_stage_id"], name: "index_check_suites_on_stopped_in_stage_id"
end
Expand Down Expand Up @@ -130,7 +132,10 @@
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.bigint "check_suite_id"
t.bigint "pull_request_id"
t.string "name", default: "", null: false
t.index ["check_suite_id"], name: "index_plans_on_check_suite_id"
t.index ["pull_request_id"], name: "index_plans_on_pull_request_id"
end

create_table "pull_request_subscriptions", force: :cascade do |t|
Expand Down Expand Up @@ -195,12 +200,14 @@
add_foreign_key "audit_retries", "github_users"
add_foreign_key "check_suites", "check_suites", column: "cancelled_previous_check_suite_id"
add_foreign_key "check_suites", "github_users"
add_foreign_key "check_suites", "plans"
add_foreign_key "check_suites", "pull_requests"
add_foreign_key "check_suites", "stages", column: "stopped_in_stage_id"
add_foreign_key "ci_jobs", "check_suites"
add_foreign_key "ci_jobs", "stages"
add_foreign_key "github_users", "organizations"
add_foreign_key "plans", "check_suites"
add_foreign_key "plans", "pull_requests"
add_foreign_key "pull_request_subscriptions", "pull_requests"
add_foreign_key "pull_requests", "github_users"
add_foreign_key "stages", "check_suites"
Expand Down
8 changes: 4 additions & 4 deletions lib/bamboo_ci/api.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ def get_status(id)
get_request(URI("https://127.0.0.1/rest/api/latest/result/#{id}?expand=stages.stage.results,artifacts"))
end

def submit_pr_to_ci(check_suite, ci_variables)
url = "https://127.0.0.1/rest/api/latest/queue/#{check_suite.pull_request.plan}"
def submit_pr_to_ci(check_suite, plan, ci_variables)
url = "https://127.0.0.1/rest/api/latest/queue/#{plan.bamboo_ci_plan_name}"

url += custom_variables(check_suite)

Expand All @@ -40,7 +40,7 @@ def submit_pr_to_ci(check_suite, ci_variables)
logger(Logger::DEBUG, "Submission URL:\n #{url}")

# Fetch Request
post_request(URI(url))
post_request(URI(url.delete(' ')))
end

def custom_variables(check_suite)
Expand All @@ -58,7 +58,7 @@ def add_comment_to_ci(key, comment)
logger(Logger::DEBUG, "Comment Submission URL:\n #{url}")

# Fetch Request
post_request(URI(url), body: "<comment><content>#{comment}</content></comment>")
post_request(URI(url.delete(' ')), body: "<comment><content>#{comment}</content></comment>")
end

def logger(severity, message)
Expand Down
12 changes: 10 additions & 2 deletions lib/bamboo_ci/plan_run.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ class PlanRun
attr_reader :ci_key
attr_accessor :checks_run, :ci_variables

def initialize(check_suite, logger_level: Logger::INFO)
def initialize(check_suite, plan, logger_level: Logger::INFO)
@logger_manager = []
@logger_level = logger_level

Expand All @@ -32,14 +32,18 @@ def initialize(check_suite, logger_level: Logger::INFO)
logger(Logger::INFO, "BambooCi::PlanRun - CheckSuite: #{check_suite.inspect}")

@check_suite = check_suite
@plan = plan
@ci_variables = []
end

def start_plan
@response = submit_pr_to_ci(@check_suite, @ci_variables)
@refs = []
@response = submit_pr_to_ci(@check_suite, @plan, @ci_variables)

case @response&.code.to_i
when 200, 201
@check_suite.update(bamboo_ci_ref: JSON.parse(@response.body)['buildResultKey'])

success(@response)
when 400..500
failed(@response)
Expand All @@ -58,6 +62,10 @@ def bamboo_reference
JSON.parse(@response.body)['buildResultKey']
end

def bamboo_references
@refs
end

private

def success(response)
Expand Down
21 changes: 11 additions & 10 deletions lib/github/build/action.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,17 @@ class Action
# Initializes the Action class with the given parameters.
#
# @param [CheckSuite] check_suite The CheckSuite to handle.
# @param [Github] github The Github instance to use.
# @param [Github::Check] github The Github::Check instance to use.
# @param [Array] jobs The jobs to create for the CheckSuite.
# @param [String] Stage Plan name.
# @param [Integer] logger_level The logging level to use (default: Logger::INFO).
def initialize(check_suite, github, jobs, logger_level: Logger::INFO)
def initialize(check_suite, github, jobs, name, logger_level: Logger::INFO)
@check_suite = check_suite
@github = github
@jobs = jobs
@loggers = []
@stages = StageConfiguration.all
@stages_config = StageConfiguration.all
@name = name

%w[github_app.log github_build_action.log].each do |filename|
@loggers << GithubLogger.instance.create(filename, logger_level)
Expand All @@ -49,11 +51,11 @@ def initialize(check_suite, github, jobs, logger_level: Logger::INFO)
#
# @param [Boolean] rerun Indicates if the jobs should be rerun (default: false).
def create_summary(rerun: false)
logger(Logger::INFO, "SUMMARY #{@stages.inspect}")
logger(Logger::INFO, "SUMMARY #{@stages_config.inspect}")

Github::Build::SkipOldTests.new(@check_suite).skip_old_tests

@stages.each do |stage_config|
@stages_config.each do |stage_config|
create_check_run_stage(stage_config)
end

Expand Down Expand Up @@ -118,7 +120,7 @@ def create_ci_job(job)

return if stage_config.nil?

stage = Stage.find_by(check_suite: @check_suite, name: stage_config.github_check_run_name)
stage = Stage.find_by(check_suite: @check_suite, name: "#{stage_config.github_check_run_name} - #{@name}")

logger(Logger::INFO, "create_jobs - #{job.inspect} -> #{stage.inspect}")

Expand All @@ -130,9 +132,8 @@ def create_ci_job(job)
#
# @param [StageConfiguration] stage_config The stage configuration.
def create_check_run_stage(stage_config)
stage = Stage.find_by(name: stage_config.github_check_run_name, check_suite_id: @check_suite.id)

logger(Logger::INFO, "STAGE #{stage_config.github_check_run_name} #{stage.inspect} - @#{@check_suite.inspect}")
logger(Logger::INFO, "create_check_run_stage - #{stage_config.github_check_run_name} - #{@name}")
stage = Stage.find_by(name: "#{stage_config.github_check_run_name} - #{@name}", check_suite_id: @check_suite.id)

return create_stage(stage_config) if stage.nil?
return unless stage.configuration.can_retry?
Expand All @@ -148,7 +149,7 @@ def create_check_run_stage(stage_config)
# @param [StageConfiguration] stage_config The stage configuration.
# @return [Stage] The created stage.
def create_stage(stage_config)
name = stage_config.github_check_run_name
name = "#{stage_config.github_check_run_name} - #{@name}"

stage =
Stage.create(check_suite: @check_suite,
Expand Down
Loading
Loading