Skip to content

Conversation

@DiogoSantoss
Copy link

@DiogoSantoss DiogoSantoss commented Dec 11, 2025

Related to ObolNetwork#387

Warning

Breaking Change

This PR changes how metrics are sent to Obol. After this update, you will no longer have to keep stashing and unstashing changes on prometheus/prometheus.yml every update. It will now be generated at runtime by ./prometheus/run.sh, and the required variables will be injected into the file from your .env. The steps you need to take to handle this change are described below.

  1. If you currently have a locally modified prometheus/prometheus.yml, make a backup copy of it outside of the repository before upgrading (just in case). Ensure you record the value of PROM_REMOTE_WRITE_TOKEN from the authorization.credentials section of that file.

  2. This token must now be provided via your .env file instead of being defined in prometheus.yml. The variable PROM_REMOTE_WRITE_TOKEN is present (commented out) in all .env.sample.* files. To configure it, copy the variable to your own .env, uncomment it and set your token, for example:

PROM_REMOTE_WRITE_TOKEN=obolH7d...
  1. We are now adding a dedicated way to add discord IDs to your deployment, allowing you to be @'d directly by our Obol Agent if there is an issue with your node. Add (uncomment) ALERT_DISCORD_IDS= in your .env, and specify one or more (with comma separation) IDs. To get the ID that corresponds to your discord account; Enable developer mode on discord with User Settings > Advanced. Then right click on a user's profile picture or name and select Copy ID to get a unique 18-digit number that represents their account. If you previously changed your CHARON_NICKNAME to your discord ID, you can set that back to a human friendly name for your node.

  2. If you have any other custom modifications to your original prometheus.yml, compare it against prometheus.yml.example, which is now used as the base template for generating the final configuration. Any additional fields or customizations should be added to prometheus.yml.example.

  3. With your environment variables set, and any other modifications ported to the prometheus.yml.example file; run docker compose up -d as normal. Confirm that your metrics are being received on the Obol Grafana. Once you have verified that the new Prometheus setup is working as expected, you can safely delete your backup of the old prometheus.yml.

@DiogoSantoss DiogoSantoss self-assigned this Dec 11, 2025
Copy link

@OisinKyne OisinKyne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs .env.sample.network commented out sample env var

# To get Alerted with Obol Agent monitoring on Discord, specify your Discord ID(s) below.
# Enable developer mode on discord with User Settings > Advanced. 
# Then right click on a user's profile picture or name and select Copy ID to get a unique 18-digit number that represents their account. 
# Specify multiple discord IDs using comma separation. (e.g. `DISCORD_IDS=123456789098765432,098765432123456789`)
#DISCORD_IDS=""

Copy link

@OisinKyne OisinKyne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small changes

Hope this isn't too forward. Keen to get everything to rc while i'm on a roll.

Signed-off-by: Oisín Kyne <4981644+OisinKyne@users.noreply.github.com>
Signed-off-by: Oisín Kyne <4981644+OisinKyne@users.noreply.github.com>
@OisinKyne OisinKyne merged commit 87f70e6 into main Jan 7, 2026
3 checks passed
@OisinKyne OisinKyne deleted the diogo/add-alert-discord-ids-env-and-label branch January 7, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants