Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _extras/Tokens.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Tokens
---

{% include Tokens.toc.md %}

## SL7 Tokens <a name="SL7_token"></a>

{% include sl7_token.md %}
Expand Down
6 changes: 4 additions & 2 deletions _includes/03-data-management.toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,10 @@
- [Getting file locations using Rucio](#getting-file-locations-using-rucio)
- [What is Rucio?](#what-is-rucio)
- [You will need to authenticate to read files](#you-will-need-to-authenticate-to-read-files)
- [Accessing `rucio` and `justin` require a bit more](#accessing-`rucio`-and-`justin`-require-a-bit-more)
- [getting a token for xroot access in AL9](#getting-a-token-for-xroot-access-in-al9)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
- [finding a file](#finding-a-file)
- [More finding files by characteristics using metacat](#more-finding-files-by-characteristics-using-metacat)
- [Accessing data for use in your analysis](#accessing-data-for-use-in-your-analysis)
Expand Down
6 changes: 4 additions & 2 deletions _includes/Tokens.toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

**Table of Contents for Tokens**
- [SL7 Tokens <a name="SL7_token"></a>](#SL7_token)
- [Accessing `rucio` and `justin` require a bit more](#accessing-`rucio`-and-`justin`-require-a-bit-more)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
- [AL9 Tokens <a name="AL9_token"></a>](#AL9_token)
- [getting a token for xroot access in AL9](#getting-a-token-for-xroot-access-in-al9)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
3 changes: 2 additions & 1 deletion _includes/al9_setup.toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@
- [a tip](#a-tip)
- [Set up software](#set-up-software)
- [Get a token](#get-a-token)
- [getting a token for xroot access in AL9](#getting-a-token-for-xroot-access-in-al9)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
47 changes: 25 additions & 22 deletions _includes/al9_setup_2025a.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,31 +2,34 @@
# find a spack environment and set it up
# setup spack (pre spack 1.0 version)

source /cvmfs/larsoft.opensciencegrid.org/spack-v0.22.0-fermi/setup-env.sh

# get the packages you need to run this - this will become simple in future
echo "ROOT"
spack load root@6.28.12%gcc@12.2.0 arch=linux-almalinux9-x86_64_v3

echo "CMAKE"
spack load cmake@3.27.9%gcc@11.4.1 arch=linux-almalinux9-x86_64_v3

source /cvmfs/dune.opensciencegrid.org/spack/setup-env.sh
echo "Activate dune-workflow"
spack env activate dune-workflow
echo "load GCC and CMAKE so don't use system"
echo "GCC"
spack load gcc@12.2.0

echo "Rucio and metacat"
spack load r-m-dd-config experiment=dune lab=fnal.gov
export RUCIO_ACCOUNT=justinreadonly

echo "IFDHC"
spack load ifdhc@2.8.0%gcc@12.2.0 arch=linux-almalinux9-x86_64_v3
spack load ifdhc-config@2.6.20%gcc@11.4.1 arch=linux-almalinux9-x86_64_v3

spack load gcc@12.5.0 arch=linux-almalinux9-x86_64_v2

echo "PY-PIP"
spack load py-pip@23.1.2%gcc@11.4.1 arch=linux-almalinux9-x86_64_v3

echo "no justIN yet"
#spack load justin
~~~
{: .language-bash}
{: .language-bash}

You can check your environment by doing this

~~~
# test-paths.sh
echo "which root"
which root
root --version
echo "which gcc"
which gcc
gcc --version
echo "which python"
which python
python --version
echo "which cmake"
which cmake
cmake --version
~~~
{: .language-bash}
3 changes: 2 additions & 1 deletion _includes/al9_speedrun.toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@

**Table of Contents for al9_speedrun**
- [2025 Speedrun of AL9 setup and test](#2025-speedrun-of-al9-setup-and-test)
- [getting a token for xroot access in AL9](#getting-a-token-for-xroot-access-in-al9)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
60 changes: 41 additions & 19 deletions _includes/al9_token.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,8 @@
### Interactive file access

> ## Note: The justin get-token method for authentication does not currently work on AL9
> The justin get-token command is not distributed on AL9/Spack currently.
> Please use [SL7]({{ site.baseurl }}/sl7_setup) if you need to use rucio.
>
> normal tokens (below) for `xroot` access do work
>
{: .caution}

### getting a token for xroot access in AL9
Make certain you have [al9 set up]({{ site.baseurl }}/al9_setup)

<!-- Get rucio:

~~~
spack load r-m-dd-config experiment=dune lab=fnal.gov # r stands for rucio
export RUCIO_ACCOUNT=justinreadonly
~~~
{: .language-bash}
-->

Then use htgettoken to get a token so you can read the files you find.
Then use `htgettoken` to get a token so you can read the files you find.

~~~
htgettoken -i dune --vaultserver htvaultprod.fnal.gov -r interactive
Expand Down Expand Up @@ -49,3 +32,42 @@ YgZTGDqHQg6NOO77NsCY5J88uyIkkoZ1tRb6iTXK0j5RsX0AjA
You should be able to read files at remote sites now.
You may need to repeat the `htgettoken` as the interactive tokens are pretty short-lived. Batch jobs do their own tokens.

### Accessing rucio and justIn resources requires a bit more

You should already be set up above. Now you can use `justIn` to get you a token.

1. First tell `justIn` knows about you

~~~
justin time
~~~
{: ..language-bash}

The first time you do this you will get asked (after the `justin time` command)

~~~
To authorize this computer to run the justin command, visit this page with your
usual web browser and follow the instructions within the next 10 minutes:
https://dunejustin.fnal.gov/authorize/XXXXX

Check that the Session ID displayed on that page is BfhVBmQ

Once you've followed the instructions on that web page, please run the command
you tried again. You won't need to authorize this computer again for 7 days.
~~~
{: ..output}

Once again go to the website that appears and authenticate.

2. After the first authentication to justIn you need to do a second justin call

~~~
justin get-token
~~~
{: ..language-bash}

You will need to do this sequence weekly as your justin access expires.

> ## Note:
> Despite the name of this command it gets you both a token and a special X.509 proxy and it is the latter you are actually using to talk to rucio in these SL7 examples
{: .callout}
1 change: 1 addition & 0 deletions _includes/all_toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ al9_speedrun.toc.md %}{% include
figures.toc.md %}{% include
guide.toc.md %}{% include
helpers.toc.md %}{% include
howToBuild.toc.md %}{% include
pnfs2xrootd.toc.md %}{% include
putty.toc.md %}{% include
setup.toc.md %}{% include
Expand Down
3 changes: 2 additions & 1 deletion _includes/sl7_setup.toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@
- [launch the Apptainer](#launch-the-apptainer)
- [then do the following to set up DUNE code](#then-do-the-following-to-set-up-dune-code)
- [then do the following to get authentication to remote data and batch systems](#then-do-the-following-to-get-authentication-to-remote-data-and-batch-systems)
- [Accessing `rucio` and `justin` require a bit more](#accessing-`rucio`-and-`justin`-require-a-bit-more)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
3 changes: 2 additions & 1 deletion _includes/sl7_speedrun.toc.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@


**Table of Contents for sl7_speedrun**
- [Accessing `rucio` and `justin` require a bit more](#accessing-`rucio`-and-`justin`-require-a-bit-more)
- [Interactive file access](#interactive-file-access)
- [Accessing rucio and justIn resources requires a bit more](#accessing-rucio-and-justin-resources-requires-a-bit-more)
- [check root](#check-root)
- [check rucio](#check-rucio)
3 changes: 2 additions & 1 deletion _includes/sl7_token.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
### Interactive file access

To get a token that allows you to access files interactively in SL7

Expand Down Expand Up @@ -36,7 +37,7 @@ Storing bearer token in /run/user/XXXX/bt_XXXX



## Accessing `rucio` and `justin` require a bit more
### Accessing rucio and justIn resources requires a bit more

in SL7 - put this in a file called `dune_data_sl7.sh` so you can use it again.

Expand Down
1 change: 1 addition & 0 deletions addtoc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ python code/tocgen.py _extras/putty.md
python code/tocgen.py _extras/howToBuild.md
python code/tocgen.py _extras/sites.md
python code/tocgen.py _extras/sl7_setup.md
python code/tocgen.py _extras/sl7_token.md
python code/tocgen.py _extras/sl7_speedrun.md
python code/tocgen.py _extras/Tokens.md
python code/tocgen.py _extras/TutorialsMasterList.md
Expand Down