Skip to content

Fixes for 2tag #1983

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sawenzel merged 1 commit into from
Apr 28, 2025
Merged

Fixes for 2tag #1983

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion MC/bin/o2dpg_workflow_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,14 @@ def adjust_RECO_environment(workflowspec, package = ""):
if len(package) == 0:
return

# we try to extract the stage from the path (can be given via '@' separation)
# example O2sim::daily-xxx@DIGI ---> apply this environment from the DIGI phase
# example O2sim::daily-xxx@RECO ---> apply this environment from the RECO phase
# example O2sim::daily-xxx ---> apply this environment from the RECO phase == default case
from_stage = "RECO"
if package.count('@') == 1:
package, from_stage = package.split('@')

# We essentially need to go through the graph and apply the mapping
# so take the workflow spec and see if the task itself or any child
# is labeled RECO ---> typical graph traversal with caching
Expand Down Expand Up @@ -305,7 +313,7 @@ def matches_or_inherits_label(taskid, label, cache):

# fills the matches_label dictionary
for taskid in range(len(workflowspec['stages'])):
if (matches_or_inherits_label(taskid, "RECO", matches_label)):
if (matches_or_inherits_label(taskid, from_stage, matches_label)):
# now we do the final adjust (as annotation) in the workflow itself
if workflowspec['stages'][taskid].get("alternative_alienv_package") == None:
workflowspec['stages'][taskid]["alternative_alienv_package"] = package
Expand Down
29 changes: 15 additions & 14 deletions MC/run/ANCHOR/anchorMC.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ fi
#<----- START OF part that should run under a clean alternative software environment if this was given ------
if [ "${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG}" ]; then
if [ "${LOADEDMODULES}" ]; then
export > env_before_stashing.env
echo "Stashing initial modules"
module save initial_modules.list # we stash the current modules environment
module list --no-pager
Expand Down Expand Up @@ -253,19 +254,6 @@ fi
ALIEN_JDL_LPMPRODUCTIONTAG=$ALIEN_JDL_LPMPRODUCTIONTAG_KEEP
echo_info "Setting back ALIEN_JDL_LPMPRODUCTIONTAG to $ALIEN_JDL_LPMPRODUCTIONTAG"

# now create the local MC config file --> config-json.json
# we create the new config output with blacklist functionality
ASYNC_CONFIG_BLACKLIST=${ASYNC_CONFIG_BLACKLIST:-${O2DPG_ROOT}/MC/run/ANCHOR/anchor-dpl-options-blacklist.json}
${O2DPG_ROOT}/MC/bin/o2dpg_dpl_config_tools.py workflowconfig.log ${ASYNC_CONFIG_BLACKLIST} config-json.json
ASYNC_WF_RC=${?}

# check if config reasonably created
if [[ "${ASYNC_WF_RC}" != "0" || `grep "ConfigParams" config-json.json 2> /dev/null | wc -l` == "0" ]]; then
echo_error "Problem in anchor config creation. Exiting."
exit 1
fi


# get rid of the temporary software environment
if [ "${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG}" ]; then
module purge --no-pager
Expand All @@ -280,6 +268,18 @@ if [ "${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG}" ]; then
fi
#<----- END OF part that should run under a clean alternative software environment if this was given ------

# now create the local MC config file --> config-json.json
# we create the new config output with blacklist functionality
ASYNC_CONFIG_BLACKLIST=${ASYNC_CONFIG_BLACKLIST:-${O2DPG_ROOT}/MC/run/ANCHOR/anchor-dpl-options-blacklist.json}
${O2DPG_ROOT}/MC/bin/o2dpg_dpl_config_tools.py workflowconfig.log ${ASYNC_CONFIG_BLACKLIST} config-json.json
ASYNC_WF_RC=${?}

# check if config reasonably created
if [[ "${ASYNC_WF_RC}" != "0" || `grep "ConfigParams" config-json.json 2> /dev/null | wc -l` == "0" ]]; then
echo_error "Problem in anchor config creation. Exiting."
exit 1
fi

# -- CREATE THE MC JOB DESCRIPTION ANCHORED TO RUN --

MODULES="--skipModules ZDC"
Expand All @@ -300,7 +300,8 @@ remainingargs="${remainingargs} -productionTag ${ALIEN_JDL_LPMPRODUCTIONTAG:-ali
remainingargs="${ALIEN_JDL_ANCHOR_SIM_OPTIONS} ${remainingargs} --anchor-config config-json.json"
# apply software tagging choice
# remainingargs="${remainingargs} ${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG:+--alternative-reco-software ${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG}}"
remainingargs="${remainingargs} ${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG:+--alternative-reco-software ${PWD}/env_async.env}"
ALIEN_JDL_O2DPG_ASYNC_RECO_FROMSTAGE=${ALIEN_JDL_O2DPG_ASYNC_RECO_FROMSTAGE:-RECO}
remainingargs="${remainingargs} ${ALIEN_JDL_O2DPG_ASYNC_RECO_TAG:+--alternative-reco-software ${PWD}/env_async.env@${ALIEN_JDL_O2DPG_ASYNC_RECO_FROMSTAGE}}"
# potentially add CCDB timemachine timestamp
remainingargs="${remainingargs} ${ALIEN_JDL_CCDB_CONDITION_NOT_AFTER:+--condition-not-after ${ALIEN_JDL_CCDB_CONDITION_NOT_AFTER}}"

Expand Down