-
Notifications
You must be signed in to change notification settings - Fork 27
1402 cg0546 support multi key param update (Execution error fix) #1447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
… fix MultiIndex handling, and correct violation detection logic
RamilCDISC
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was running a validation using the dataset and rule attached in the Issue by mhungria. In the dataset i changed the SM.MIDSTYPE from DIAGNOSIS to not DIAGNOSIS. When running validation in dev editor this throws an error as following:
{
"SM": [
{
"executionStatus": "skipped",
"dataset": "sm.xpt",
"domain": "SM",
"variables": [],
"message": "rule evaluation error - operation failed",
"errors": [
{
"dataset": "sm.xpt",
"error": "Error occurred during operation execution",
"message": "Failed to execute rule operation. Operation: record_count, Target: None, Domain: SM, Error: single positional indexer is out-of-bounds"
}
]
}
],
"TM": [
{
"executionStatus": "skipped",
"dataset": "tm.xpt",
"domain": "TM",
"variables": [],
"message": "Rule skipped - doesn't apply to domain for rule id=CDISC.SDTMIG.CG0546, dataset=TM",
"errors": []
}
]
}
I believe this change should not make the engine through and exception. The rule compares the SM.MIDSTYPE value with TM.MIDSTYPE. If I am misunderstanding here something please let me know.
…github.com/cdisc-org/cdisc-rules-engine into 1402-CG0546-support-multi-key-param-update
…tch_Datasets filtering
…github.com/cdisc-org/cdisc-rules-engine into 1402-CG0546-support-multi-key-param-update
…ataFrame flattening in _process_grouped_result
Fixed MultiIndex handling to properly detect when all index levels need to be removed. Instead of calling
droplevel()on a MultiIndex with equal levels, the code now extracts the innermost level values and reconstructs a Series with the correct row index mapping.Added checks to handle cases where
groupby().apply()returns a DataFrame instead of a Series, extracting the first column to ensure consistent Series handling.Updated
check_basic_sort_orderto enforce strict ordering by using>=for ascending checks and<=for descending checks. This correctly flags identical values (e.g., "DIAG", "DIAG") as violations, ensuring the operator detects missing sequence numbers as required by rules like CG0546.Fixed index alignment issues that were masking errors. Both
basic_sort_checkanddate_overlap_checkare now reindexed tosorted_df.indexbefore combining, ensuring all boolean Series are properly aligned and preventingfill_value=Truefrom masking actual violations due to index mismatches.