Description of the issue
Discussed in email chain 4/30.
There are cases where the L1a priority counts are missing epoch values which are present in the L2 direct events, resulting in a failure when calculating the normalization.
Steps to reproduce the issue
imap_cli --instrument codice --data-level l3a --descriptor lo-direct-events --start-date 20251213 --version v001 --dependency imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.json
Expected vs Actual behavior
Building imap-l3-processing @ file:///
Built imap-l3-processing @ file:///
Uninstalled 1 package in 0.64ms
Installed 1 package in 0.79ms
2026-05-14 17:52:58,742 - imap_data_access.io - INFO - Downloading file imap/dependency/codice/l3a/2025/12/imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.json from https://api.imap-mission.com/api-key/download/imap/dependency/codice/l3a/2025/12/imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.json to /data/imap/dependency/codice/l3a/2025/12/imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.json
2026-05-14 17:52:59,210 - imap_data_access.io - INFO - File /data/imap/dependency/codice/l3a/2025/12/imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.json downloaded successfully
2026-05-14 17:52:59,212 - imap_data_access.io - INFO - Downloading file imap/codice/l1a/2025/12/imap_codice_l1a_lo-sw-priority_20251213_v005.cdf from https://api.imap-mission.com/api-key/download/imap/codice/l1a/2025/12/imap_codice_l1a_lo-sw-priority_20251213_v005.cdf to /data/imap/codice/l1a/2025/12/imap_codice_l1a_lo-sw-priority_20251213_v005.cdf
2026-05-14 17:52:59,722 - imap_data_access.io - INFO - File /data/imap/codice/l1a/2025/12/imap_codice_l1a_lo-sw-priority_20251213_v005.cdf downloaded successfully
2026-05-14 17:52:59,723 - imap_data_access.io - INFO - Downloading file imap/codice/l1a/2025/12/imap_codice_l1a_lo-nsw-priority_20251213_v005.cdf from https://api.imap-mission.com/api-key/download/imap/codice/l1a/2025/12/imap_codice_l1a_lo-nsw-priority_20251213_v005.cdf to /data/imap/codice/l1a/2025/12/imap_codice_l1a_lo-nsw-priority_20251213_v005.cdf
2026-05-14 17:53:00,219 - imap_data_access.io - INFO - File /data/imap/codice/l1a/2025/12/imap_codice_l1a_lo-nsw-priority_20251213_v005.cdf downloaded successfully
2026-05-14 17:53:00,220 - imap_data_access.io - INFO - Downloading file imap/codice/l2/2025/12/imap_codice_l2_lo-direct-events_20251213_v002.cdf from https://api.imap-mission.com/api-key/download/imap/codice/l2/2025/12/imap_codice_l2_lo-direct-events_20251213_v002.cdf to /data/imap/codice/l2/2025/12/imap_codice_l2_lo-direct-events_20251213_v002.cdf
2026-05-14 17:53:00,883 - imap_data_access.io - INFO - File /data/imap/codice/l2/2025/12/imap_codice_l2_lo-direct-events_20251213_v002.cdf downloaded successfully
2026-05-14 17:53:00,884 - imap_data_access.io - INFO - Downloading file imap/ancillary/codice/imap_codice_mass-coefficient-lookup_20241110_v003.csv from https://api.imap-mission.com/api-key/download/imap/ancillary/codice/imap_codice_mass-coefficient-lookup_20241110_v003.csv to /data/imap/ancillary/codice/imap_codice_mass-coefficient-lookup_20241110_v003.csv
2026-05-14 17:53:01,305 - imap_data_access.io - INFO - File /data/imap/ancillary/codice/imap_codice_mass-coefficient-lookup_20241110_v003.csv downloaded successfully
2026-05-14 17:53:01,306 - imap_data_access.io - INFO - Downloading file imap/ancillary/codice/imap_codice_lo-energy-per-charge_20241110_v002.csv from https://api.imap-mission.com/api-key/download/imap/ancillary/codice/imap_codice_lo-energy-per-charge_20241110_v002.csv to /data/imap/ancillary/codice/imap_codice_lo-energy-per-charge_20241110_v002.csv
2026-05-14 17:53:01,731 - imap_data_access.io - INFO - File /data/imap/ancillary/codice/imap_codice_lo-energy-per-charge_20241110_v002.csv downloaded successfully
/imap_l3_processing/codice/l3/lo/models.py:132: RuntimeWarning: invalid value encountered in cast
...].astype(int),
/imap_l3_processing/codice/l3/lo/science/codice_lo_calculations.py:42: RuntimeWarning: divide by zero encountered in log
energy = np.log(apd_energy)
/imap_l3_processing/codice/l3/lo/science/codice_lo_calculations.py:42: RuntimeWarning: invalid value encountered in log
energy = np.log(apd_energy)
/imap_l3_processing/codice/l3/lo/science/codice_lo_calculations.py:43: RuntimeWarning: invalid value encountered in log
tof = np.log(tof)
2026-05-14 17:53:08,001 - __main__ - ERROR - Unhandled Exception:
Traceback (most recent call last):
File "//imap_l3_data_processor.py", line 180, in <module>
raise e
File "//imap_l3_data_processor.py", line 177, in <module>
imap_l3_processor()
File "//imap_l3_data_processor.py", line 146, in imap_l3_processor
paths = processor.process()
^^^^^^^^^^^^^^^^^^^
File "/imap_l3_processing/codice/l3/lo/codice_lo_processor.py", line 40, in process
data_product = self.process_l3a_direct_event_data_product(dependencies)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/imap_l3_processing/codice/l3/lo/codice_lo_processor.py", line 201, in process_l3a_direct_event_data_product
normalization = calculate_normalization_factor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/imap_l3_processing/codice/l3/lo/science/codice_lo_calculations.py", line 94, in calculate_normalization_factor
np.divide(numerator, denominator, out=division_result, where=denominator!=0)
ValueError: operands could not be broadcast together with shapes (356,7,128,24) (357,7,128,24) (356,7,128,24) (357,7,128,24) Code Snippet (If applicable)
Additional notes, affected areas, and suggested fixes
A previous example I looked at was 2026-4-23, where timestamp 5:10:32 is only in the direct events, and not in the priority.
L3 code needs to match direct events to priority rates by timestamp, not just by index. If the priority is missing for a given direct event timestamp, make the normalization fill and the normalization per event 0. This indicates that we're missing something and can't calculate the normalization, but the norm per event makes the event "fall out" when we're computing VDFs later.
If there's a priority timestamp but no direct event timestamp, no worries...we aren't making a direct event record for that time anyhow.
Description of the issue
Discussed in email chain 4/30.
There are cases where the L1a priority counts are missing epoch values which are present in the L2 direct events, resulting in a failure when calculating the normalization.
Steps to reproduce the issue
imap_cli --instrument codice --data-level l3a --descriptor lo-direct-events --start-date 20251213 --version v001 --dependency imap_codice_l3a_lo-direct-events-a8091f36_20251213_v001.jsonExpected vs Actual behavior
Code Snippet (If applicable)
CodeAdditional notes, affected areas, and suggested fixes
A previous example I looked at was 2026-4-23, where timestamp 5:10:32 is only in the direct events, and not in the priority.
L3 code needs to match direct events to priority rates by timestamp, not just by index. If the priority is missing for a given direct event timestamp, make the normalization fill and the normalization per event 0. This indicates that we're missing something and can't calculate the normalization, but the norm per event makes the event "fall out" when we're computing VDFs later.
If there's a priority timestamp but no direct event timestamp, no worries...we aren't making a direct event record for that time anyhow.