Skip to content

ENA - Add spacecraft position to Lo L1C and L2 datasets#3127

Merged
ahotasu merged 13 commits intoIMAP-Science-Operations-Center:devfrom
ahotasu:Lo-L1c-Add-SC-Pos
May 8, 2026
Merged

ENA - Add spacecraft position to Lo L1C and L2 datasets#3127
ahotasu merged 13 commits intoIMAP-Science-Operations-Center:devfrom
ahotasu:Lo-L1c-Add-SC-Pos

Conversation

@ahotasu
Copy link
Copy Markdown
Collaborator

@ahotasu ahotasu commented May 4, 2026

Change Summary

Overview

Added spacecraft position and position unit vector to ENA utility function that already grabbed spacecraft velocity and the velocity unit vector. Added call to Lo that adds this information to the L1c PSET; in Hi processing, the velocity (and now position) data were added only when needed in the L2 product generation (when the CG corrections are applied). The Lo team specifically asked for the position to be added at the L1c PSET level, however.

File changes

Added just a few lines of code to add spacecraft position to the function previously called add_spacecraft_velocity_to_pset() in ena_maps\corrections.py. Renamed function accordingly and updated all associated test cases across Lo and Hi.

Testing

Validated output with Lo team in sample PSETs provided to them that covered the period from 1 Jan 2026 to 16 April 2026. Also, validated that all test cases for Hi and Lo work properly.

ahotasu added 4 commits April 16, 2026 12:33
velocity data is included in the dependencies for CG correction in
lo_l2. This involved adding a call to
add_spacecraft_position_and_velocity_to_pset to the test_lo_l2 function,
and ensuring that the necessary attributes are set on the pset for the
function to work correctly. The test should now run without crashing,
confirming that the CG correction can be applied with the new
dependencies.
@ahotasu ahotasu added this to the May 2026 milestone May 4, 2026
@ahotasu ahotasu requested a review from tmplummer May 4, 2026 18:48
@ahotasu ahotasu self-assigned this May 4, 2026
Copilot AI review requested due to automatic review settings May 4, 2026 18:48
@ahotasu ahotasu added the enhancement New feature or request label May 4, 2026
@ahotasu ahotasu added this to IMAP May 4, 2026
@ahotasu ahotasu added the Ins: Lo Related to the IMAP-Lo instrument label May 4, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends ENA pointing-set (PSET) enrichment to include spacecraft position (and a position unit vector) alongside the existing spacecraft velocity fields, and shifts IMAP-Lo so this enrichment happens at L1C generation (instead of during Lo L2 processing).

Changes:

  • Renamed and expanded the ENA corrections helper to add spacecraft position + velocity (and their unit vectors) to PSET datasets.
  • Updated IMAP-Lo L1C to add spacecraft state at the L1C PSET stage; removed the corresponding addition from Lo L2 processing.
  • Updated Hi/Lo/ENA-map tests and call sites to use the renamed helper.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
imap_processing/ena_maps/utils/corrections.py Renames velocity-only helper and adds spacecraft position + position direction vector fields.
imap_processing/lo/l1c/lo_l1c.py Calls the new helper during Lo L1C PSET creation.
imap_processing/lo/l2/lo_l2.py Removes the Lo L2 call to add spacecraft state; minor formatting change near type-ignore.
imap_processing/hi/hi_l2.py Updates Hi L2 to call the renamed helper.
imap_processing/tests/ena_maps/test_corrections.py Updates tests to validate new position fields and renamed helper.
imap_processing/tests/hi/test_hi_l2.py Updates mocks/patches for renamed helper.
imap_processing/tests/lo/test_lo_l1c.py Updates patch target for renamed helper.
imap_processing/tests/lo/test_lo_l2.py Updates integration setup and removes outdated velocity-call expectations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/lo/l2/lo_l2.py Outdated
Comment thread imap_processing/tests/lo/test_lo_l2.py Outdated
Comment thread imap_processing/lo/l1c/lo_l1c.py Outdated
ahotasu and others added 2 commits May 4, 2026 12:55
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments. In addition to those, you will need to add entries for the new Lo PSET variable to the CDF metadata yaml file. Hi has an example you can follow here: https://github.com/IMAP-Science-Operations-Center/imap_processing/blob/dev/imap_processing/cdf/config/imap_hi_variable_attrs.yaml#L512

Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/lo/l2/lo_l2.py
ahotasu added 2 commits May 5, 2026 14:13
spacecraft velocity and position, but not the unit
vectors. Added in unit vector calculation to the
L2 processing. Updated all tests to reflect this
change. Updated the L1c variable attributes file
to reflect the new variables added to the PSET.
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through all of these comments. They are all related and hopefully make more sense with direct suggestions.

Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml Outdated
Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml Outdated
Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/ena_maps/utils/corrections.py Outdated
Comment thread imap_processing/lo/l1c/lo_l1c.py Outdated
Comment thread imap_processing/lo/l1c/lo_l1c.py Outdated
Comment thread imap_processing/lo/l2/lo_l2.py Outdated
Comment thread imap_processing/lo/l2/lo_l2.py Outdated
Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml
ahotasu added 4 commits May 5, 2026 15:58
of add_spacecraft_position_and_velocity_to_pset in
corrections.py. The new implementation adds sc_position
and sc_velocity but does not add the direction vectors
that were previously added. The tests have been updated to verify
that sc_position and sc_velocity are added with the
correct values, and the assertions for the direction
vectors have been removed.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

imap_processing/lo/l2/lo_l2.py:397

  • process_single_pset() no longer adds spacecraft velocity/position, but calculate_ram_mask() (and CG correction) require sc_velocity to exist. This is a behavioral/API change: running Lo L2 against older L1C PSETs (or any PSET missing sc_velocity) will now raise a KeyError deep in the correction code. Consider adding an explicit validation (with a clear error) or a backward-compatible fallback that calls add_spacecraft_position_and_velocity_to_pset() when sc_velocity is absent.
    # Step 4: Optionally apply CG correction and calculate ram-mask
    if cg_correct:
        # NOTE: Heliospheric frame energy selection for CG correction
        # The heliospheric (HF) energies passed to the CG correction algorithm
        # could in principle be completely different from the ESA central energies.
        # However, for Lo, the instrument team has chosen to use the same HF
        # energies as the ESA central energies (from the geometric factor files).
        # This decision aligns the energy grid between the spacecraft frame and
        # heliospheric frame representations.

        # Convert energy coordinate from keV to eV for CG correction
        # (energy coordinate was set in normalize_pset_coordinates in keV)
        energy_values_ev: xr.DataArray = pset_processed["energy"] * 1000.0
        pset_processed = apply_compton_getting_correction(
            pset_processed, energy_values_ev
        )
        # Prepare energy_sc for exposure time weighted projection
        pset_processed["energy_sc_exposure_factor"] = (
            pset_processed["energy_sc"] * pset_processed["exposure_factor"]
        )

    # Always calculate ram-mask to identify ram/anti-ram bins
    pset_processed = calculate_ram_mask(pset_processed)

Comment thread imap_processing/lo/l1c/lo_l1c.py Outdated
Comment thread imap_processing/lo/l1c/lo_l1c.py
Comment thread imap_processing/tests/lo/test_lo_l1c.py
Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are almost there. I mostly agree with Copilot. Ping me on slack when you push these changes and I can approve.

Comment on lines +20 to +25
component:
CATDESC: Cartesian components (x,y,z)
FIELDNAM: component
FORMAT: A3
LABLAXIS: component
VAR_TYPE: metadata
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be the vector components label variable?

label_vector_HAE:
  CATDESC: Cartesian components (x,y,z)
  FIELDNAM: Cartesian components
  FORMAT: A5
  VAR_TYPE: metadata

Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml Outdated
Comment thread imap_processing/cdf/config/imap_lo_l1c_variable_attrs.yaml
Comment thread imap_processing/lo/l1c/lo_l1c.py Outdated
Comment thread imap_processing/tests/lo/test_lo_l1c.py
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@ahotasu ahotasu merged commit 1968702 into IMAP-Science-Operations-Center:dev May 8, 2026
14 checks passed
@github-project-automation github-project-automation Bot moved this to Done in IMAP May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Ins: Lo Related to the IMAP-Lo instrument

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants