Processing for a lo/l1b/goodtimes product by vineetbansal · Pull Request #3117 · IMAP-Science-Operations-Center/imap_processing

vineetbansal · 2026-05-01T12:09:25Z

Change Summary

Changed the goodtimes Lo/l1b product algorithm. Both the algorithm as well as the structure of the output datasets have changed. I've run the algorithm by the instrument team. Since no other products depend on these, this should be okay.

Overview

File changes

l1b_bgrates_and_goodtimes function in imap_processing/lo/l1b/lo_l1b.py changed to adhere to new algorithm from IMAP-Lo team (not yet officially documented). Tests related to this function l1b_bgrates_and_goodtimes have been tweaked.

Testing

Modified tests to test some edge cases for the new algorithm. Not all edge cases have been tested, but I'll add more in a subsequent commit.

tmplummer

Just one major concern with this implementation. I left some other relevant comments too.

tmplummer · 2026-05-04T18:07:13Z

+        start_et_hk = (
+            hk_epoch_ets[0] + timedelta(hours=PIVOT_HK_HOUR_RANGE[0]).total_seconds()
+        )
+        end_et_hk = (
+            hk_epoch_ets[0] + timedelta(hours=PIVOT_HK_HOUR_RANGE[1]).total_seconds()
+        )


Fragile code. Nominal pointings are close to 24 hours, but not all pointings are that long. For example around the TCM on April 7, the repoint table has the following:

513165615,0,513165816,0,2026-04-06 10:00:00.748931,2026-04-06 10:03:21.748756,208 513252017,0,513252218,0,2026-04-07 10:00:02.673645,2026-04-07 10:03:23.673469,209 513287744,0,513287946,0,2026-04-07 19:55:29.642514,2026-04-07 19:58:51.642338,210 513288856,0,513289642,0,2026-04-07 20:14:01.641545,2026-04-07 20:27:07.640860,211 513338414,0,513338796,0,2026-04-08 09:59:59.598459,2026-04-08 10:06:21.598127,212

Pointing 209 is ~9:52 in duration, pointing 210 is ~0:16 in duration, etc.

I'll bring this up with @nschwadron tomorrow..

Good point. The current implementation assumes a nominal long pointing and uses a fixed 3–15 hour window from the beginning of the HK file to estimate the pivot angle. That approach may not be robust for shortened pointings, such as those around TCM activities. Defining the HK selection window dynamically based on the actual pointing duration sounds like a better approach. I would recommend applying only a small edge trim to avoid transition periods.

@tmplummer - my understanding from @frahmanif is that instead of a +3hr and +15hr offset from the start of the hk file, we'll do a +p hr offset from the start and -q hr offset from the end for the window (both constants in constants.py. Around an hour to start. I'll implement this change tomorrow, unless you see a problem with this approach

Ok, that would be an improvement, but will break if the pointing is less than 2 hours long.

Pointings shorter than ~2 hours are most likely associated with tests or special operational modes and probably would not need to be included in the nominal goodtimes list. I will double-check this with Nathan, but my understanding is that we should likely define a minimum pointing duration required for a pointing to qualify for the goodtimes analysis.

tmplummer · 2026-05-04T20:47:54Z

Does this work have an associated issue? It would be good to keep with best practices of capturing all work in a issue and linking PRs using the "Closes: #nnnn"

vineetbansal · 2026-05-04T21:17:54Z

Opened issue #3135 for this PR specifically.

tmplummer · 2026-05-07T22:31:03Z

+}
+
+# One histogram accumulation cycle duration [s]
+HISTOGRAM_CYCLE_EPOCHS: int = 420


Does it matter that this is just an estimate? I am assuming this is 7 ESA steps * 4 spins/step * 15 seconds/spin. But the true spin duration is not 15 seconds. This is the type of hard coded value that adds lots of technical debt.

It looks like N_ESA_LEVELS is defined below, so perhaps this should actually be:
HISTOGRAM_CYCLE_EPOCHS = N_ESA_LEVELS * N_SPINS_PER_ESA_LEVEL * 15?

@vineetbansal If this is going to stay as is, please add a TODO and link it to a github issue.

Good catch! I've introduced a N_SPINS_PER_ESA_LEVEL and am deriving HISTOGRAM_CYCLE_EPOCHS from it now. 2 other existing places that had a hardcoded 4 now refer to N_SPINS_PER_ESA_LEVEL instead.

For the spin duration, I've added a NOMINAL_SPIN_PERIOD_SEC: float = 15.0, with an explanation that it is not exact. I couldn't find another place in the code where I could grab this "constant". Hope this works for now.

tmplummer · 2026-05-07T22:32:25Z

+ANTI_RAM_HISTOGRAM_BINS: tuple[slice, ...] = (slice(20, 50),)
+
+# Nominal background rates [counts/s] for each species
+BG_RATES = {"H": 0.0014925, "O": 0.000136635}


I envision this will also want to be changed in the future.

tmplummer · 2026-05-07T22:36:37Z

+        start_et_hk = (
+            hk_epoch_ets[0] + timedelta(hours=PIVOT_HK_HOUR_RANGE[0]).total_seconds()
+        )
+        end_et_hk = (
+            hk_epoch_ets[0] + timedelta(hours=PIVOT_HK_HOUR_RANGE[1]).total_seconds()
+        )


Ok, that would be an improvement, but will break if the pointing is less than 2 hours long.

…stants

tmplummer

Looks good once the checks pass.

vineetbansal · 2026-05-08T22:09:16Z

@tmplummer - for the time window to get a median pivot angle, @nschwadron mentioned that he'd think about it (his idea was slightly different from what @frahmanif suggested above). I've opened an issue #3169 on this so we can revisit it (shortly in the next couple of days I think).

Re-implementation of the lo/l1b/goodtimes product

378b5bd

vineetbansal force-pushed the vb/lo_pipeline_merge branch from af822aa to 378b5bd Compare May 4, 2026 17:59

vineetbansal added 2 commits May 4, 2026 14:22

pre-commit fixes

bcb9cc6

code tolerant of hk/de dependencies missing

545f15a

vineetbansal changed the title ~~WIP: Processing for a lo/l1b/bettertimes product~~ Processing for a lo/l1b/bettertimes product May 4, 2026

ahotasu requested review from ahotasu and tmplummer May 4, 2026 18:59

vineetbansal changed the title ~~Processing for a lo/l1b/bettertimes product~~ Processing for a lo/l1b/goodtimes product May 4, 2026

tmplummer reviewed May 4, 2026

View reviewed changes

vineetbansal marked this pull request as ready for review May 4, 2026 19:55

tmplummer added this to IMAP May 4, 2026

added some concerns with PR 3117

591a7e6

tmplummer reviewed May 7, 2026

View reviewed changes

ancillary file for per-day overrides; other minor refactoring wrt con…

fdcd5f8

…stants

tmplummer approved these changes May 8, 2026

View reviewed changes

vineetbansal added 3 commits May 8, 2026 17:50

merged dev (changes pending)

a1a4cae

changes after dev merge

6e3ef58

rearranged vars in alphabetical order

b382172

vineetbansal mentioned this pull request May 8, 2026

IMAP-Lo - Fragile code #3169

Open

tmplummer merged commit 15f176b into IMAP-Science-Operations-Center:dev May 8, 2026
14 checks passed

github-project-automation Bot moved this to Done in IMAP May 8, 2026

Conversation

vineetbansal commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Overview

File changes

Testing

Uh oh!

tmplummer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vineetbansal May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tmplummer commented May 4, 2026

Uh oh!

vineetbansal commented May 4, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tmplummer left a comment

Choose a reason for hiding this comment

Uh oh!

vineetbansal commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vineetbansal commented May 1, 2026 •

edited

Loading

vineetbansal May 7, 2026 •

edited

Loading

vineetbansal commented May 8, 2026 •

edited

Loading