Behaviour of detect bad channels outside channel detection #4434
Description
After #4391, the bad channel detection seems to work well on the IBL data that @oliche posted on the linked PR. I've tested 1/7 datasets but the results match almost perfectly, there is a off-by-one error for outside the brain (Olivier's outside channel ides are 274-383, the SI implementation gives 275-383) but this was the case before the changes. It is worth investigating this discrepancy but it is not a huge deal as at the very worst, one good channel at the very edge will be marked 'outside'.
However, the version before #4391 (I used commit 81e89fd19cffdbf031e48c54100b6ab9e024e3ca) was giving incorrect results, in this test case at least. When outside_channels_location="top" or outside_channels_location="bottom" the outside channels are not detected (they are labelled "good"). However, when outside_channels_location="both" these outside channels are detected. This goes back to 58293022cba9f4ec646a70c8c7d2f1ff4bf3d66d (Nov 2023) when handling of top/bottom of probe was added to the detect bad channels function.
I think everything is working well now, but my two questions @ecobost @alejoe91 are (1) out of interest, what was the change in #4391 that fixed this and (2) is this caused by something strange in the test data (it should be okay, IBL NP1 data) or is it a more general problem that should be flagged as an issue for old versions? I can look into this myself but am a bit stuck for time and thought you might have some immediate insights. It would also be good to have another pair of eyes on this just to make sure I didn't make a mistake during testing.
Unfortunately the way I have been testing this is a bit convoluted, but you can use the below script to download the IBL dataset of interest then run the bad channel detection at different commits. Note the path to the downloaded data will need to be changed. Also I ran into an error with cbin_ibl.py looking for "probe_type" annotation that does not exist, and had to quickly patch to get this to run (you will see this when you try, a quick patch is simple just setting num_channels_per_adc = 12), but will make another issue/PR on that. Once discussed we should also add a test for this behaviour as I don't think there is coverage on the probe top/bottom/both options.
from one.api import ONE
import spikeinterface.extractors as se
import spikeinterface.preprocessing as spre
from pathlib import Path
""" From Olivier
annotations = {
'1d547041-230a-4af3-ba6a-7287de2bdec3': {'idead': [191], 'inoisy': [], 'ioutside': []}, # DOES NOT WORK OUTSIDE BRAIN DUE TO LOW THRESHOLD # eid:e5c75b62-6871-4135-b3d0-f6464c2d90c0, probe01, ('KS043', '2020-12-07', '001')
'4cb60c5c-d15b-4abd-8cfd-776bc5a81dbe': {'idead': [16, 29, 191, 357], 'inoisy': [], 'ioutside': [358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383]},
'ce0dc660-f19e-46a3-94f9-646bebae6805': {'idead': [29, 36, 39, 40, 191], 'inoisy': [133, 235], 'ioutside': [274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286,
287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299,
300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312,
313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325,
326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338,
339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351,
352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364,
365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
378, 379, 380, 381, 382, 383]}, # eid:8ca740c5-e7fe-430a-aa10-e74e9c3cbbe8, probe01, ('NYU-40', '2021-04-14', '001')
'3b729602-20d5-4be8-a10e-24bde8fc3092': {'idead': [191], 'inoisy': [], 'ioutside': [356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383]}, # eid:fc43390d-457e-463a-9fd4-b94a0a8b48f5, probe00, ('NYU-47', '2021-06-25', '001')
'a9c9df46-85f3-46ad-848d-c6b8da4ae67c': {'idead': [191], 'inoisy': [], 'ioutside': [348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360,
361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373,
374, 375, 376, 377, 378, 379, 380, 381, 382, 383]}, # eid:41431f53-69fd-4e3b-80ce-ea62e03bf9c7, probe01, ('CSH_ZAD_022', '2020-05-21', '001')
'03d2d8d1-a116-4763-8425-4ef7b1c1bd35': {'idead': [131, 191, 235], 'inoisy': [], 'ioutside': []}, # eid:81a78eac-9d36-4f90-a73a-7eb3ad7f770b, probe01, ('CSH_ZAD_026', '2020-08-17', '001')
'fe380793-8035-414e-b000-09bfe5ece92a': {'idead': [131, 191], 'inoisy': [], 'ioutside': []}, # eid:ff48aa1d-ef30-4903-ac34-8c41b738c1b9, probe01, ('CSH_ZAD_025', '2020-08-03', '001')
'19c9caea-2df8-4097-92f8-0a2bad055948': {'idead': [], 'inoisy': [191], 'ioutside': []}, # eid:aa20388b-9ea3-4506-92f1-3c2be84b85db, probe01, ('DY_016', '2020-09-14', '001')
'25a9182c-4795-4768-af47-98975d2d2a8a': {'idead': [], 'inoisy': [191], 'ioutside': []}, # eid:26aa51ff-968c-42e4-85c8-8ff47d19254d, probe01, ('DY_020', '2020-10-03', '001')
# '41a3b948-13f4-4be7-90b9-150705d39005': {'idead': [191], 'inoisy': [], 'ioutside': []}, #DOES NOT PASS, WRONG DEAD CHANNEL # eid:7691eeb3-715b-4571-8fda-6bb57aab8253, probe00, ('NYU-30', '2020-10-19', '001')
}
"""
DOWNLOAD = True
PIDS_TO_DOWNLOAD = [
"ce0dc660-f19e-46a3-94f9-646bebae6805",
]
TO_TRY_LATER = [
"3b729602-20d5-4be8-a10e-24bde8fc3092",
"a9c9df46-85f3-46ad-848d-c6b8da4ae67c",
"03d2d8d1-a116-4763-8425-4ef7b1c1bd35",
"fe380793-8035-414e-b000-09bfe5ece92a",
"19c9caea-2df8-4097-92f8-0a2bad055948",
"25a9182c-4795-4768-af47-98975d2d2a8a",
]
if DOWNLOAD:
one = ONE(
base_url="https://openalyx.internationalbrainlab.org",
password="international",
silent=True,
)
for pid in PIDS_TO_DOWNLOAD:
eid, probe = one.pid2eid(pid)
print(f"\nDownloading {pid} ({probe})...")
one.load_collection(eid, collection=f"raw_ephys_data/{probe}", download_only=True)
print("Done.")
# Ground truth from Oliche's comment in PR #4391
SESSIONS = {
"ce0dc660-f19e-46a3-94f9-646bebae6805": { # YOU WILL NEED TO CHANGE THIS PATH <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
"cbin": Path(r"C:\Users\Joe\Downloads\ONE\openalyx.internationalbrainlab.org\angelakilab\Subjects\NYU-40\2021-04-14\001\raw_ephys_data\probe01"),
"dead": [29, 36, 39, 40, 191],
"noisy": [133, 235],
"outside": list(range(274, 384)),
},
}
for pid, info in SESSIONS.items():
print(f"\n{'='*60}")
print(f"PID: {pid}")
recording = se.read_cbin_ibl(info["cbin"])
print(recording)
# since 58293022cba9f4ec646a70c8c7d2f1ff4bf3d66d (Nov 23, 2023) added top/bottom/both arguments
# at 81e89fd19cffdbf031e48c54100b6ab9e024e3ca outside_channels_location="both" is required for correct results ("top" or "bottom" alone do not work)
# on main default "top" works
# Note that for outside channels there is 1 false negative (GT is 274-383 vs 275-383).
bad_ids, bad_labels = spre.detect_bad_channels(recording, method="coherence+psd", outside_channels_location="both")
all_ids = recording.get_channel_ids()
label_dict = dict(zip(all_ids, bad_labels))
detected = {
"dead": sorted(
int(ch.replace("AP", "")) for ch, l in label_dict.items() if l == "dead"),
"noisy": sorted(
int(ch.replace("AP", "")) for ch, l in label_dict.items() if l == "noise"),
"outside": sorted(
int(ch.replace("AP", "")) for ch, l in label_dict.items() if l == "out"),
}
for label in ("dead", "noisy", "outside"):
gt = set(info[label])
det = set(detected[label])
print(f"\n {label.upper()}")
print(f" GT : {sorted(gt) or '[]'}")
print(f" Detected : {sorted(det) or '[]'}")
print(f" TP={len(gt & det)} FP={len(det - gt)} FN={len(gt - det)}")