Processing on Linux - high number of "DROPPED" detections

[5559175]

Following on from my other issue, I ran this tool against a different 4hr video capture where the capture brightness had been increased and this time it captured a number of events correctly.

I first transcoded the 4K stream down to 720p to speed up processing time. The file I used with MetDetPy is uploaded here

The MetDetPy script took ~1hr to complete on my Debian 13 system (Intel Xeon E3-1220L (2C/4T), 16GB RAM, Nvidia Quadro P600).

MetDetPy on this system is running within a Python 3.10 venv created with "uv" and "pip" so as not to taint any system libs - these are the pip package versions for reference:

Using Python 3.10.20 environment at: /root/MetDetPy/.venv
Package                  Version
------------------------ ---------
av                       17.0.0
coloredlogs              15.0.1
dacite                   1.9.2
dill                     0.4.1
flatbuffers              25.12.19
humanfriendly            10.0
mpmath                   1.3.0
multiprocess             0.70.19
numpy                    1.26.4
nvidia-cublas-cu11       11.11.3.6
nvidia-cuda-nvrtc-cu11   11.8.89
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cudnn-cu11        8.9.6.50
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.3.0.86
onnxruntime-gpu          1.16.3
opencv-python            4.13.0.92
packaging                26.0
protobuf                 7.34.1
pyexiv2                  2.15.5
rawpy                    0.26.1
sympy                    1.14.0
tqdm                     4.67.3

I call MetDetPy from within a bash script - worth pointing out that I needed to strip the executable stack from some of the ONNX runtimes (apparently even when running things in a venv Debian 13's glibc will block execution otherwise). The lib_compat dir contains symlinks of all the nvidia CUDA/cuDNN related libs in one place. I am not using anything else CUDA-related on the underlying system except the closed-source nvidia driver (v 580.126.20-1) so all dependencies to get this to work are in the venv.

# Clear the execstack flag on the 3 critical ONNX GPU files
patchelf --clear-execstack .venv/lib/python3.10/site-packages/onnxruntime/capi/libonnxruntime_providers_cuda.so
patchelf --clear-execstack .venv/lib/python3.10/site-packages/onnxruntime/capi/libonnxruntime_providers_shared.so
patchelf --clear-execstack .venv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.cpython-310-x86_64-linux-gnu.so

Here is the bash script calling MetDetPy:

#!/bin/bash
export ORT_TENSORRT_FP16_ENABLE=0
export ORT_CUDA_FLAGS="device_id=0"
export GLIBC_TUNABLES=glibc.rtld.execstack=2
export LD_LIBRARY_PATH="/root/MetDetPy/lib_compat:$LD_LIBRARY_PATH"

/root/MetDetPy/.venv/bin/python /root/MetDetPy/MetDetPy.py /export/media/skycam/results/20260325/capture_0051_720p.mp4 --provider cuda --s
ave-path /root/MetDetPy/detections/detections.json

...and the console output (sorry I forgot to turn on debug but hopefully this is enough):

Info: Parsing "exp_option"=auto
Info: Metainfo FPS = 23.88
Error: EOFError(541478725, 'End of file', 'avcodec_send_packet()') encountered when readingvideo frame with PyAVVideoWrapper.
Warning: Load frame failed at 343926
Info: Acceptable exception occured.
Info: ThreadVideoLoader summary:
    Video path: "/export/media/skycam/results/20260325/capture_0051_720p.mp4"; Mask: None
    Video frames = 343931; Apply grayscale = True;
    Raw resolution = [1280, 720]; Running-time resolution = [960, 540];
    Video decode backend: PyAVVideoWrapper(FFmpeg);
Apply exposure time of 0.13s.(MinTimeFlag = 879)
Total frames = 343931 ; FPS = 23.88 (rFPS = 7.96)
Info: Preprocessing finished. Time cost: 1.5s.
Info: Parsing "exp_option"=real-time
Info: Metainfo FPS = 23.88
Info: Sucessfully load /root/MetDetPy/weights/yolov5s_v2.onnx on device= CUDAExecutionProvider with Warmup=True.
  0%|                                                           | 89/114644 [00:01<31:05, 61.42it/s]Warning: Model input shape (544x960) is not strictly matched with config (540x960). Extra resize is applied to avoid error (which may increase time cost.)
 23%|████████████▊                                           | 26153/114644 [13:19<46:40, 31.60it/s]Meteor: {"start_time": "00:54:03.799", "end_time": "00:54:09.829", "video_size": [1280, 720], "target": [{"start_frame": 77475, "start_time": "00:54:03.799", "end_time": "00:54:09.829", "last_activate_frame": 77619, "last_activate_time": "00:54:09.829", "duration": 145, "speed": 0.537, "dist": 77.389, "fix_dist": 8.061, "fix_speed": 1.337, "fix_motion_duration": 6.029, "fix_duration": 6.071, "num_pts": 12, "category": "METEOR", "pt1": [497, 453], "pt2": [584, 509], "drct_loss": 0.196, "score": 0.81, "real_dist": 103.18533333333332, "raw_score": 0.67, "recheck_score": 0.941, "relative_brightness": 1.364, "aesthetic_score": 8.906}], "start_frame": 77475, "end_frame": 77619}
 35%|███████████████████▌                                    | 39979/114644 [19:10<38:32, 32.28it/s]Meteor: {"start_time": "01:22:47.378", "end_time": "01:22:48.509", "video_size": [1280, 720], "target": [{"start_frame": 118641, "start_time": "01:22:47.378", "end_time": "01:22:48.509", "last_activate_frame": 118668, "last_activate_time": "01:22:48.509", "duration": 28, "speed": 2.794, "dist": 75.432, "fix_dist": 7.858, "fix_speed": 6.951, "fix_motion_duration": 1.13, "fix_duration": 1.172, "num_pts": 24, "category": "METEOR", "pt1": [829, 84], "pt2": [854, 181], "drct_loss": 0.022, "score": 0.86, "real_dist": 100.576, "raw_score": 1, "recheck_score": 0.721, "relative_brightness": 4.163, "aesthetic_score": 28.133}], "start_frame": 118641, "end_frame": 118668}
 45%|████████████████████████▍                             | 51964/114644 [24:58<1:18:55, 13.24it/s]Meteor: {"start_time": "01:45:30.088", "end_time": "01:45:31.219", "video_size": [1280, 720], "target": [{"start_frame": 151188, "start_time": "01:45:30.088", "end_time": "01:45:30.842", "last_activate_frame": 151215, "last_activate_time": "01:45:31.219", "duration": 28, "speed": 4.352, "dist": 78.339, "fix_dist": 8.16, "fix_speed": 10.828, "fix_motion_duration": 0.754, "fix_duration": 1.172, "num_pts": 30, "category": "METEOR", "pt1": [1237, 30], "pt2": [1212, 132], "drct_loss": 0.024, "score": 0.92, "real_dist": 104.452, "raw_score": 1, "recheck_score": 0.834, "relative_brightness": 6.586, "aesthetic_score": 49.442}], "start_frame": 151188, "end_frame": 151215}
 80%|█████████████████████████████████████████████           | 92136/114644 [46:53<15:47, 23.76it/s]Meteor: {"start_time": "03:11:24.747", "end_time": "03:11:25.626", "video_size": [1280, 720], "target": [{"start_frame": 274302, "start_time": "03:11:24.747", "end_time": "03:11:25.249", "last_activate_frame": 274323, "last_activate_time": "03:11:25.626", "duration": 22, "speed": 1.951, "dist": 23.409, "fix_dist": 2.438, "fix_speed": 4.853, "fix_motion_duration": 0.502, "fix_duration": 0.921, "num_pts": 18, "category": "METEOR", "pt1": [976, 570], "pt2": [965, 600], "drct_loss": 0.044, "score": 0.78, "real_dist": 31.211999999999996, "raw_score": 1, "recheck_score": 0.551, "relative_brightness": 1.213, "aesthetic_score": 2.307}], "start_frame": 274302, "end_frame": 274323}
100%|██████████████████████████████████████████████████████▉| 114638/114644 [58:06<00:00, 33.76it/s]Error: EOFError(541478725, 'End of file', 'avcodec_send_packet()') encountered when readingvideo frame with PyAVVideoWrapper.
100%|██████████████████████████████████████████████████████▉| 114643/114644 [58:06<00:00, 32.89it/s]
Warning: Load frame failed at 343930
Info: Acceptable exception occured.
Info: VideoLoader-stop detected.
Info: Time cost: 3497.6204s.

Those 4x events in the output are all confirmed meteors, leaving a bright trail BUT the timestamps don't match up exactly with my player of choice (MPC-HC) when I play back the file however - for reference the first which says it starts at 00:54:03 actually occurs at 00:53:50 in MPC-HC so I don't know if I'm misinterpreting that or not or if they are measuring timestamps differently?

Before I ran MetDetPy, I also created a more manageable timelapse of the same entire 4h video (at 30x transcoded down to 1080p, so 8mins in length) just for my own viewing to see if I could spot any bright ones myself when watching, and I actually didn't spot the ones MetDetPy detected. HOWEVER, I did spot 2x more (even at 30x) that this tool missed - they were obvious bright trails.

They occur at ~00:13:31 and ~01:09:58

However, I think the real issue is with the "detections" json produced uploaded here and how it interacts with ClipToolkit.py.

As you can see, it is 33MB in size and includes thousands of other events - almost 3900 events with the category "DROPPED". I cannot find much about what the categories actually mean in the documentation provided.

When I run ClipToolkit.py against this json, it started creating hundreds and hundreds of tiny clips with no meteors - I left it running for a short while but gave up before it even got to the first "real" meteor.

How can I run it to ONLY clip out those positive detections?

Am I expecting too much of this tool or is my camera stream incompatible with it currently do you think?

I'm happy to abandon this tool and revert to manually scanning timelapse footage myself to find only the brightest/most impressive meteors to clip out manually otherwise...but thought this could be as useful time-saving software if it worked as I had hoped...

[Designerspr]

Thank you for the feedback.

About Filtering "DROPPED" Events (Fixed):
The "DROPPED" category contains false positives kept for debugging; these should not have been exported by default. I've pushed an emergency fix to the master branch. Please git pull the latest code to get this update. Now you can run ClipToolkit.py with the --enable-filter-rules flag to export only positive detections.

To fully utilize, you can further refine this in global/clip_cfg.json under filter_rules. Add any unwanted categories from class_name.txt to the exclude_category_list.

P.S: about time padding of the video clipping, we also add --padding-before and --padding-after option for ClipToolkit, as discussed in discussion. See config docs and tool docs for more information.

Due to time constraints, I haven't had a chance to deep-dive into the 13s offset and the missed meteors yet. I’ll investigate these as a priority and will sync with you as soon as I have a conclusion.

[Designerspr]

I’ve conducted a preliminary investigation and it appears the issue is related to Variable Frame Rate (VFR) or timestamp drift.

Specifically, when processing from the very beginning, the meteor expected at 00:13:31 shifts to 00:13:21 (about 10-second lead). However, if you use the --start-time 00:13:00 flag to begin detection closer to the event, the timestamp aligns perfectly.

This offset doesn't just affect the clips; it also degrades detection accuracy. The offset causes "recheck" model to miss the actual meteor frames, leading to false negatives (missed detections). As a result, barely no meteors are recalled.

Update: Issues confirmed and resolved. The root cause of timestamp drift was a problematic frame near the video start with a ~10-second duration, which caused the entire timeline to drift when processing from the beginning. I have fixed this by synchronizing the actual frame timestamps with the video-reader's internal counters. Besides, regrading the second meteor you mentioned, it was incorrectly classified as a satellite streak. Refining the classification accuracy is a long-going TODO. As mentioned, please git pull from the master branch to apply these latest updates.

Looking forward for your further feedback.