Simplifying to a single file, perf improvements, better user experience #7

JesseBuesking · 2022-12-03T21:14:46Z

I've broken up my changes into several commits to make it easier to understand what's going on.

I've combined the picam and usb independent files into a single file, that way any code changes done won't have to be replicated over multiple files. README updated to reflect the change. Basically pass --picam, --webcam, or --device X to choose your target.

Next up I made it possible to change the frameWidth and frameHeight without breaking the code. That way i can make the frame bigger so that selecting a specific pixel becomes easier.

After that we have several performance improvements. I basically profiled for both speed and memory, and made changes to address both. Nothing too crazy, just moving some logic out of the main loop that can be computed once and reused from within the loop. Also trading for loops for builtin numpy functions in spots, and finally reusing arrays instead of allocating new ones. Note: to simplify the review of a particular change here, I'd add ?w=1 to the end of the url above. That ignores whitespace changes, and there are a bunch of lines that were indented which will make it harder to review (IMO).

Finally I added logic when selecting a point to do calibration that looks for the nearest peak (within 5 places at the moment). That way if your abilities with a mouse are rubbish, we'll help you select the peak you were aiming for. Right-clicking undoes the last selection in case you make a mistake.

Let me know if you have any questions. Perf boosts result in a roughly 1.8x FPS increase on my Pi4, which makes the experience a bit smoother. I mainly started doing the perf changes because I increased the frameWidth and frameHeight which slows down cv2s drawing speed, making the experience less pleasant.

- allocating arrays ahead of time to save mem/improve speed - prebuilding graph outside of main loop - prebuilding rgb associated colors for calibrated data

JesseBuesking · 2022-12-03T21:23:37Z

src/PySpectrometer2.py

+
+			cv2.imshow(title2,waterfall_vertical)
+
+		while True:


Put this in a loop so that if I hold down t or g to adjust the gain, it responds smoothly.

JesseBuesking · 2022-12-03T21:24:31Z

src/PySpectrometer2.py

+	# from skewing measurements below
+	cap.isOpened()
+
+def runall():


I turned this into a function so that I could more easily profile the logic.

JesseBuesking · 2022-12-03T21:25:13Z

src/PySpectrometer2.py

+		result.append(rgb)
+	return result
+
+wavelength_data_rgbs = compute_wavelength_rgbs(wavelengthData)


Precomputing rgb values outside the main while loop for speed/memory.

JesseBuesking · 2022-12-03T21:25:38Z

src/PySpectrometer2.py

 tens = (graticuleData[0])
 fifties = (graticuleData[1])

+# load the banner image once


Preload the banner once outside the loop.

JesseBuesking · 2022-12-03T21:26:52Z

src/PySpectrometer2.py

+
+	return result
+
+graph_base = build_graph_base()


Build the background of the display once, this way we can use np.copy to just overwrite the display at the start of each loop which is much faster and uses less memory.

JesseBuesking · 2022-12-03T21:27:29Z

src/PySpectrometer2.py

+		# reset the graph to the base
+		np.copyto(graph, graph_base)
+
+		num_mean = 3 # average the data from this many rows of pixels


This means you could possibly make the number of rows that are averaged a passable argument.

JesseBuesking · 2022-12-03T21:29:10Z

src/PySpectrometer2.py

 		mouseX = x
 		mouseY = y-mouseYOffset
-		clickArray.append([mouseX,mouseY])
+		if len(peak_intensities) > 0:


I'm not sure if selecting a peak within +/- 5 pixels is better, or if choosing the closest label is. I've left +/-5 on, but maybe this could also be an option enabled by passable arguments?

JesseBuesking · 2022-12-03T21:31:41Z

src/specFunctions.py

+
+    return peaks    
+
+def compute_wavelengths(width, pixels, wavelengths, errors):


Splitting readcal in two. This way we can call the function separately elsewhere.

I use this when calibrating a set of points so that I can output the error of the current selection. That way I can choose to update the caldata file or undo my changes if I've managed to do worse than my previous calibration.

JesseBuesking · 2022-12-03T21:32:47Z

src/specFunctions.py

+    pxdata = ','.join(map(str, pxdata)) #convert array to string
+    wldata = ','.join(map(str, wldata)) #convert array to string
+    with open('caldata.txt','w') as f:
+        f.write(str(frameWidth)+'\r\n')


Include the current frameWidth with the data. This way if the frame size is later changed, the calibration data still applies.

JesseBuesking · 2022-12-05T20:40:26Z

src/PySpectrometer2.py

-		#find peaks and label them
-		textoffset = 12
+
+		np.clip(intensity, 0, 255, out=intensity)


Using numpy to draw the intensity data using vector ops over python loops.

JesseBuesking · 2022-12-05T20:41:14Z

src/specFunctions.py

-            dy[plateau[plateau < median]] = dy[plateau[0] - 1]
-            # set rightmost and middle values to rightmost non zero values
-            dy[plateau[plateau >= median]] = dy[plateau[-1] + 1]
+            lplat = plateau.shape[0]


I noticed that the data in plateau is always in sorted order, so we can skip finding medians and know that the middle element is the median.

JesseBuesking · 2022-12-05T20:43:59Z

src/specFunctions.py

+            right = dy[plateau[lplat-1] + 1]
+            middle = lplat // 2
+            # always in sorted order
+            median = plateau[middle]


It's actually faster to just loop over the plateau elements once and do something with each element than to do the vectorized ops they were performing before. The changes in this function result in a roughly 4+x speedup which is awesome because the peak finding logic is a big cause of slowness when running the loop.

jlpoolen · 2025-10-25T02:44:19Z

I've reviewed most of JesseBuesking patch and they look to be good improvements. Yet, I am perplexed why there has been no response on Jesse's request. So, here's what I did:

downloaded 2022-09-22-raspios-bullseye-arm64-full.img.xz from https://downloads.raspberrypi.org/raspios_full_arm64/images/raspios_full_arm64-2022-09-26/ This is the closest Bullseye release to Jesse's request of December 2022. Les' project specifically states that Bullseye is the supporting release of the operating system.
unzipped and burned the image onto an SD card and booted up the Bullseye instance
installed opencv
git clones Les' project into /usr/local/src/PySpectrometer2
successfully launched src/PySpectrometer2-Picam2-v1.0.py
git cloned Jesse's project into /usr/local/src/jesse
attempted to run PySpectrometer2.py and it errored out:

jlpoole@raspberrypi:/usr/local/src/jesse/src $ date; python PySpectrometer2.py 
Fri 24 Oct 2025 07:17:39 PM PDT
Loading calibration data...
Loading of Calibration data failed (missing caldata.txt or corrupted data!
Loading placeholder data...
You MUST perform a Calibration to use this software!


Calculating second order polynomial...
           2
9.125e-05 x + 0.4375 x + 380
Generating Wavelength Data!


Done! Note that calibration with only 3 wavelengths will not be accurate!
Traceback (most recent call last):
  File "/usr/local/src/jesse/src/PySpectrometer2.py", line 295, in <module>
    cap.isOpened()
AttributeError: 'NoneType' object has no attribute 'isOpened'
jlpoole@raspberrypi:/usr/local/src/jesse/src $

I then tried specifying "--picam" as a command parameter, that allowed the program to launch.

jlpoole@raspberrypi:/usr/local/src/jesse/src $ date; python PySpectrometer2.py --picam
Fri 24 Oct 2025 07:30:32 PM PDT
[0:37:47.858740752] [3558]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3866-0c55e522
[0:37:47.886550582] [3559]  INFO RPI raspberrypi.cpp:1374 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media4 and ISP device /dev/media1
{'AnalogueGain': 10.0,
 'FrameDurationLimits': (33333, 33333),
 'NoiseReductionMode': <NoiseReductionModeEnum.Fast: 1>}
[0:37:47.890701517] [3558]  INFO Camera camera.cpp:1035 configuring streams: (0) 800x600-RGB888
[0:37:47.891086065] [3559]  INFO RPI raspberrypi.cpp:761 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
Loading calibration data...
Loading of Calibration data failed (missing caldata.txt or corrupted data!
Loading placeholder data...
You MUST perform a Calibration to use this software!


Calculating second order polynomial...
           2
3.125e-05 x + 0.4375 x + 380
Generating Wavelength Data!


Done! Note that calibration with only 3 wavelengths will not be accurate!
jlpoole@raspberrypi:/usr/local/src/jesse/src $

I'm working on a version for Raspberry Pi 'Trixie' and cannot think of a reason why I should not incorporate all, or some of Jesse's changes. Jesse's changes and explanation of each change looks very well considered and documented, thank you, Jesse.

Jesse's patch does alter the behavior insofar as it requires the user to specify "--picam" instead of no parameter. My Raspberry 4B has the Adafruit #5661 "Raspberry Pi High Quality Camera – M12 Lens Mount" which has the Sony IMX477R stacked, back-illuminated sensor, 12.3 megapixels, 7.9 mm sensor diagonal, 1.55 μm × 1.55 μm pixel size. (Note: this larger sensor size changes the assumptions of lens choice, e.g. 12mm, in this project's README due to the increased sensor size.)

Conclusion:

Unless someone speaks up soon, I'll incorporate Jesse's proposed changes into my Trixie build.

JesseBuesking added 7 commits December 3, 2022 12:02

combine logic into a single fle

d45e46b

updating readme

8ead4f0

more readme changes

61dd6c1

support for different frameWidth and frameHeight

eca08d0

performance improvements

7f5bd00

- allocating arrays ahead of time to save mem/improve speed - prebuilding graph outside of main loop - prebuilding rgb associated colors for calibrated data

update the graticule when modified

ab47a5a

take the peak nearest to where the user clicked

25c3738

JesseBuesking commented Dec 3, 2022

View reviewed changes

JesseBuesking added 2 commits December 3, 2022 17:46

fixing bugs in mouse clicks and graticule

005248d

a bit of cleanup, and a few more perf improvements

7e21695

JesseBuesking commented Dec 5, 2022

View reviewed changes

codeflight1 mentioned this pull request Mar 6, 2023

Refactoring for readability #5

Closed


		return peaks

		def compute_wavelengths(width, pixels, wavelengths, errors):

Simplifying to a single file, perf improvements, better user experience #7

Are you sure you want to change the base?

Simplifying to a single file, perf improvements, better user experience #7

Uh oh!

Conversation

JesseBuesking commented Dec 3, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jlpoolen commented Oct 25, 2025

Conclusion:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants