tutorial on how to write a dashboard extension (and potentially more)

[bernhardlang]

In parallel of getting together my first dashboard extension I've written up a draft of a tutorial on how to do just that. I got the feeling, the spread of PyMoDAQ will quite depend on people in labs being capable to implement specific experimental procedures which go further than what the generic means out of the PyMoDAQ box like a DAQ scan can provide.

Since the matter is yet rather new to me, I'd be happy if the PyMoDAQxperts out there (@seb5g, @malik-irain, @quantumm, @rgeneaux, @Ashwolaa ...) could have a look at it. The idea would be first getting the code into a decent PyMoDAQian form and then continue working on / updating the tutorial text.

git@github.com:/bernhardlang/pymodaq_extension_tutorial.git
git@github.com:/bernhardlang/pymodaq_plugins_tutorial_extension.git

Once out of draft state, the idea is to get all into a single repo with tagged commits corresponding to the sections / mile stones in the tutorial, at points where the code is ready to run so that one can play around with the result.

As a starting point I've chosen a diode-array spectro-photometer (like Ocean Optics, Avantes, etc) used to measure absorption spectra in a semi-automated fashion. But all on a simulated instrument, no need for hardware. With real hardware and in form of a custom app, that actually runs since a couple of month in our physical chemistry lab classes, being heavily exposed to "fuzzy testing" by undergrad students :-)

As for now the tutorial makes a recap on plugin and controller coding to set up the simulated devices, then populates the extension template with basic stuff so that it is just capable of doing what the plugin already does, displaying currently recorded data. Last step for now setting up and controlling actions needed for measuring an absorption spectrum.

Planned further route: handling H5 storage, using the configurator, implementing a data mixer, simulating the observation of a photo-chemical reaction using a sequencer I've started to implement. I could also imagine further chapters to cover other advanced material, perhaps like lecoizing things, implementing a pid model and whatever else could be useful and appropriate.

If you like the general idea but have suggestions of a better suited starting point for covering such stuff, at this early state it wouldn't be too much of work to change that.

[seb5g]

do you think you could send us the built files? Either a zip with htmls or a pdf (make pdf). That would be either than cloning, building each time on our side.

[seb5g]

actually I built it from the source and it already looks quite nice. There are however a few things one should do:

• use new methods from CustomApp (setup_docks_and_widgets, setup_menus_and_toolbars, ...)
• modify the templates of CustomApp and CustomExt to reflect this change in the API

A few other comments:

• please try using only MaterialIcons icons (from icons.toml)
• Saving the app geometry in qsettings is actually a good and simple idea. No need for those to be in human readable form. However one also has to save the extension Parameters if one want to restore them =>we could also use the qsettings of the extension. But as for the default/initial value of those parameters, they should be included in the config_template.toml of the plugin package it is defined in. Question: should we update the config file to reflect the last values of the Parameters? Both things, I mean default value from human readable toml and last values from qsettings are two similar way of storing things. We should choose one (probably the human readable form)
• Sometimes, choosing a given value in the parameters, should also recall some last set values. This is the case of the DAQScan, where picking a set of actuators/detectors will populate the last set values of the steps (start, stop, step). It's using a mapping between Parameter and toml but is not completely satisfactory as toml cannot encode for all Paramter types (and here no need for human readable file). One could use Qsettings for this.

From this, I'm envisaging to include a general mechanism in CustomApp to store/restore a set of things an user could further increase using QSettings:

• builtin in CustomApp would be the layout/window geometries.
• including some methods to add/activate some cached mechanism
• one folder per extension will be created in .pymodaq (user) to store these

Otherwise and on the subject of the tutorial, I think is it good but because I'm so fluent in this, I'm not the best to tell if the tutorial is fit. @quantumm you're our tutorial expert, what do you think?

[bernhardlang]

* use new methods from CustomApp (setup_docks_and_widgets, setup_menus_and_toolbars, ...)

The joy of things evolving rapidly :-) I discovered this out-of-sync today myself, part of it is already updated, yay!

* modify the templates of CustomApp and CustomExt to reflect this change in the API

I'll take care of that once the dust has settled.

* please try using only MaterialIcons icons (from icons.toml)

I thought I did, probably some left-overs from my Toulouse rapid prototype hacking. I'll do the necessary clean-up ...

* Saving the app geometry in qsettings

Using one QSettings file for both config/last used and for geometry was supposed to be first draft only (and triggered the question it was supposed to trigger :-)

QSettings files are actually human readable as well, as long as human readable stuff goes in there (on Windows that may be different because by default the registry is used). However, since PyMoDAQ uses toml files basically anywhere else, I'd say human readable things should go as much as possible into toml files to keep things as uniform as possible.

Caveat for geometry storage in QSettings on Windows: when things get screwed up, being forced to edit the registry tends to be a pain in the butt. I suggest to stick to the way telling QSettings to use a dedicated file.

* Sometimes, choosing a given value in the parameters, should also recall some last set values. This is the case of the DAQScan, where picking a set of actuators/detectors will populate the last set values of the steps (start, stop, step). It's using a mapping between Parameter and toml but is not completely satisfactory as toml cannot encode for all Paramter types (and here no need for human readable file). One could use Qsettings for this.

QSettings provides groups and arrays of bundled parameters. This could be useful in such case.

From this, I'm envisaging to include a general mechanism in CustomApp to store/restore a set of things an user could further increase using QSettings:

* builtin in CustomApp would be the layout/window geometries.

* including some methods to add/activate some cached mechanism

* one folder per extension will be created in .pymodaq (user) to store these

I'd use separate files for defaults, last used and geometry, respectively. Especially the latter gets sometimes messed up. The easiest solution is then to just erase the file in question. Separate files avoid trashing still usable info.

Otherwise and on the subject of the tutorial, I think is it good but because I'm so fluent in this, I'm not the best to tell if the tutorial is fit. @quantumm you're our tutorial expert, what do you think?

I'll go through it with my colleagues in Geneva who have followed the training at EPFL, using them as guinea pigs. That should give plenty of valuable feedback as well.

[quantumm]

Hello everyone,

Just having a quick look, I think this is a huge work, so thank you for that Bernhard! We can see the teacher's hand it is very pedagogical ;)
Just a few remarks:

• the introduction should be polished a bit (but i guess you know that ;) )
• the first picture preset_tutorial.png (in Sec. Preparing the stage) is missing

As a general remark, i would say that it is not necessary to have a completely polished tutorial before publishing it. You can modify it afterward without problem. So I would rather say that it is quite already ready for publication as it is.

I hope the tutorial for publication is still quite clear and updated: https://pymodaq.cnrs.fr/en/latest/tutorials/write_documentation.html#
let me know if it is not the case ;)

That will be a very nice contribution for our documentation :)