Project goals

[MichaelClerx]

Let's try to work out the goals for this project, so that we can design the software around it.

I think we can split into three types of tasks: per cell operations, batch processing, collating/reporting

Proposed scope

Stage 1: Per cell/well operations

1. Load data and meta data (by interfacing w existing libraries, plus user input)
2. Ramp-based leak correction
3. Drug subtraction
4. Reversal potential estimation
5. Capacitance spike removal
6. Classical analysis, e.g. fitting time constants
7. Individual QC measures
8. Combining QC measures into an accept/reject decision
9. Export (to CSV?)

Stage 2: Batch processing

1. Handling I/O for large numbers of cells/wells
   • knowing where to read from
   • knowing where to write to
   • knowing where to store plots (naming etc.)
2. Parallelisation
3. Order of operation for single well/cell actions, actions based on accept/reject
4. Writing CSV overview file, w all QC results + combined decision, for each cell
5. An overview of exported traces and generated plots, per cell?

Stage 3: Reporting

1. A quarto reporto
2. JSON for online QC viewer

[MichaelClerx]

Some questions next.

Plots

Should we generate these inside the methods for per cell/well operations?

Cons

• Feels like bad coding / poor separation of tasks
• Chance of repeated work when we have e.g. 2 slightly different leak correction methods

Pros

• Reduces risk that plot redoes work done by main method, creating potential for plot to be wrong!
• Operations might have internal variables that we want to use in the plot, but not store / pass around
• Generic plots might be hard to achieve, e.g. 2 slightly different leak correction methods may want to accentuate different things in a graph

[MichaelClerx]

Will try to do it clean, for now

[MichaelClerx]

Outputs

What do we want the final outputs to be?

• A json for the online viewer
• A quarto report

Should reporting be split off into a separate project?

[MichaelClerx]

Should reporting be split off into a separate project?

Was discussed earlier. Could go either way.

I think maybe joining it together is easiest, but does mean we need latex as a non-python dependency

[MichaelClerx]

Console output

At the moment, the scripts generate heaps of "logging" output in the console.
Should we (A) get rid of this, (B) keep it as a feature, and formalise the way we do this

My instinct would be to get rid of it, only printing this kind of output when we're debugging something (which ideally would be very rarely, as the per cell/well operations would all be thoroughly tested and debugged individually, never at the batch processing stage)

[MichaelClerx]

Stages 2 and 3

Much like the current set-up, I'm expecting Stage 1 to be implemented in lots of small functions, and Stages 2 and 3 in scripts (but using lots of utility functions, so waaaay shorter than we have now.

Rather than a single "does everything" script, we'd have scripts per project, allowing e.g. for different leak subtraction procedures or slight variations in protocol

[MichaelClerx]

This is the image from a year ago

[MichaelClerx]

Multiple experiment runs?

At the moment, pcpostprocess dumps all data in an output folder with deleting anything that was in there first.
Is this by design, so that we can combine multiple runs on the syncropatch into one report?
Please say no

[MichaelClerx]

If we reaaaaally wanted this, I suppose you could handle it by just having a different script in stage 2 or 3