Skip to content

Commit b20cd24

Browse files
committed
[QC-1286] up to modules dev
1 parent 400ebaa commit b20cd24

File tree

8 files changed

+56
-65
lines changed

8 files changed

+56
-65
lines changed

README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ For a general overview of our (O2) software, organization and processes, please
1717
* [Environment loading](doc/QuickStart.md#environment-loading)
1818
* [Execution](doc/QuickStart.md#execution)
1919
* [Basic workflow](doc/QuickStart.md#basic-workflow)
20-
* [Readout chain (optional)](doc/QuickStart.md#readout-chain-optional)
2120
* [Post-processing example](doc/QuickStart.md#post-processing-example)
2221
* [Modules development](doc/ModulesDevelopment.md)
2322
* [Context](doc/ModulesDevelopment.md#context)

doc/Configuration.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,6 @@ In production at P2 and in staging, some common items are defined globally in th
147147
* consul
148148
* conditionDB
149149
* bookkeeping
150-
*
151150

152151
It is mandatory to use them by including the file:
153152

@@ -578,7 +577,9 @@ http://alio2-cr1-hv-mvs00.cern.ch:32188/components/qc/ANY/any/templating_demo?pr
578577

579578
--> the file is included and the condition is true thus we have an extra line.
580579

581-
## Definition and access of simple user-defined task configuration ("taskParameters")
580+
## Definition and access of user-defined task configuration
581+
582+
### Simple and limited approach ("taskParameters")
582583

583584
The new, extended, way of defining such parameters, not only in Tasks but also in Checks, Aggregators and PP tasks,
584585
is described in the next section.
@@ -597,7 +598,7 @@ The syntax is
597598

598599
It is accessed with : `mCustomParameters["myOwnKey"]`.
599600

600-
## Definition and access of user-defined configuration ("extendedTaskParameters")
601+
### Advanced approach with beam and run type parametrization ("extendedTaskParameters")
601602

602603
User code, whether it is a Task, a Check, an Aggregator or a PostProcessing task, can access custom parameters declared in the configuration file.
603604
They are stored inside an object of type `CustomParameters` named `mCustomParameters`, which is a protected member of `TaskInterface`.
@@ -718,4 +719,4 @@ In a postprocessing task, it is available in the objects manager: `getObjectsMan
718719

719720
---
720721

721-
[← Go back to Framework](Framework.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Frequently Asked Questions →](FAQ.md)
722+
[← Go back to Framework](Framework.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to FLP Suite →](FLPsuite.md)

doc/FLPsuite.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,4 +205,4 @@ See the details [here](https://github.com/AliceO2Group/QualityControl/blob/maste
205205

206206
---
207207

208-
[← Go back to Post-processing](PostProcessing.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Configuration](Configuration.md)
208+
[← Go back to Configuration](Configuration.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Miscellaneous](Miscellaneous.md)

doc/Miscellaneous.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,15 @@ Miscellaneous
1010
* [Display a non-standard ROOT object in QCG](#display-a-non-standard-root-object-in-qcg)
1111
* [Canvas options](#canvas-options)
1212
* [Local QCG (QC GUI) setup](#local-qcg-qc-gui-setup)
13-
* [Data Sampling monitoring](#data-sampling-monitoring)
14-
* [Monitoring metrics](#monitoring-metrics)
15-
* [Common check IncreasingEntries](#common-check-increasingentries)
16-
* [Common check TrendCheck](#common-check-trendcheck)
17-
* [Full configuration example](#full-configuration-example)
18-
* [Update the shmem segment size of a detector](#update-the-shmem-segment-size-of-a-detector)
19-
* [Readout chain (optional)](#readout-chain-optional)
20-
* [Getting real data from readout](#getting-real-data-from-readout)
21-
* [Readout data format as received by the Task](#readout-data-format-as-received-by-the-task)
13+
* [Data Sampling monitoring](#data-sampling-monitoring)
14+
* [Monitoring metrics](#monitoring-metrics)
15+
* [Common check IncreasingEntries](#common-check-increasingentries)
16+
* [Common check TrendCheck](#common-check-trendcheck)
17+
* [Full configuration example](#full-configuration-example)
18+
* [Update the shmem segment size of a detector](#update-the-shmem-segment-size-of-a-detector)
19+
* [Readout chain (optional)](#readout-chain-optional)
20+
* [Getting real data from readout](#getting-real-data-from-readout)
21+
* [Readout data format as received by the Task](#readout-data-format-as-received-by-the-task)
2222
<!--te-->
2323

2424
[← Go back to Post-processing](PostProcessing.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Frequently Asked Questions →](FAQ.md)
@@ -360,4 +360,4 @@ To change the fraction of the data being monitored, change the option `fraction`
360360

361361
---
362362

363-
[← Go back to Post-processing](PostProcessing.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Configuration](Configuration.md)
363+
[← Go back to FLP Suite](FLPsuite.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to FAQ](FAQ.md)

doc/ModulesDevelopment.md

Lines changed: 32 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -3,38 +3,35 @@
33
<!--TOC generated with https://github.com/ekalinin/github-markdown-toc-->
44
<!--./gh-md-toc --insert --no-backup --hide-footer --indent 3 /path/to/README.md-->
55
<!--ts-->
6-
* [Modules development](#modules-development)
7-
* [Context](#context)
8-
* [QC architecture](#qc-architecture)
9-
* [DPL](#dpl)
10-
* [Data Sampling](#data-sampling)
11-
* [Custom Data Sampling Condition](#custom-data-sampling-condition)
12-
* [Bypassing the Data Sampling](#bypassing-the-data-sampling)
13-
* [Code Organization](#code-organization)
14-
* [Developing with aliBuild/alienv](#developing-with-alibuildalienv)
15-
* [User-defined modules](#user-defined-modules)
16-
* [Repository](#repository)
17-
* [Paths](#paths)
18-
* [Module creation](#module-creation)
19-
* [Test run](#test-run)
20-
* [Saving the QC objects in a local file](#saving-the-qc-objects-in-a-local-file)
21-
* [Modification of the Task](#modification-of-the-task)
22-
* [Check](#check)
23-
* [Configuration](#configuration)
24-
* [Implementation](#implementation)
25-
* [Results](#results)
26-
* [Quality Aggregation](#quality-aggregation)
27-
* [Quick try](#quick-try)
28-
* [Configuration](#configuration-1)
29-
* [Implementation](#implementation-1)
30-
* [Naming convention](#naming-convention)
31-
* [Committing code](#committing-code)
32-
* [Data sources](#data-sources)
33-
* [Readout](#readout)
34-
* [DPL workflow](#dpl-workflow)
35-
* [Run number and other run attributes (period, pass type, provenance)](#run-number-and-other-run-attributes-period-pass-type-provenance)
36-
* [A more advanced example](#a-more-advanced-example)
37-
* [Monitoring](#monitoring)
6+
* [Context](#context)
7+
* [QC architecture](#qc-architecture)
8+
* [DPL](#dpl)
9+
* [Data Sampling](#data-sampling)
10+
* [Code Organization](#code-organization)
11+
* [Developing with aliBuild/alienv](#developing-with-alibuildalienv)
12+
* [User-defined modules](#user-defined-modules)
13+
* [Repository](#repository)
14+
* [Paths](#paths)
15+
* [Module creation](#module-creation)
16+
* [Test run](#test-run)
17+
* [Saving the QC objects in a local file](#saving-the-qc-objects-in-a-local-file)
18+
* [Modification of the Task](#modification-of-the-task)
19+
* [Check](#check)
20+
* [Configuration](#configuration)
21+
* [Implementation](#implementation)
22+
* [Results](#results)
23+
* [Quality Aggregation](#quality-aggregation)
24+
* [Quick try](#quick-try)
25+
* [Configuration](#configuration-1)
26+
* [Implementation](#implementation-1)
27+
* [Naming convention](#naming-convention)
28+
* [Committing code](#committing-code)
29+
* [Data sources](#data-sources)
30+
* [Readout](#readout)
31+
* [DPL workflow](#dpl-workflow)
32+
* [Run number and other run attributes (period, pass type, provenance)](#run-number-and-other-run-attributes-period-pass-type-provenance)
33+
* [A more advanced example](#a-more-advanced-example)
34+
* [Monitoring](#monitoring)
3835
<!--te-->
3936

4037
[← Go back to Quickstart](QuickStart.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Post-processing →](PostProcessing.md)
@@ -47,13 +44,13 @@ Before developing a module, one should have a bare idea of what the QualityContr
4744

4845
![alt text](images/Architecture.png)
4946

50-
The main data flow is represented in blue. Data samples are selected by the Data Sampling (not represented) and sent to the QC tasks, either on the same machines or on other machines. The tasks produce TObjects, usually histograms, encapsulated in a MonitorObject that are merged (if needed) and then checked. The checkers output a QualityObject along with the MonitorObjects which might have been modified. The MonitorObjects and the QualityObjects are stored in the repository. The QualityObjects can also optionally be aggregated by the Aggregators to produce additional QualityObjects that are also saved in the database.
47+
The main data flow is represented in blue. Data samples are selected by the Data Sampling (not represented) and sent to the QC tasks, either on the same machines or on other machines. The tasks produce TObjects, usually histograms, encapsulated in a MonitorObject that are merged (if needed) and then checked. The checkers output a QualityObject along with the MonitorObjects which might have been modified. The MonitorObjects and the QualityObjects are stored in the repository. The QualityObjects can also be aggregated by the Aggregators to produce additional QualityObjects that are also saved in the database.
5148

5249
Asynchronously, the Post-processing can retrieve MonitorObjects from the database when certain events happen (new version of an object, new run) and produce new TObjects such as a trending plot.
5350

5451
### DPL
5552

56-
[Data Processing Layer](https://github.com/AliceO2Group/AliceO2/blob/dev/Framework/Core/README.md) is a software framework developed as a part of O2 project. It structurizes the computing into units called _Data Processors_ - processes that communicate with each other via messages. DPL takes care of generating and running the processing topology out of user declaration code, serializing and deserializing messages, providing the data processors with all the anticipated messages for a given timestamp and much more. Each piece of data is characterized by its `DataHeader`, which consists (among others) of `dataOrigin`, `dataDescription` and `SubSpecification` - for example `{"MFT", "TRACKS", 0}`.
53+
[Data Processing Layer](https://github.com/AliceO2Group/AliceO2/blob/dev/Framework/Core/README.md) is a software framework developed as a part of the O2 project. It defines and runs workflows made of _DataProcessors_ (aka _Devices_) communicating with each other via messages.
5754

5855
An example of a workflow definition which describes the processing steps (_Data Processors_), their inputs and their outputs can be seen in [runBasic.cxx](https://github.com/AliceO2Group/QualityControl/blob/master/Framework/src/runBasic.cxx). In the QC we define the workflows in files whose names are prefixed with `run`.
5956

@@ -150,13 +147,11 @@ One can of course build using `aliBuild` (`aliBuild build --defaults o2 QualityC
150147
After the initial use of `aliBuild`, which is necessary, the correct way of building is to load the environment with `alienv` and then go to the build directory and run `make` or `ninja`.
151148

152149
```
153-
alienv load QualityControl/latest
150+
alienv enter QualityControl/latest
154151
cd sw/BUILD/QualityControl-latest/QualityControl
155152
make -j8 install # or ninja -j8 install , also adapt to the number of cores available
156153
```
157154

158-
If you need to use the QCG or Readout, load `O2Suite` instead of `QualityControl`.
159-
160155
### User-defined modules
161156

162157
The Quality Control uses _plugins_ to load the actual code to be executed by the _Tasks_, the _Checks_, the _Aggregators_ and the _PostProcessing_. A module, or plugin, can contain one or several of these classes. They must subclass the corresponding interfaces, for example `TaskInterface.h` or `CheckInterface.h`. We use the Template Method Design Pattern.

doc/PostProcessing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1155,4 +1155,4 @@ Use the Activity which leaves the run number empty, but indicate the pass and pe
11551155
}
11561156
```
11571157

1158-
[← Go back to Modules Development](ModulesDevelopment.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Advanced Topics →](Advanced.md)
1158+
[← Go back to Modules Development](ModulesDevelopment.md) | [↑ Go to the Table of Content ↑](../README.md) | [Continue to Framework →](Framework.md)

doc/QuickStart.md

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,13 @@
33
<!--TOC generated with https://github.com/ekalinin/github-markdown-toc-->
44
<!--./gh-md-toc --no-backup --hide-footer --indent 3 /path/to/README.md-->
55
<!--ts-->
6-
* [QuickStart](#quickstart)
7-
* [Read this first!](#read-this-first)
8-
* [Requirements](#requirements)
9-
* [Setup](#setup)
10-
* [Environment loading](#environment-loading)
11-
* [Execution](#execution)
12-
* [Basic workflow](#basic-workflow)
13-
* [Readout chain (optional)](#readout-chain-optional)
14-
* [Getting real data from readout](#getting-real-data-from-readout)
15-
* [Readout data format as received by the Task](#readout-data-format-as-received-by-the-task)
16-
* [Post-processing example](#post-processing-example)
6+
* [Read this first!](#read-this-first)
7+
* [Requirements](#requirements)
8+
* [Setup](#setup)
9+
* [Environment loading](#environment-loading)
10+
* [Execution](#execution)
11+
* [Basic workflow](#basic-workflow)
12+
* [Post-processing example](#post-processing-example)
1713
<!--te-->
1814

1915
[↑ Go to the Table of Content ↑](../README.md) | [Continue to Modules Development →](ModulesDevelopment.md)

doc/images/Architecture.png

-4.18 KB
Loading

0 commit comments

Comments
 (0)