Ingesting other formats than NXS datafiles

[Pasarus]

This might go away if we start using ICAT as a source of data. Then we can evaluate if this needs to be done.

What?

Allow ingestion of more than NXS files, e.g. WISH would like to also ingest .s01 files. They can be passed into the reduction as normal.

It may be possible to handle this as part of the REST API submission, specifically manual_submission.

Requires some further communication with the user!

Would it be acceptable to submit 2 messages for each file, with a different description, so that different runs show up:

• RUN NUMBER - nxs
• RUN NUMBER - s01

Does the nxs file need to be processed when an s01 is also submitted?

My understanding is that s01 are “checkpoints” of a longer-running data-acquisition run. The format is sN where N starts at 01 and may go up to infinity (clarify end number with user, e.g. it may be 99, although it seems unlikely to have more than a few)

Proposed solution

The ingestion no longer directly submits the file path, but instead a POST request to the REST-API which then creates the message and submits it to ActiveMQ.

The services need to be taught how to handle different file extensions. For the REST API this can be done via a new POST parameter, but the ingestion also needs to be updated, as currently it won’t submit requests for anything other than *.nxs

Report to: Pascal