-
Notifications
You must be signed in to change notification settings - Fork 170
Description
Is your feature request related to a problem? Please describe.
I was going to spend a couple of days adding support for loading different file types using rosetasciio directly into py4DSTEM. Is there is anything that I might need to consider before doing this.
Describe the solution you'd like
Most of the complications are going to be limiting what filetypes should be allowed to be loaded and which filetypes are not supported. Currently Rosettasciio requires dask as a dependancy, is this an issue? I could potentially remove it as a requirement and only allow for non-lazy loading if this is something that you would prefer not to add. Do you have a current process for handling memory mapped arrays, or is treating them like numpy arrays and then having numpy automatically convert to in memory arrays fine.
Additional context
This should be a good start to understanding how to generalize the process of loading/ transferring data. I think that ultimately we need to have a .yaml file for each file format which details if it outputs 4D Data, and what metadata it has.
Edit:
It seems like there is already a quite nice version here for loading using memmap vs RAM that we could extend.