Mechanisms for selective lazy loading/caching #36
Description
Currently, IOs are quite limited in terms of the being able to either load everything (or whatever the IO provides), or simply to lazy load. For that reason, often, one needs to work with two Neo Blocks: a lazy one with the complete Neo Block, and a second Neo Block with only the subset of data of interest.
A better mechanism would be one where each data object could be explicitly loaded or removed from memory (and container objects doing this recursively for all their data children). Since this meachnism would imply that each IO is aware of how to load a specific part of the data, which may require massive IO restructuring, a more straight forward way to do this would be a caching system, where each data object has a "stash" and a "pop" method that temporarily staches the contents to disk and frees memory.
An even more elaborate, but complicated project is one where the stashing and poping is performed on-demand, automatically, in the background.