Extracting data to json #2
Description
(1) When saving the skewers, we should save the ID.
(2) Also, could we change the data structure so that each skewer has 'wavelength', 'flux', 'error', 'redshift' keys that contain arrays? Currently what's being saved is a nested list, where the spectral data drill down as follows:
[skewer1, skewer2,....]
skewer1 -> [['CIV', 'HI', 'RA', 'endpoint','startpoint'],galdata1,galdata2,...]
where galdata = ['DEC','NSAID','RA','color','log_sSFR','mstars','position','redshift','rvir','sfr','sfr_err']
HI -> [['flux_norm','redshift','wavelength'],['flux_norm','redshift','wavelength'],...]
So, the part of the data including the spectrum is such that each pixel is its own dictionary. My suggestion is to change this such that each 'HI' entry (for example) would contain 'wavelength', 'flux', etc., keys that each hold arrays with length the number of spectral pixels.
(3) The 'Dec' appears to missing from the skewer level 2nd level in the above schematic.
(4) I recommend making the top level a dictionary as well such that each key is named with the skewer ID. In this way, I can get to the spectral data by jdata['TON1448']['HI']['flux_norm']
This would return the flux array. Then, I can access the galaxy data for NSAID 663587 by jdata['TON1448']['663587']['mstars']
I think this data structure will be a bit more intuitive.