MILESTONE: Download and transform free nutritional data from USDA

[jeturcotte]

Feel free to break the following into smaller tasks

• SPEC: Go to the source, get the .zip download
  • URL: https://ndb.nal.usda.gov/ndb/ ... go to downloads and get both the BFPD and SR
  • NOTE: From a very quick examination, the 'legacy' SR data looks like the better option, but that's to be determined.
• SPEC: Examine and study the contents of the data to learn how to filter out prepared meals and leave behind/import only ingredients
  • SPEC: We consider prepared meals and processed items no good
  • SPEC: We consider raw ingredients like oils, meats, vegetables, et cetera, as desirable
• SPEC: Examine and study the structure of the data to get an understanding of where the data we want actually is
  • REQ: First, let's report on what data the two OFFER and discuss that
• SPEC: Propose and discuss a proposal or two for NEW schemas that will store the parts from the chosen data set that we wish to keep or even extrapolate
• SPEC: Write a script or scripts to import the wanted subset of the data into our new structures.

[jeturcotte]

In the SR data, FD_GROUP.txt would be a good starting point for determining which things definitely are RAW ingredients and which are processed foods we wish to exclude

~0100~^~Dairy and Egg Products~
~0200~^~Spices and Herbs~
~0300~^~Baby Foods~
~0400~^~Fats and Oils~
~0500~^~Poultry Products~
~0600~^~Soups, Sauces, and Gravies~
~0700~^~Sausages and Luncheon Meats~
~0800~^~Breakfast Cereals~
~0900~^~Fruits and Fruit Juices~
~1000~^~Pork Products~
~1100~^~Vegetables and Vegetable Products~
~1200~^~Nut and Seed Products~
~1300~^~Beef Products~
~1400~^~Beverages~
~1500~^~Finfish and Shellfish Products~
~1600~^~Legumes and Legume Products~
~1700~^~Lamb, Veal, and Game Products~
~1800~^~Baked Products~
~1900~^~Sweets~
~2000~^~Cereal Grains and Pasta~
~2100~^~Fast Foods~
~2200~^~Meals, Entrees, and Side Dishes~
~2500~^~Snacks~
~3500~^~American Indian/Alaska Native Foods~
~3600~^~Restaurant Foods~

NOTE: it LOOKS like something are "something" and A~B is basically A,B ... to get around the need for quotes and commas in some of the tables

so the snacks and fast foods are almost certainly out; fruits are in (though fruit juices maybe not), et cetera...

BAKED goods, for example, if we're concerned about health, ought to be excluded since most brands these days take out the fiber and add sugar, leading to what folk from europe would call 'cake' ... so we'd hope folk would use this app to devise their own healthier baked goods.

[jeturcotte]

Anyhow; let's use this to have remote conversation about this task

[jeturcotte]

Not that it's late by any means, but I haven't really heard from anyone nor have I been able to identify the three students from among the list of group members. @mattva01 or @jelkner ... anyone on there you recogize that I can add so that we don't fail to communicate?