Create a new setup page

[CarolineDenis]

Fixes #2931
Fixes #4832
Fixes #6210

Adds an initial set up page for configuring your database much like the Spwizard for Specify 6.

TODO:

Details

• Title colors are white-on-white.
• Make certain fields required (at least for user)
• Error on backend doesn't stop you from progressing onto the next step.
• Defaults for fields (So certain tree ranks can be true by default)
• Progress indicator?
• Backend should tell frontend which step to go to next. Needed to conditionally skip geography tree setup pages.
• Setup can be finished before schema is finished being created. Need to have some sort of waiting page, or speed improvement? Thinking I can change the setup so you fill out the forms first, then at the very end the database is created.
• Use new default files Feat (setup tool): Create default files for the new setup tool #7444
  • Schema overrides
  • Schema localization (english only)
  • Picklists and Global Picklists
  • Preptypes
  • Taxon treedef (per discipline) Add a new treeDefault import feature #6429
  • Geography treedef
  • Storage treedef
• Create trees
• Add documentation links
• Only use localized strings
• Add better error handling. User should be notified and sent back to the forms if anything goes wrong.
• Check if worker is set up before continuing.
• specifyuser_spprincipal table needs to exist before the setup process works on a new sp7 database. It also looks like the admin user isn't actually a proper admin. (fixed)

Future TODO:

Details

• Password confirmation input doesn't work if you input the password into it first, then fill out the normal password field.
• Notification polling seems to be causing errors on the backend when lingering on the setup page. Will investigate.
• Need schema for API
• Save form to localStorage. Also change the Save & Continue text maybe.
• Normalize all keys (recursively) on backend api requests
• Institution should have a value for all treedef fields once all trees are created. There should be api.finalize_institution or something similar.
• There should be a functioning progress bar during the setup.
• Speed up schema config localization setup. This is the biggest bottleneck in the setup process and it can be sped up significantly.
• Fix two geography tree forms showing up on the overview sidebar.
• Add documentation links and descriptions to all resources
• Move over default tree creation functions to tree folder

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)
• Add relevant issue to release milestone
• Add pr to documentation list
• Add automated tests
• Add a reverse migration if a migration is present in the PR

Testing instructions

• Use a blank DB to test this PR.
  • If using the test panel: Download this and upload a copy: https://drive.google.com/file/d/1-y1T0ecMWEhGczuSsFNW4qV7As2u5zvD/view?usp=sharing
• Try to create dbs with different disciplines. At least a paleo/geo discipline and a non-paleo/geo discipline.
• Fill out the setup forms
  • Make sure the forms look good in light mode and dark mode (It uses your system settings.)
  • You can only progress to the next form if you filled out all required fields.
  • Make sure your choices are shown in the Overview sidebar.
  • Make sure you can submit at the end.
• Make sure you can log into the database.
• NOTE: The dialog about the DB schema is expected behavior in this PR
• Make sure the defaults schema config captions and descriptions appear (not blank).
• Make sure default picklists were created correctly.
  • Note: The Agent Type picklist shouldn't exist in the schema config
• Make sure prep types were created correctly.
• Make sure all tree viewer pages load.
• Make sure the admin user's agent was created correctly. It should always have a last name.
• Make sure an empty Global Preferences file was created in App Resources.
• Note: The accounts page and stats page will show an error when opened, these will be fixed separately in Check default database constraint differences between Sp6 and Sp7 #7551
• Local testing:
  • Use an empty DB again (restore it or use a different one).
  • Stop the specify worker and try to start the setup. You should get a clear error on the frontend.
  • Make sure no migration patches are needed initially (Currently only fix_schema_config is run upon database creation, which re-runs schema related migrations)

Test the following PRs (which have been merged into this one) if you haven't already:

• Add a new treeDefault import feature #6429
• Create a configuration tool #7312

[grantfitzsimmons]

Any specifyuser can be set as an institution admin by inserting the following, replacing the specifyuser_id with the ID of the newly created user:

INSERT INTO spuserpolicy (id, resource, action, collection_id, specifyuser_id) VALUES (1, '%', '%', null, 1);

[CarolineDenis]

NOTES:

• Add schema config
  ==> Add schema config splocalcontainer when discipline is created
  Issue:
  

• In the discipline creation backend def we need (once the disc is created):

• Create a splocalcontainer for each table in models.py (fix issue mentioned above)

• Use a default schema config json file to populate

• Compare the default with the specific one for the discipline and update only what's needed. (See what we have in https://github.com/specify/specify6/tree/master/config/botany for each disc)

• (Need to add a default schema config json in sp7 + specific ones for each discipline)

• Include discipline type picklist Create a configuration tool #7312 (comment)

• Create and Populate trees: some done with: https://github.com/specify/specify7/pull/6429/files

• Need to do geography, storage, ... [Guided Setup] - Preload tree data at instantiation #4976

• Need to add geology: [Guided Setup] - Add Geology discipline to guided setup #6279

• Databases that are not used with Specify 6 are missing Remote and Global prefs

[alesan99]

@emenslin @grantfitzsimmons Ready to review again 👍 I made the following changes:

• Fixed discipline schema defaults not being applied correctly
• Fixed CollectionObjectType picklist not being created, along with other COT related fixes
• Fixed issue with empty setup error showing up
• Fixed brief screen flashing when the database setup is running

Not addressed:

• The picklist sizes match what are defined in the default picklist files, though these can be improved in the future [Guided Setup] : Fix default picklist sizelimits in default files #7640

Note:

• Some code was commented to allow for testing while certain DB tables are missing. I need to undo this before merging.
• Weblate needs to be set up for the new localization file after this PR is merged

[emenslin]

• Make sure the forms look good in light mode and dark mode (It uses your system settings.)
  • You can only progress to the next form if you filled out all required fields.
  • Make sure your choices are shown in the Overview sidebar.
  • Make sure you can submit at the end.
• Make sure you can log into the database.
• Make sure the defaults schema config captions and descriptions appear (not blank).
• Make sure default picklists were created correctly.
• Make sure prep types were created correctly.
• Make sure all tree viewer pages load.
• Make sure the admin user's agent was created correctly. It should always have a last name.
• Make sure an empty Global Preferences file was created in App Resources.

---

I'm still running into issues when creating the db where after pressing create it sends me to a loading page for a minute and then sends me back to the last setup page but with no error this time

01-12_12.29.mp4

In the setup the catalog number formats should also include a preview of what they look like instead of just the format name.

[alesan99]

@emenslin

I'm still running into issues when creating the db where after pressing create it sends me to a loading page for a minute and then sends me back to the last setup page but with no error this time

I think I figured out the issue, the loading page should be more reliable now 👍
Creating new tabs or refreshing the page should show the loading page also.

In the setup the catalog number formats should also include a preview of what they look like instead of just the format name.

I added the format to the name like schema config

Ready to test again 👍

[emenslin]

• Make sure the forms look good in light mode and dark mode (It uses your system settings.)
  • You can only progress to the next form if you filled out all required fields.
  • Make sure your choices are shown in the Overview sidebar.
  • Make sure you can submit at the end.
• Make sure you can log into the database.
• Make sure the defaults schema config captions and descriptions appear (not blank).
• Make sure default picklists were created correctly.
• Make sure prep types were created correctly.
• Make sure all tree viewer pages load.
• Make sure the admin user's agent was created correctly. It should always have a last name.
• Make sure an empty Global Preferences file was created in App Resources.

---

Looks good, all my previous comments were fixed or addressed and I didn't run into any new issues!

[Iwantexpresso]

• Make sure the forms look good in light mode and dark mode (It uses your system settings.)

  • You can only progress to the next form if you filled out all required fields.
  • Make sure your choices are shown in the Overview sidebar.
  • Make sure you can submit at the end.

• Make sure you can log into the database.

• NOTE: The dialog about the DB schema is expected behavior in this PR

• Make sure the defaults schema config captions and descriptions appear (not blank).

• Make sure default picklists were created correctly.

  • Note: The Agent Type picklist shouldn't exist in the schema config

• Make sure prep types were created correctly.

• Make sure all tree viewer pages load.

• Make sure the admin user's agent was created correctly. It should always have a last name.

• Make sure an empty Global Preferences file was created in App Resources.

• Local testing:

• Stop the specify worker and try to start the setup. You should get a clear error on the frontend.

• Make sure no migration patches are needed initially (Currently only fix_schema_config is run upon database creation, which re-runs schema related migrations)

[kwhuber]

• Make sure the forms look good in light mode and dark mode (It uses your system settings.)
• You can only progress to the next form if you filled out all required fields.
• Make sure your choices are shown in the Overview sidebar.
• Make sure you can submit at the end.
• Make sure you can log into the database.
• Make sure the defaults schema config captions and descriptions appear (not blank).
• Make sure default picklists were created correctly.
  • Note: The Agent Type picklist shouldn't exist in the schema config
• Make sure prep types were created correctly.
• Make sure all tree viewer pages load.
• Make sure the admin user's agent was created correctly. It should always have a last name.
• Make sure an empty Global Preferences file was created in App Resources.

All looked good on my end!