[Guided Setup] Add configuration for default trees in guided setup

[alesan99]

Fixes #7593, #7641

Changes:

• Adds preload option to Geography and Taxon trees during setup. This will trigger the background tree creation like normal (for simplicity).
• Adds rank configuration to setup forms (Ugly, but can be improved)
• In empty taxon trees (with no root), there is now an import button that allows you to import a populated tree into an existing tree.
• Fix full name direction not being reverse for new chronostrat trees
• Fix nested subviews in setup forms
• Also some helpful refactoring

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)
• Add relevant issue to release milestone
• Add pr to documentation list
• Add automated tests
• Add a reverse migration if a migration is present in the PR

Testing instructions

• Guided Setup on a blank database
• Use a blank DB to test this PR.
  • If using the test panel: Download this and upload a copy: https://drive.google.com/file/d/1-y1T0ecMWEhGczuSsFNW4qV7As2u5zvD/view?usp=sharing
• Configure ranks for the geography.
• Test the new pre-load tree check box for geography and taxon trees.
• The pre-load checkbox for taxon trees should only be enabled if your discipline has a default tree (EX: Mammalogy has one, Geology doesn't).
• After the setup, make sure the default tree creation process started for the trees you selected to pre-load. (You will see notifications)
  • If you chose a paleo/geo database then a chronostrat tree should also be imported automatically.
• If you chose not to pre-load the taxon tree, the tree that was created should not have a root node.
• After the setup, check all the new trees, make sure the ranks respect your configuration.
• Pre-loaded trees should have had full names generated. You can check the full name preview on a taxon record for example.
• Configuration tool and existing trees
• Creating a new discipline with the configuration tool should create an empty taxon tree.
• An empty taxon tree should have an import button
• Make sure you can import a taxon tree into the existing empty tree
• Make sure you can still create a new populated tree
• Tree speedup testing (You may have already done some of these)
• Test with the setup page
  • Use a blank database
  • Pre-load all the trees you can and complete the setup
  • Make sure the trees were pre-loaded correctly.
• Test in the taxon tree viewer
  • Add a new taxon tree and choose to import a populated one.
  • Make sure the tree was created correctly. An easy way to check is to compare it with the same tree created before the speed ups.

[emenslin]

Preloading Taxon trees seems to generally work for me but sometimes if you edit the defaults the tree won't upload. I also ran into an issue where I preloaded the botany tree and all ranks uploaded are checked as enforced, but when uploaded there's no node in the phylum which causes an error for the node in the Class rank.

01-16_12.24.mp4

Also I have just a few concerns with the preloading taxon trees in general. First, if we do not have the taxon defaults for a discipline can we remove the preload tree checkbox? I feel like making it appear like a tree can be preloaded and then nothing happens could be very confusing.

Second, as I said above sometimes when editing the taxon tree setup and then checking preload tree it doesn't upload even for a tree that has defaults. I think it might have to do with either checking enforced for ranks that aren't a part of the defaults or with removing ranks that are included in the defaults, but I am not sure which. I think we need to find someway either for the tree to upload anyway or to prevent the user from continuing if the tree isn't going to upload. I am not sure what the best solution here is but I think something needs to be done so the user won't be led to believe a tree will be preloaded if it's not going to be.

[grantfitzsimmons]

Also I have just a few concerns with the preloading taxon trees in general. First, if we do not have the taxon defaults for a discipline can we remove the preload tree checkbox? I feel like making it appear like a tree can be preloaded and then nothing happens could be very confusing.

Yes, that should not be present if the user will end up needing to populate it after the discipline is created.

Second, as I said above sometimes when editing the taxon tree setup and then checking preload tree it doesn't upload even for a tree that has defaults. I think it might have to do with either checking enforced for ranks that aren't a part of the defaults or with removing ranks that are included in the defaults, but I am not sure which. I think we need to find someway either for the tree to upload anyway or to prevent the user from continuing if the tree isn't going to upload. I am not sure what the best solution here is but I think something needs to be done so the user won't be led to believe a tree will be preloaded if it's not going to be.

@CarolineDenis has changed the work plan for this feature, so for now I think that option will be removed. It will be handled in the fix for #7641. For now the user will have to go to the taxon tree viewer to populate it.

[kwhuber]

Testing with the Geology and Botany databases...

• Configure ranks for the geography and taxon trees.
  • Modifications to ranks applied correctly
• After the setup, make sure the default tree creation process started for the trees you selected to pre-load. (You will see notifications)
  • Verified in notifications, however Taxon did not pre-load a tree (Geography, Storage, and Chronostrat did pre-load)
  • If you chose a paleo/geo database then a chronostrat tree should also be imported automatically.
  • Verified with Geology database
• If you chose to -not- pre-load the taxon tree, the taxon tree that was created should not have a root node.
• After the setup, check all the new trees, make sure the ranks respect your configuration.
• Creating a new discipline with the configuration tool should create an empty taxon tree.
• An empty taxon tree should have an import button
• Make sure you can import a taxon tree into the existing empty tree
• Make sure you can still create a new populated tree

[alesan99]

Also I have just a few concerns with the preloading taxon trees in general. First, if we do not have the taxon defaults for a discipline can we remove the preload tree checkbox? I feel like making it appear like a tree can be preloaded and then nothing happens could be very confusing.

As @grantfitzsimmons mentioned I have now hidden the option to pre-load the taxon tree.

If I configure a tree, remove some ranks, continue, and then open the tree viewer to populate it, a new tree is created with all default ranks and settings. The original tree remains empty, resulting in two trees. ==> Sometimes it does work though

Note from discussion: this might be because its still being created

I’m also frequently encountering runtime errors when creating a discipline in the system configuration.

I believe this is because of the schema config creation, which would be solved by this: #7615

[acwhite211]

Barring the known issues and front-end unit tests, the setup tool is functioning as expected.

• Configure ranks for the geography and taxon trees.
• Test the new pre-load tree check box for geography and taxon trees.
• After the setup, make sure the default tree creation process started for the trees you selected to pre-load. (You will see notifications)
  • If you chose a paleo/geo database then a chronostrat tree should also be imported automatically.
  • Note: A taxon tree will only be preloaded if defaults exist. If there exists no default tree, it will do nothing.
• If you chose to -not- pre-load the taxon tree, the taxon tree that was created should not have a root node.
• After the setup, check all the new trees, make sure the ranks respect your configuration.
• Configuration tool and existing trees
• Creating a new discipline with the configuration tool should create an empty taxon tree.
• An empty taxon tree should have an import button
• Make sure you can import a taxon tree into the existing empty tree
• Make sure you can still create a new populated tree

I noticed that there are now two “default tree” creation implementations now:

• backend/setup_tool/tree_defaults.py has create_default_tree() and a TODO “Preload tree: pass”
• backend/trees/utils.py adds initialize_default_tree()

We could maybe write some functions for their shared functionality, but not something I want to hold this release up for.

[alesan99]

The following simplifications have been implemented:

Simplifications:

For the taxon tree configuration in the setup, some simplifications will be introduced.
The updated flow will be as follows:

• The user selects the discipline type and name.
• The user chooses whether or not to load a taxon tree (via a simple checkbox).
• All taxon tree customization options are removed from the setup process.
• Once in Specify 7, the user can access the Tree Viewer to make any desired configuration changes or remove ranks as needed.

@emenslin With these changes all your issues should have been addressed (aside from trees randomly not preloading, haven't recreated that yet so hopefully its fixed)

I noticed that there are now two “default tree” creation implementations now:

• backend/setup_tool/tree_defaults.py has create_default_tree() and a TODO “Preload tree: pass”
• backend/trees/utils.py adds initialize_default_tree()

We could maybe write some functions for their shared functionality, but not something I want to hold this release up for.

@acwhite211 Yeah I agree its kind of bad. I renamed the setup tool-specific ones to start_default_tree_from_configuration, start_preload_default_tree. Still not the best names, but it should be more clear that they serve different purposes now.
The latest version shouldn't have that TODO either 👍

[grantfitzsimmons]

• Test with the setup page
  • Use a blank database
  • Pre-load all the trees you can and complete the setup
  • Make sure the trees were pre-loaded correctly.
• Test in the taxon tree viewer
  • Add a new taxon tree and choose to import a populated one.
  • Make sure the tree was created correctly. An easy way to check is to compare it with the same tree created before the speed ups.

---

• Configure ranks for the geography.
• Test the new pre-load tree check box for geography and taxon trees.

I find it unintuitive that the tree configuration options are hidden behind a collapsed element that is just called "Details". For a user, what are these details of?

Current appearance:

I would much prefer this where we display it as a table (concept):

• The pre-load checkbox for taxon trees should only be enabled if your discipline has a default tree (EX: Mammalogy has one, Geology doesn't).

Note, Botany doesn't have 'Phylum' or 'Subphylum', this is supposed to be 'Division' and 'Subdivision'. Despite this, the default tree ranks show this:

Using the 'Pre-load Tree' checkbox leaves some issues here:

1. The user does not know what the source of these defaults are. This is important for many disciplines (Botany and Entomology, for example) where the focus can be radically different depending on the work being done in the collection. This is not enough.

   

2. The ranks need to match those that make sense for the discipline. Bizarrely, we've already solved this problem per discipline, but we are not using those definitions. See here:
   

   specify7/config/botany/taxon_botany_tree.json

   Lines 23 to 32 in 5f5f386

   	"name": "Division",
   	"enforced": true,
   	"infullname": false,
   	"rank": 30
   	},
   	{
   	"name": "Subdivision",
   	"enforced": false,
   	"infullname": false,
   	"rank": 40

3. "Pre-loading" (which I would prefer to be named "Populate with default taxonomy" or something much more intuitive) here fails as it doesn't recognize Phylum, therefore it completely skips populating that rank. There simply isn't any information now in the 'Phylum' rank despite there being data in the defaults.

---

After trying to run the build on a new Botany discipline and collection, I see this:

MissingSchema("Invalid URL 'None': No scheme supplied. Perhaps you meant https://None?")

Checking or unchecking "Pre-load Tree" makes no difference here.

---

Also problematic, we are not handling the case where

• There are multiple default trees for a single discipline (provided by CoL or WoRMs, for example)
• The default trees have ranks that are not included in the definition the user can customize