Met Office Site Archive ML tutorial notebooks #175
Conversation
Pull Request Test Coverage Report for Build 17853138883Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
There was a problem hiding this comment.
I would suggest pulling this file out of the PR and getting users to train the model themselves, or else put the model weights in a local repository. It's a bit awkward putting model weights directly into the repo, mostly for reasons of size.
There was a problem hiding this comment.
Hi Tennessee, I can pull them out, but mainly included them as training can take quite alot of time, in the region of hours. Thus this will enable users to skip running training if they don't have time / compute to do so. The file is around 11mb. Let me know if you want it removed and I will store it somewhere else.
There was a problem hiding this comment.
Is it possible to put the trained weights in a known location (like an s3 bucket or known place on disk) and then reference it? If it's too awkward we can merge it and hope for the best, but generally it's preferred to keep large artefacts out of the repo. But it's more important to have something that works, so if those other things aren't possible we can just do it.
There was a problem hiding this comment.
That's fine. The easiest solution is to probably add them to my github and provide a download link.
There was a problem hiding this comment.
Weights removed and added to a separate repo and link provided in notebook.
There was a problem hiding this comment.
This notebook looks great
There was a problem hiding this comment.
This notebook looks great
|
Looks good, the code makes sense, I think this is all doing the right thing. It would be best to pull the model weights out however unless they really need to be there. I'm happy to include them if needed, but it would be better to keep them out of the main repo. Please run the 'black' code formatter over the code. |
* Add MO-ML tutorial notebooks and update code to support * rm MO notebook which is now in the MO site archive package * Add norm stats
3 participants
Changes:
Environment:
I used the PET Tutorials env.
Testing:
Find the ML notebook in the Met Office site archive package in the new notebooks folder. You should be able to run the notebook with the PET Tutorials env. You can choose to deactivate training and use the include weights if you would like.