Skip to content

Conversation

@kjdoore
Copy link
Collaborator

@kjdoore kjdoore commented Oct 31, 2024

This is a major update to the introductory (101) notebooks. It breaks the contents down into five parts:

  1. Introduction to chunking overall with toy example
  2. How to read existing data sets and determine their chunking
  3. Basics on selecting chunk shape and size
  4. How to write chunked data to zarr
  5. How to rechunk existing data using rechunker

I was thinking that these notebooks cover the basics to help someone understand chunking and how to perform it in a basic way. Other more complex topics (rechunking on the cloud, optimizing chunk shape, etc) would be placed in a different "Advanced Topics" (201) section of this juypter book.
Let me know your thoughts and if you think I may have missed a topic you would consider "introductory" to chunking.

@kjdoore
Copy link
Collaborator Author

kjdoore commented Nov 7, 2024

To view the notebooks in a JupyterBook view, first switch to the kjdoore-updates branch. You will then create the conda environment from the env.yml file:

conda env create -f env.yml
conda activate chunking

The book can then be created with

jupyter-book build ./

To view the book, open the following file: _build/html/index.html

theobarnhart and others added 30 commits June 11, 2025 17:18
… a few minor text edits for clarity and updated dataset sizing
… JB page from building, remove zarr that was accidentally added
- fix typos
- when using independent lat/lon chunk sizes the size of the last chunk used the incorrect space_chunk_size variable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants