Skip to content

Add Pangeo CMIP6 ML training demo notebook - #289

Open
ahmedmohiduet wants to merge 14 commits into
ACCESS-Community-Hub:developfrom
ahmedmohiduet:286-ml-notebook
Open

ahmedmohiduet wants to merge 14 commits into
ACCESS-Community-Hub:developfrom
ahmedmohiduet:286-ml-notebook

Conversation

@ahmedmohiduet

Copy link
Copy Markdown

Closes #286 (part of #276)

Adds docs/notebooks/data/Pangeo_CMIP6_ML.ipynb, ML demo notebook for PangeoCMIP6 accessor from #284.

Covers:

  • config cell: variables list plus start/end dates per split
  • data pipeline: PangeoCMIP6, ConvertUnits, Drop, Deviation, TemporalRetrieval (delta_unit='month'), ToNumpy, Squeeze, Cache
  • monthly DateRange train and validation splits
  • small CNN trained on CPU via Train and PipelineLightningDataModule
  • prediction plot against target

Also adds row in Gallery.ipynb and .gitignore entry for training output.

Notes:

  • Depends on Create ArchiveIndex derived accessor class #284 and Demo Notebook #285, both unmerged. Branch has both merged in, so diff includes their commits until they land in develop.
  • Executed in full on kernel pet_nb_cpu. Outputs saved, no errors.
  • Needs network access for Google Cloud storage.
  • Demo only: 3 epochs, so prediction is rough and plot shows cold bias.
  • Normalisation uses fixed norm_mean and norm_std values.

ahmedmohiduet and others added 14 commits October 5, 2026 01:31
Adds packages/third_party/pangeo_site_archive with PangeoCMIP6, an ArchiveIndex that reads CMIP6 variables from the Pangeo ARCO cloud archive, plus offline unit tests, CI install/coverage entries and a docs section. Closes ACCESS-Community-Hub#284.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Unqqtp91DWVC8ro9Y724iN
Returns the Pangeo CMIP6 catalogue as a pandas.DataFrame so users can discover available data. Includes a unit test.
Demonstrate catalogue discovery, lazy `retrieve`, global map, annual global mean and multiple variables with `PangeoCMIP6`, derived from `ArchiveIndex`. Link notebook from `Gallery.ipynb`.

Refs ACCESS-Community-Hub#285
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Demo ML notebook with Pangeo CMIP6

1 participant