update week 7

Author: mhjensen

doc/pub/week7/ipynb/ipynb-week7-src.tar.gz

@@ -134,10 +134,10 @@
     "h_{l, 1}^{(t)} \\\\ \\vdots \\\\ h_{l, n_l}^{(t)}\n",
     "\\end{bmatrix},$$\n",
     "with $n_l$ being the number of features in the $l$'th hidden layer. The output of the RNN at time step $t$ is denoted\n",
-    "$$\\hat{\\vec{y}}^{(t)} = \\begin{bmatrix}\n",
+    "$$\\hat{\\mathbf{y}}^{(t)} = \\begin{bmatrix}\n",
     "\\hat{y}_1 \\\\ \\vdots \\\\ \\hat{y}_m\n",
     "\\end{bmatrix},$$\n",
-    "where the hat is there to distinguish the RNN output $\\hat{\\vec{y}}^{(t)}$ from the target value, which is denoted $\\vec{y}^{(t)}$.\n",
+    "where the hat is there to distinguish the RNN output $\\hat{\\mathbf{y}}^{(t)}$ from the target value, which is denoted $\\mathbf{y}^{(t)}$.\n",
     "The RNN will then look like this."
    ]
   },

doc/pub/week7/ipynb/rnnmath.ipynb

@@ -2,8 +2,10 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "b234bc94",
-   "metadata": {},
+   "id": "e00f1a44",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
     "doconce format html week7.do.txt --no_mako -->\n",
@@ -12,8 +14,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "498a8311",
-   "metadata": {},
+   "id": "0521f5b9",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "# Advanced machine learning and data analysis for the physical sciences\n",
     "**Morten Hjorth-Jensen**, Department of Physics and Center for Computing in Science Education, University of Oslo, Norway\n",
@@ -23,8 +27,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3a476efd",
-   "metadata": {},
+   "id": "794145b7",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Plans for the week of March 3-7\n",
     "\n",
@@ -49,8 +55,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1e075e8d",
-   "metadata": {},
+   "id": "e5e42fd9",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## What is a recurrent NN?\n",
     "\n",
@@ -68,8 +76,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bed88d90",
-   "metadata": {},
+   "id": "8ea5b91a",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Why RNNs?\n",
     "\n",
@@ -88,8 +98,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "91ce0ede",
-   "metadata": {},
+   "id": "6dadec9c",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Basic layout,  [Figures from Sebastian Rashcka et al, Machine learning with Sickit-Learn and PyTorch](https://sebastianraschka.com/blog/2022/ml-pytorch-book.html)\n",
     "\n",
@@ -102,8 +114,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1043ac0c",
-   "metadata": {},
+   "id": "f69c94e7",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail\n",
     "\n",
@@ -116,8 +130,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8404606d",
-   "metadata": {},
+   "id": "3e66ff38",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 2\n",
     "\n",
@@ -130,8 +146,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "23676fa0",
-   "metadata": {},
+   "id": "a485dcbf",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 3\n",
     "\n",
@@ -144,8 +162,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f42b4f52",
-   "metadata": {},
+   "id": "ea3e2dd7",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 4\n",
     "\n",
@@ -158,8 +178,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d0ccb5c5",
-   "metadata": {},
+   "id": "5eed88f5",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 5\n",
     "\n",
@@ -172,8 +194,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "38d64d97",
-   "metadata": {},
+   "id": "f4dd3d82",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 6\n",
     "\n",
@@ -186,8 +210,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5edcb189",
-   "metadata": {},
+   "id": "f2f57990",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## RNNs in more detail, part 7\n",
     "\n",
@@ -200,8 +226,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0afc9ec6",
-   "metadata": {},
+   "id": "f86b6161",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Backpropagation through time\n",
     "\n",
@@ -219,8 +247,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d35cf606",
-   "metadata": {},
+   "id": "40adbc7d",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## The backward pass is linear\n",
     "\n",
@@ -236,8 +266,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3d010953",
-   "metadata": {},
+   "id": "bc9082ba",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## The problem of exploding or vanishing gradients\n",
     "* What happens to the magnitude of the gradients as we backpropagate through many layers?\n",
@@ -257,32 +289,12 @@
     "RNNs have difficulty dealing with long-range dependencies."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "a44f39f1",
-   "metadata": {},
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'rnnmath' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
-      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
-      "Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mrnnmath\u001b[49m\u001b[38;5;241m.\u001b[39mipynb\n",
-      "\u001b[0;31mNameError\u001b[0m: name 'rnnmath' is not defined"
-     ]
-    }
-   ],
-   "source": [
-    "rnnmath.ipynb"
-   ]
-  },
   {
    "cell_type": "markdown",
-   "id": "474791b6",
-   "metadata": {},
+   "id": "9f380618",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## The mathematics of RNNs, the basic architecture\n",
     "\n",
@@ -291,8 +303,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "476206c8",
-   "metadata": {},
+   "id": "2b6f4cbd",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Four effective ways to learn an RNN and preparing for next week\n",
     "1. Long Short Term Memory Make the RNN out of little modules that are designed to remember values for a long time.\n",
@@ -306,8 +320,10 @@
   },
   {
    "cell_type": "markdown",
-   "id": "28845405",
-   "metadata": {},
+   "id": "ba57c26d",
+   "metadata": {
+    "editable": true
+   },
    "source": [
     "## Long Short Term Memory (LSTM)\n",
     "\n",
@@ -327,25 +343,7 @@
    ]
   }
  ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.15"
-  }
- },
+ "metadata": {},
  "nbformat": 4,
  "nbformat_minor": 5
 }