Skip to content

Commit a386f61

Browse files
mhjensenmhjensen
committed
update
1 parent 0b0b0cd commit a386f61

File tree

11 files changed

+1027
-128
lines changed

11 files changed

+1027
-128
lines changed

‎doc/pub/week8/html/week8-bs.html‎

Lines changed: 102 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,15 @@
5858
'implementing-a-memory-cell-in-a-neural-network'),
5959
('LSTM details', 2, None, 'lstm-details'),
6060
('Basic layout', 2, None, 'basic-layout'),
61+
('Basic layout', 2, None, 'basic-layout'),
62+
('Basic layout', 2, None, 'basic-layout'),
63+
('Basic layout', 2, None, 'basic-layout'),
64+
('Basic layout', 2, None, 'basic-layout'),
65+
('Basic layout', 2, None, 'basic-layout'),
66+
('Basic layout', 2, None, 'basic-layout'),
67+
('Basic layout', 2, None, 'basic-layout'),
68+
('Basic layout', 2, None, 'basic-layout'),
69+
('Basic layout', 2, None, 'basic-layout'),
6170
('More LSTM details', 2, None, 'more-lstm-details'),
6271
('The forget gate', 2, None, 'the-forget-gate'),
6372
('Input gate', 2, None, 'input-gate'),
@@ -176,6 +185,15 @@
176185
<!-- navigation toc: --> <li><a href="#implementing-a-memory-cell-in-a-neural-network" style="font-size: 80%;">Implementing a memory cell in a neural network</a></li>
177186
<!-- navigation toc: --> <li><a href="#lstm-details" style="font-size: 80%;">LSTM details</a></li>
178187
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
188+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
189+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
190+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
191+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
192+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
193+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
194+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
195+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
196+
<!-- navigation toc: --> <li><a href="#basic-layout" style="font-size: 80%;">Basic layout</a></li>
179197
<!-- navigation toc: --> <li><a href="#more-lstm-details" style="font-size: 80%;">More LSTM details</a></li>
180198
<!-- navigation toc: --> <li><a href="#the-forget-gate" style="font-size: 80%;">The forget gate</a></li>
181199
<!-- navigation toc: --> <li><a href="#input-gate" style="font-size: 80%;">Input gate</a></li>
@@ -306,7 +324,7 @@ <h2 id="reading-recommendations-autoencoders-ae" class="anchor">Reading recommen
306324
<!-- !split -->
307325
<h2 id="gating-mechanism-long-short-term-memory-lstm" class="anchor">Gating mechanism: Long Short Term Memory (LSTM) </h2>
308326

309-
<p>Besides a simple recurrent neural network layer, as discussed above, there are two other
327+
<p>Besides a simple recurrent neural network layer, as discussed during the last two weeks, there are two other
310328
commonly used types of recurrent neural network layers: Long Short
311329
Term Memory (LSTM) and Gated Recurrent Unit (GRU). For a short
312330
introduction to these layers see <a href="https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b" target="_self"><tt>https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b</tt></a>
@@ -328,6 +346,8 @@ <h2 id="gating-mechanism-long-short-term-memory-lstm" class="anchor">Gating mech
328346
<li> The information stays in the cell so long as its <b>keep</b> gate is on.</li>
329347
<li> Information can be read from the cell by turning on its <b>read</b> gate.</li>
330348
</ol>
349+
<p>The LSTM were first introduced to overcome the vanishing gradient problem.</p>
350+
331351
<!-- !split -->
332352
<h2 id="implementing-a-memory-cell-in-a-neural-network" class="anchor">Implementing a memory cell in a neural network </h2>
333353

@@ -365,6 +385,87 @@ <h2 id="basic-layout" class="anchor">Basic layout </h2>
365385
</center>
366386
<br/><br/>
367387

388+
<!-- !split -->
389+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
390+
391+
<br/><br/>
392+
<center>
393+
<p><img src="figslides/LSTM1.png" width="700" align="bottom"></p>
394+
</center>
395+
<br/><br/>
396+
397+
<!-- !split -->
398+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
399+
400+
<br/><br/>
401+
<center>
402+
<p><img src="figslides/LSTM2.png" width="700" align="bottom"></p>
403+
</center>
404+
<br/><br/>
405+
406+
<!-- !split -->
407+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
408+
409+
<br/><br/>
410+
<center>
411+
<p><img src="figslides/LSTM3.png" width="700" align="bottom"></p>
412+
</center>
413+
<br/><br/>
414+
415+
<!-- !split -->
416+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
417+
418+
<br/><br/>
419+
<center>
420+
<p><img src="figslides/LSTM4.png" width="700" align="bottom"></p>
421+
</center>
422+
<br/><br/>
423+
424+
<!-- !split -->
425+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
426+
427+
<br/><br/>
428+
<center>
429+
<p><img src="figslides/LSTM5.png" width="700" align="bottom"></p>
430+
</center>
431+
<br/><br/>
432+
433+
<!-- !split -->
434+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
435+
436+
<br/><br/>
437+
<center>
438+
<p><img src="figslides/LSTM6.png" width="700" align="bottom"></p>
439+
</center>
440+
<br/><br/>
441+
442+
<!-- !split -->
443+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
444+
445+
<br/><br/>
446+
<center>
447+
<p><img src="figslides/LSTM7.png" width="700" align="bottom"></p>
448+
</center>
449+
<br/><br/>
450+
451+
<!-- !split -->
452+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
453+
454+
<br/><br/>
455+
<center>
456+
<p><img src="figslides/LSTM8.png" width="700" align="bottom"></p>
457+
</center>
458+
<br/><br/>
459+
460+
<!-- !split -->
461+
<h2 id="basic-layout" class="anchor">Basic layout </h2>
462+
463+
<br/><br/>
464+
<center>
465+
<p><img src="figslides/LSTM9.png" width="700" align="bottom"></p>
466+
</center>
467+
<br/><br/>
468+
368469
<!-- !split -->
369470
<h2 id="more-lstm-details" class="anchor">More LSTM details </h2>
370471

‎doc/pub/week8/html/week8-reveal.html‎

Lines changed: 93 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,7 @@ <h2 id="reading-recommendations-autoencoders-ae">Reading recommendations: Autoen
244244
<section>
245245
<h2 id="gating-mechanism-long-short-term-memory-lstm">Gating mechanism: Long Short Term Memory (LSTM) </h2>
246246

247-
<p>Besides a simple recurrent neural network layer, as discussed above, there are two other
247+
<p>Besides a simple recurrent neural network layer, as discussed during the last two weeks, there are two other
248248
commonly used types of recurrent neural network layers: Long Short
249249
Term Memory (LSTM) and Gated Recurrent Unit (GRU). For a short
250250
introduction to these layers see <a href="https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b" target="_blank"><tt>https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b</tt></a>
@@ -266,6 +266,8 @@ <h2 id="gating-mechanism-long-short-term-memory-lstm">Gating mechanism: Long Sho
266266
<p><li> The information stays in the cell so long as its <b>keep</b> gate is on.</li>
267267
<p><li> Information can be read from the cell by turning on its <b>read</b> gate.</li>
268268
</ol>
269+
<p>
270+
<p>The LSTM were first introduced to overcome the vanishing gradient problem.</p>
269271
</section>
270272

271273
<section>
@@ -310,6 +312,96 @@ <h2 id="basic-layout">Basic layout </h2>
310312
<br/><br/>
311313
</section>
312314

315+
<section>
316+
<h2 id="basic-layout">Basic layout </h2>
317+
318+
<br/><br/>
319+
<center>
320+
<p><img src="figslides/LSTM1.png" width="700" align="bottom"></p>
321+
</center>
322+
<br/><br/>
323+
</section>
324+
325+
<section>
326+
<h2 id="basic-layout">Basic layout </h2>
327+
328+
<br/><br/>
329+
<center>
330+
<p><img src="figslides/LSTM2.png" width="700" align="bottom"></p>
331+
</center>
332+
<br/><br/>
333+
</section>
334+
335+
<section>
336+
<h2 id="basic-layout">Basic layout </h2>
337+
338+
<br/><br/>
339+
<center>
340+
<p><img src="figslides/LSTM3.png" width="700" align="bottom"></p>
341+
</center>
342+
<br/><br/>
343+
</section>
344+
345+
<section>
346+
<h2 id="basic-layout">Basic layout </h2>
347+
348+
<br/><br/>
349+
<center>
350+
<p><img src="figslides/LSTM4.png" width="700" align="bottom"></p>
351+
</center>
352+
<br/><br/>
353+
</section>
354+
355+
<section>
356+
<h2 id="basic-layout">Basic layout </h2>
357+
358+
<br/><br/>
359+
<center>
360+
<p><img src="figslides/LSTM5.png" width="700" align="bottom"></p>
361+
</center>
362+
<br/><br/>
363+
</section>
364+
365+
<section>
366+
<h2 id="basic-layout">Basic layout </h2>
367+
368+
<br/><br/>
369+
<center>
370+
<p><img src="figslides/LSTM6.png" width="700" align="bottom"></p>
371+
</center>
372+
<br/><br/>
373+
</section>
374+
375+
<section>
376+
<h2 id="basic-layout">Basic layout </h2>
377+
378+
<br/><br/>
379+
<center>
380+
<p><img src="figslides/LSTM7.png" width="700" align="bottom"></p>
381+
</center>
382+
<br/><br/>
383+
</section>
384+
385+
<section>
386+
<h2 id="basic-layout">Basic layout </h2>
387+
388+
<br/><br/>
389+
<center>
390+
<p><img src="figslides/LSTM8.png" width="700" align="bottom"></p>
391+
</center>
392+
<br/><br/>
393+
</section>
394+
395+
<section>
396+
<h2 id="basic-layout">Basic layout </h2>
397+
398+
<br/><br/>
399+
<center>
400+
<p><img src="figslides/LSTM9.png" width="700" align="bottom"></p>
401+
</center>
402+
<br/><br/>
403+
</section>
404+
313405
<section>
314406
<h2 id="more-lstm-details">More LSTM details </h2>
315407

‎doc/pub/week8/html/week8-solarized.html‎

Lines changed: 93 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,15 @@
8585
'implementing-a-memory-cell-in-a-neural-network'),
8686
('LSTM details', 2, None, 'lstm-details'),
8787
('Basic layout', 2, None, 'basic-layout'),
88+
('Basic layout', 2, None, 'basic-layout'),
89+
('Basic layout', 2, None, 'basic-layout'),
90+
('Basic layout', 2, None, 'basic-layout'),
91+
('Basic layout', 2, None, 'basic-layout'),
92+
('Basic layout', 2, None, 'basic-layout'),
93+
('Basic layout', 2, None, 'basic-layout'),
94+
('Basic layout', 2, None, 'basic-layout'),
95+
('Basic layout', 2, None, 'basic-layout'),
96+
('Basic layout', 2, None, 'basic-layout'),
8897
('More LSTM details', 2, None, 'more-lstm-details'),
8998
('The forget gate', 2, None, 'the-forget-gate'),
9099
('Input gate', 2, None, 'input-gate'),
@@ -252,7 +261,7 @@ <h2 id="reading-recommendations-autoencoders-ae">Reading recommendations: Autoen
252261
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
253262
<h2 id="gating-mechanism-long-short-term-memory-lstm">Gating mechanism: Long Short Term Memory (LSTM) </h2>
254263

255-
<p>Besides a simple recurrent neural network layer, as discussed above, there are two other
264+
<p>Besides a simple recurrent neural network layer, as discussed during the last two weeks, there are two other
256265
commonly used types of recurrent neural network layers: Long Short
257266
Term Memory (LSTM) and Gated Recurrent Unit (GRU). For a short
258267
introduction to these layers see <a href="https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b" target="_blank"><tt>https://medium.com/mindboard/lstm-vs-gru-experimental-comparison-955820c21e8b</tt></a>
@@ -274,6 +283,8 @@ <h2 id="gating-mechanism-long-short-term-memory-lstm">Gating mechanism: Long Sho
274283
<li> The information stays in the cell so long as its <b>keep</b> gate is on.</li>
275284
<li> Information can be read from the cell by turning on its <b>read</b> gate.</li>
276285
</ol>
286+
<p>The LSTM were first introduced to overcome the vanishing gradient problem.</p>
287+
277288
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
278289
<h2 id="implementing-a-memory-cell-in-a-neural-network">Implementing a memory cell in a neural network </h2>
279290

@@ -311,6 +322,87 @@ <h2 id="basic-layout">Basic layout </h2>
311322
</center>
312323
<br/><br/>
313324

325+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
326+
<h2 id="basic-layout">Basic layout </h2>
327+
328+
<br/><br/>
329+
<center>
330+
<p><img src="figslides/LSTM1.png" width="700" align="bottom"></p>
331+
</center>
332+
<br/><br/>
333+
334+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
335+
<h2 id="basic-layout">Basic layout </h2>
336+
337+
<br/><br/>
338+
<center>
339+
<p><img src="figslides/LSTM2.png" width="700" align="bottom"></p>
340+
</center>
341+
<br/><br/>
342+
343+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
344+
<h2 id="basic-layout">Basic layout </h2>
345+
346+
<br/><br/>
347+
<center>
348+
<p><img src="figslides/LSTM3.png" width="700" align="bottom"></p>
349+
</center>
350+
<br/><br/>
351+
352+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
353+
<h2 id="basic-layout">Basic layout </h2>
354+
355+
<br/><br/>
356+
<center>
357+
<p><img src="figslides/LSTM4.png" width="700" align="bottom"></p>
358+
</center>
359+
<br/><br/>
360+
361+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
362+
<h2 id="basic-layout">Basic layout </h2>
363+
364+
<br/><br/>
365+
<center>
366+
<p><img src="figslides/LSTM5.png" width="700" align="bottom"></p>
367+
</center>
368+
<br/><br/>
369+
370+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
371+
<h2 id="basic-layout">Basic layout </h2>
372+
373+
<br/><br/>
374+
<center>
375+
<p><img src="figslides/LSTM6.png" width="700" align="bottom"></p>
376+
</center>
377+
<br/><br/>
378+
379+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
380+
<h2 id="basic-layout">Basic layout </h2>
381+
382+
<br/><br/>
383+
<center>
384+
<p><img src="figslides/LSTM7.png" width="700" align="bottom"></p>
385+
</center>
386+
<br/><br/>
387+
388+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
389+
<h2 id="basic-layout">Basic layout </h2>
390+
391+
<br/><br/>
392+
<center>
393+
<p><img src="figslides/LSTM8.png" width="700" align="bottom"></p>
394+
</center>
395+
<br/><br/>
396+
397+
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
398+
<h2 id="basic-layout">Basic layout </h2>
399+
400+
<br/><br/>
401+
<center>
402+
<p><img src="figslides/LSTM9.png" width="700" align="bottom"></p>
403+
</center>
404+
<br/><br/>
405+
314406
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
315407
<h2 id="more-lstm-details">More LSTM details </h2>
316408

0 commit comments

Comments
 (0)