Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion posts/2014-10-Visualizing-MNIST/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ <h2 id="the-mnist-cube">The MNIST Cube</h2>
</script>

<p>Exploring this visualization, we can see some glimpses of the structure of MNIST. Looking at the <a href="#raw_mnist" onclick="raw_mnist_show([7,13], [18,16])">pixels <span class="math">\(p_{18,16}\)</span> and <span class="math">\(p_{7,12}\)</span></a>, we are able to separate a lot of zeros to the bottom right and a lot of nines to the top left. Looking at <a href="#raw_mnist" onclick="raw_mnist_show([5,7], [7,10])">pixels <span class="math">\(p_{5,6}\)</span> and <span class="math">\(p_{7,9}\)</span></a> we can see a lot of twos at the top right and threes at the bottom right.</p>
<p>Despite minor successes like these, one can’t really can’t understand MNIST this way. The small insights one gains feel very fragile and feel a lot like luck. The truth is, simply, that very little of MNIST’s structure is visible from these perspectives. You can’t understand images by looking at just two pixels at a time.</p>
<p>Despite minor successes like these, one can’t really understand MNIST this way. The small insights one gains feel very fragile and feel a lot like luck. The truth is, simply, that very little of MNIST’s structure is visible from these perspectives. You can’t understand images by looking at just two pixels at a time.</p>
<p>But there’s lots of other perspectives we could look at MNIST from! In these perspectives, instead of looking a face straight on, one looks at it from an angle.</p>
<p>The challenge is that we need to choose what perspective we want to use. What angle do we want to look at it from horizontally? What angle do we want to look at it from vertically? Thankfully, there’s a technique called <a href="http://en.wikipedia.org/wiki/Principal_component_analysis">Principal Components Analysis</a> (PCA) that will find the best possible angle for us. By this, we mean that PCA will find the angle that spreads out the points the most (captures the most variance possible).</p>
<p>But, what does it even mean to look at a 784-dimensional cube from an angle? Well, we need to decide which direction every axis of the cube should be tilted: to one side, to the other, or somewhere in between?</p>
Expand Down
2 changes: 1 addition & 1 deletion posts/2015-09-NN-Types-FP/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ <h2 id="representations-are-types">Representations are Types</h2>
A layer <span class="math">\(f_1\)</span> followed by a layer <span class="math">\(f_2\)</span>. The output representation of <span class="math">\(f_1\)</span> is the input of <span class="math">\(f_2\)</span>.
</div>
<p>In the case of very simple neural network architectures, where there’s just a linear sequence of layers, this isn’t very interesting. The representation of one layer’s output needs to match the representation of the next layer’s input – so what? It’s a trivial and boring requirement.</p>
<p>But many neural networks have more complicated architectures where this becomes a more interesting constraint. For a very simple example, let’s imagine a neural network with multiple similar kinds of inputs, which performs multiple, related tasks. Perhaps it takes in RGB images and also grayscale images. Maybe it’s looking at pictures of people, and trying to predict age and gender. Because the similarities between the kinds of inputs and between the kinds of tasks, it can be helpful to do all of this in one model, so that training data helps them all. The result is multiple input layers mapping into one representation, and multiple outputs mapping from the same representation.</p>
<p>But many neural networks have more complicated architectures where this becomes a more interesting constraint. For a very simple example, let’s imagine a neural network with multiple similar kinds of inputs, which performs multiple, related tasks. Perhaps it takes in RGB images and also grayscale images. Maybe it’s looking at pictures of people, and trying to predict age and gender. Because of the similarities between the kinds of inputs and between the kinds of tasks, it can be helpful to do all of this in one model, so that training data helps them all. The result is multiple input layers mapping into one representation, and multiple outputs mapping from the same representation.</p>
<div style="width:80%; margin-left:auto; margin-right:auto; margin-bottom:5px; margin-top:17px;">
<img src="img/types-branchmerge.png" alt>
</div>
Expand Down