Update week5.do.txt

Author: mhjensen

doc/src/week5/week5.do.txt

@@ -300,7 +300,8 @@ $\rho_B$, that is we have for a given probability distribution $p_i$
 !split
 ===== Entropies and density matrices =====
 
-
+We discuss first the classical information entropy.
+Thereafter we define the quantum-mechanical quantity, commonly known as the Von-Neumann entropy.
 
 !split
 ===== Shannon information entropy =====
@@ -320,9 +321,89 @@ Why this expression? What does it mean?
 
 !split
 ===== Mathematics of entropy =====
-What is the basic idea of the entropy as it is used in information theory (we leave out the standard description from statistical physics here)?
 
-We want to have a measure of unlikelihod. Consider a simple binary system, with two outcomes, true and false with true given by a probability $p$ and false given by $1-p$. Since $p$ represents a probability 
+What is the basic idea of the entropy as it is used in information
+theory (we leave out the standard description from statistical physics
+here)?
+
+We want to have a measure of unlikelihood. Consider a simple binary
+system, with two outcomes, true and false with true given by a
+probability $p$ and false given by $1-p$. Since $p$ represents a
+probability, and assuming the probabilities are properly normalized,
+we have $p\in [0,1]$.
+
+Suppose know that we know precisely $p$, for example we could set
+$p=1$. This means that there is only one outcome, namely the true one
+and false is equal to zero. We know thus (exactly) which state the
+system is in.  If we set $p=0$, then we know that the outcome is
+false, without doubt.  Can we find a way to, through a specific
+function to say with a given certainty that the system is in a
+specific state? Let us call this function for $S$.
+
+!split
+===== More on the mathematics of entropy =====
+
+We could for example use
+!bt
+\[
+S(x)= -\log_2{p(x)},
+\]
+!et
+as potential
+function. Note that if we change the log-base, we only change the results by a scaling constant.
+
+When $p(x)=1$, then $S(x)=0$ and we could say that the
+probability of being in one specific state is uniquely defined since
+$S$ is then zero.  This function is also continuous, it is additive
+(which is very useful of we have independent and identically
+distributed stochastic variables), it is also high for unlikely events
+since when $p$ goes to zero (unlikely event) it becomes very
+large. It is also a non-negative function.
+
+!split
+===== Changing the expression =====
+
+We note however that if we go back to our binary model and would like to
+describe the entropy in terms of the variable $p$, we end up with $p$ being zero for the false outcome.
+The latter has probability one ($p=0)$.
+Using that
+!bt
+\[
+\lim_{x\to -\infty} -x \log_2{x}=0,
+\],
+!et
+we can define the classical information entropy as
+!bt
+\[
+S(x) = -p(x) \log_2{p(x)}.
+\]
+!et
+If we now consider a series of stochastic variables 
+$X=\{x_0,x_1,\dots,x_{n-1}\}$ with probability for an outcome $x\in X$ given by $p_X(x)$, the entropy become
+!bt
+\[
+S_X = -\sum_{x\in X}p_X(x_i) \log_2{p_X(x_i)}.
+\]
+!et
+
+!split
+===== Binary example =====
+
+If we now use the above binary example, we have (using that $p_1=p$ and $p_2=1-p$)
+!bt
+\[
+S = -\sum_{i=1}^{i=2}p_i \log_2{p_i}=-p\log_2{p}-(1-p)\log_2{1-p}.
+\]
+!et
+For $p=0$ we have $S=0$ and for $p=1$ we get $S=0$. Thus, for the two
+uniquely defined states we have entropy zero and a precise knowledge of the state. The maximum value is at
+$p=0.5$ and this is a situation where our level of knowledge is no
+better than us tossing a coin. This simple example demonstrates how we
+can use the above mathematical expression as a way to say whether we
+have a well-defined outcome or not. Our lack of knowledge is expressed
+by the largest entropy value.
+
+
 !split
 ===== Von Neumann entropy =====
 
@@ -334,6 +415,24 @@ S=-\mathrm{Tr}[\rho\log_2{\rho}].
 !et
 This is the so-called Von Neumann entropy. How did we arrive at this expression? 
 
+!split
+===== The density matrix again =====
+
+Let us assume we are studying specific system $A$. The density matrix,
+using an ONB basis $\psi_j$ is defined as
+!bt
+\[
+\rho_A=\sum_j\lambda_j\vert \psi_j\rangle\langle \psi_j\vert=\sum_j p_j\vert \psi_j\rangle\langle \psi_j\vert,
+\]
+!et
+and the eigenvalues $\lambda_j$ are the probabilities (or overlap coefficients) of being in a
+specific state $\psi_j$. 
+
+!split
+===== Linking with a new expression for the entropy =====
+
+
+
 !split
 ===== Two-qubit system and calculation of density matrices and exercise =====