Scoring, continued

In the previous post, we saw measure the average amount of surprise resulting from realizing one of $2^S$ equally-likely outcomes is $S$.

Each outcome therefore having probability $p=2^{-S}$, we may rewrite $$ S=-\lg{p} $$ where $\lg{2^x}=x$. Again, this is the average surprise. Sensibly, we may write that the surprise of any single outcome with probability $p_i$ is $$ s_i=-\lg{p_i} $$ This makes intuitive sense. If something happens which is absolutely certain ($p=1$) then we are not at all surprised ($s=-\lg{1}=0$). Likewise, if something is absolutely impossible, then we are infinitely surprised ($s=-\lg{0}=\infty$) if that impossible something comes to pass.

Now, let us use this to construct a bet based on how surprised we are. There is some event which may or may not happen (say, your favorite hockey team winning its next game.) Suppose that we agree that the probability of the event actually happening is $p$. Then with probability $p$ we will be surprised by an amount $-\lg{p}$ and with probability $1-p$ we will be surprised by an amount $-\lg{\!\left(1-p\right)}$. We expect, then, to be surprised by an amount $$ S=-p\lg{p}-\left(1-p\right)\lg{\!\left(1-p\right)} $$ Thus, we could construct a fair bet based on surprise. If you pay me $\$1$ per unit of surprise you actually experience, then to be fair I could pay you $\$S$ up front. If we played such a game over and over again with events of probability $p$ (my paying you $\$S$ each time, and you paying me $\$1$ per unit of surprise you experience) then on balance we should both break even.

In the next part, we will consider how to bet we do not actually know $p$.