role_of_noise/firing_rate.tex

\section*{High and low firing rates}

One key factor that determines how well a neuron can encode a given signal is
the firing rate of the neuron. Though it has been shown to be possible for
neurons to encode signals with frequencies above their firing rate \citep{knight1972},
in general higher firing rates lead to a better encoding of the signal. 
In our simulations the firing rate of the neurons depends on the noise added
to the neurons. The effect can be seen in figure \ref{firingrates}, which shows
average firing rate as a function of the mean input for different noise strengths.
The largest differences can be seen for inputs with a mean around the firing threshold
(10\milli\volt) and below. While for weak noise (gray) there is an obvious non-linearity
at the firing threshold, increasing noise strength linearizes the average firing rate.
This illustrates subthreshold SR quite well, as the noise induces firing of the neurons
for signals which would otherwise be too weak to elicit a response.
For stronger mean inputs like the 15\milli\volt\usk (second vertical black line) 
we use in our simulations firing rate is roughly linear with input strength and 
is not sensitive to changes in the noise strength. Therefore, the effect of noise
on the average firing rate of the neurons plays at most a weak role in the
explanation of SSR. However, the mean input strength is very important because of 
its effects on the average firing rate, as we will show below.\footnote{could use a plot comparing high/low directly}
We found that in our simulations, the amplitude of the signal has a negligible 
influence on the firing rate.




Previously we looked at the optimum noise value for a given a population size. Now we look
at optimal population sizes for a given noise value: We define a
"maximum" coding fraction as the coding fraction at a population size of 4096 neurons,
if the coding faction at this population size is no more than 2\% greater than the coding
fraction for a population of 2048 neurons. This ensures that coding
fraction has reasonably converged at this point. To see why this is important,
compare the yellow line (noise strength \(10^{-3}\milli\volt\square\per\hertz\)) 
in figure \ref{CodingFrac} B to the other lines in that figure.
The coding fraction is still rising with increasing population size and there is no reliable
way to estimate for which population size it will ultimately converge and what the maximum coding
fraction will be. We are only using the noise strengths for which coding fraction has
converged in the following analysis.
Then, we define the "optimum" population size as the size where coding fraction is 95\% of the maximum
coding fraction, using linear interpolation between the different population sizes. Exponential interpolation
yielded essentially the same results. We call this population size "optimal" because we assume that, for efficiency reasons, 
population size should be as small as possible while having very good encoding capabilities.  

\subsection*{Strong average input (high firing rates)}



\begin{figure}
	\centering
	\includegraphics[width=0.32\linewidth]{{img/popsize_15.0_1.0}.pdf}
	\includegraphics[width=0.32\linewidth]{{img/max_cf_15.0_1.0}.pdf}	
	\includegraphics[width=0.32\linewidth]{{img/improvement_15.0_1.0}.pdf}
	
	\includegraphics[width=0.32\linewidth]{{img/popsize_10.5_1.0}.pdf}
	\includegraphics[width=0.32\linewidth]{{img/improvement_10.5_1.0}.pdf}
    \includegraphics[width=0.32\linewidth]{{img/max_cf_10.5_1.0}.pdf}	
	\caption{An overview of the effect of different noise strengths on maximum coding fraction and population size. We only considered noise values where the difference in coding fraction between population sizes n=4096 and n=2048 is less than 2\%. Average firing rate of the neurons was about 91\hertz. Input strengths where chosen so taht the power of the signal in the corresponding bands is the same (1.0\milli\volt for the broadband and 0.5\milli\volt for the narrowband signals. Top to bottom:  a) Minimum population size needed to have coding fraction be at least 95\% of the maximum as a function of noise. Optimal population size grows with increasing noise. Optimal population size is larger for the narrowband signals (dots) than for the broadband signal (crosses). For weak noise and narrowband signals, there is little difference in the optimal population size for the high frequency interval
	(brown) and the low frequency interval (blue). As noise becomes stronger, optimal population size 
	is larger for the higher interval. For the broadband signal, optimum population size is always largest for the higher frequency interval, then for the broadband signal and finally the low frequency narrowband signal. That minimum population size is larger for the narrowband signals than
	for the broadband signal can be explained by the fact that the maximum coding fraction (b) is higher for the narrowband signals. b) Maximum coding fraction is higher for lower intervals and narrower bands. In the case of the narrowband signal and the lwoer interval, maximum coding fraction is close to 1 for all noise strengths. For weak noise, the low interval in the
	broadband signal and the higher narrowband signal have very similar maximum coding fractions. With increasing noise, the maximum coding fraction rises much 
	faster for the high frequency narrowband signal. The broadband signal and the high frequency interval inside that signal have very low maximum coding fraction for weak noise. Increasing the strength of the noise at some point coding fraction begins to increase rapidly. c) Frequency band, not signal bandwidth appears to be the main factor in the relative improvement of coding fraction with increasing population size. For the slow narrowband signal the improvement is mostly not because the maximum becomes higher, as it changes very little. Instead the reason is that a single neuron has a diminished ability to encode the signal with increasing noise levels (see fig. \ref{CodingFrac} B and C). For the whole broadband signal and the low frequency interval within, for weak noise there is almost no improvement in coding fraction through increasing population size. Interestingly, with stronger noise the relative improvement for the low
	frequency interval is the same for both the narrowband and the broadband signal, even though maximum coding fraction is very different, at least for intermediate noise strength (\(10^{-4}\) to \(10^{-3}\)\milli\volt\squared\per\hertz).
		}
	\label{fig:popsizenarrow15}
\end{figure}

First, we consider the case of strong input (15 \milli\volt) which leads to a an average firing
rate of about 91\hertz. As before, we see great differences for the different frequency intervals. In figure \ref{fig:popsizenarrow15} A
we see population sizes necessary to reach an encoding quality close to the maximum. 
To encode the high frequency parts of the spectrum much larger populations are required
than for the low frequency parts. As expected, the 
size of the optimal population increases with increasing noise strength. To encode the broadband
signal over its entire spectrum, a population size between the population sizes 
for the narrowband intervals is optimal. This can
be understood because the broadband signal contains both the "easy" and the "difficult" intervals
and should therefore fall in between. The same is true for the maximum coding fraction (figure \ref{fig:popsizenarrow15} C):
Again, the broadband signal falls in between the intervals. As we could see before (figure \ref{smallbroad}), 
the lower frequency interval is easier to encode than the higher frequency interval
and the whole broadband signal. 
We also considered the relative effect of 
increasing population size. To quantify this, we divided the maximum coding fraction by 
the coding fraction of a single neuron. Figure \ref{fig:popsizenarrow15} E shows that 
the increase in coding fraction gained by increasing population size
is larger for the higher frequency interval. In contrast, a single neuron can 
encode the broadband signal or the low frequency interval about as well as a 
larger population can.
That relative improvement increases for stronger noise 
is a consequence of the reduction in the encoding 
capability of a single neuron (compare figure \ref{CodingFrac} C).

For the narrowband signals we see a similar picture: Except for very weak noise, 
optimal population size to encode the high frequency signal is larger than that
for the low frequency signal (figure \ref{fig:popsizenarrow15} B). Optimal
population size for encoding of the low frequency signal now starts at a much 
higher level than before. 
Maximum coding fraction
again is larger for the low frequency signal (figure \ref{fig:popsizenarrow15} D). 
For the high frequency signal, coding fraction
is at a much higher level than it is for the high frequency interval in the broadband signal
for all noise strengths.
With increasing noise strength the coding fraction increases rapidly and almost reaches the level achieved
for the low frequency signal for the population sizes considered here.  
The relative increase in coding fraction
(figure \ref{fig:popsizenarrow15} F) is similar to what
we saw for the broadband signal. For high frequencies the increase in
coding fraction from a single neuron to a population of neurons becomes apparent
even at a relatively low noise strength. Whereas for the low frequency
signal the relative improvement starts at much higher noise levels and is
mostly explained by the decrease in encoding capabilities of a single neuron.

For both narrowband and broadband signals, the results here qualitatively do not depend 
on the input amplitude. See \textit{appendix} for more information.

\subsection*{Weak average input (low firing rates)}

\begin{figure}

	\includegraphics[width=0.49\linewidth]{{img/0to50_broad_small_coherence_10.5_0.5}.pdf}

	\caption{Coherence curves for broad and narrow frequency range inputs. The average firing rate of the cells is marked with a black vertical line. a) Broad spectrum. Coherence for low frequencies is much lower than for the case of higher mean input (fig. \ref{fig:coherence_narrow_15.0} A). For the weak noise level (blue), population sizes n=4 and n=4096 show indistinguishable coding fraction. In case of a small population size, coherence is higher for stronger noise (green), contrary to what we have seen before. This can not be explained by an increase in firing rate, as the difference in firing rate is only about 1\%.  b) Narrow band inputs for two frequency ranges. Low frequency range: coherence for slow parts of the signal is similar to those in the broadband signal. High frequency range: In contrast to the case of higher mean input (fig. \ref{fig:coherence_narrow_15.0}) the high-frequency signal is encoded better than the low-frequency signal. Even for comparatively weak noise (blue), an increase in population size offers better encoding.
}
	\label{fig:coherence_narrow_10.5}
\end{figure}

We also consider the case of a weaker mean input (10.5\milli\volt) which leads
to lower average firing rates (about 34\hertz). Results can be seen in figure
\ref{fig:popsizenarrow10}. For the broadband signal, in general 
results are similar to the results in the strong average input case. 
We again see (fig. \ref{fig:popsizenarrow10} A) that optimum population size is
larger for the high frequency interval than for the low frequency interval, with
the entire broadband signal somewhere in between. Now the optimal population sizes
are much closer than in the case of high average firing rates. The values being
closer together is also true for the maximum coding fraction 
(fig. \ref{fig:popsizenarrow10} C). The value for the low frequency interval
  is now much lower for very weak noise than for the high mean input.
The curves for both intervals and the broadband signal now look very similar
to each other.
Maximum coding fraction for the broadband signal and the high
frequency interval are almost equal for noise strengths greater
than \(10^{-3}\milli\volt\squared\per\hertz\).
Relative improvement (fig. \ref{fig:popsizenarrow10} E) is very similar for all
intervals. For increasing noise strength relative improvement starts to show
the pattern we have seen before, with improvement being greatest for
the high frequency interval, followed by the broadband signal and then the
low frequency interval. 

For the narrowband signals we see some striking differences to what we saw
for the high mean input. Optimal population sizes are now very different for the low frequency
signal and the high frequency signal for all noise strengths (fig. \ref{fig:popsizenarrow10} B).
The high frequency interval signal again needs a larger population for
encoding being close to optimal. The optimal population size for
the low frequency signal is now very similar to the optimal population
size for the low frequency interval in the broadband signal. 
This contrasts with the high input case (fig. \ref{fig:popsizenarrow15} A \& B). For
weak noise the optimal population size was much larger for the narrowband signal 
than for the interval in the broadband signal.

A striking change happens for the maximum coding fraction (fig. \ref{fig:popsizenarrow10} D).
As opposed to every case we have looked at before, now the higher frequency
signal appears to be more easily encoded than the low frequency signal.
This can be explained by looking at the coherence curves of the two signals 
(fig. \ref{fig:coherence_narrow_10.5} B). As the firing rate of the neurons
is inside the frequency range of the low frequency signal, encoding
of those frequencies is suppressed. This is not the case for the
high frequency signal, where the coherence behaves similarly to
what we have seen for the strong input case (fig. \ref{fig:coherence_narrow_15.0} B).
Maximum coding fraction for the low frequency signal is very similar to
the maximum coding fraction for the low frequency interval of the broadband signal.
This is also different to what we have seen before, as for every other case 
coding fraction can get much higher for the narrowband signals. 

Relative improvement is higher for the high frequency signal 
than for the low frequency signal (fig. \ref{fig:popsizenarrow10} F).
For the high frequency signal,  the improvement is greater
than for the equivalent interval in the broadband signal for weak noise. 
Relative improvement is also greater
than it was for high average input (fig. \ref{fig:popsizenarrow15} E \& F).		
Here, even for weak noise increasing the population size has a large effect.
In other words, to encode the high frequency signal in the case of a 
low average firing rate, population size is very critical to the quality
of the encoding. For the low frequency signal there is no large difference
in relative improvement to the other cases.