Initial commit/First pass over introduction

.gitignore

@@ -0,0 +1,7 @@
+*.log
+*.bbl
+*.blg
+*.aux
+*.dvi
+*.toc
+*swp

bands.tex

@@ -0,0 +1,126 @@
+
+\subsection*{Narrowband stimuli}
+Using the \(f_{cutoff} = 200 \hertz\usk\) signal, we repeated the analysis for only a part of the spectrum. We did so for two "low frequency" (0--8Hz, 0--50Hz) and two "high frequency" (192--200Hz, 150--200Hz) intervals. We then compared the results to the results we get from narrowband stimuli, with power only in those frequency bands.
+To keep the power of the signal inside the two intervals the same as in the broadband stimulus, amplitude of the narrowband signals was less than that of the broadband signal. For the 8Hz intervals, amplitude (i.e. standard deviation) of the signal was 0.2mV, or a fifth of the amplitude of the broadband signal. Because signal power is proportional to the square of the amplitude, this was appropriate for a stimulus with a spectrum 25 times smaller. Similarly, for the 50Hz intervals we used a 0.5mV amplitude, or half of that of the broadband stimulus.
+As the square of the amplitude is equal to the integral over the frequency spectrum, for a signal with a quarter of the width we need to half the amplitude to have the same power in the interval defined by the narrowband signals.
+
+\subsection*{Smaller frequency intervals in broadband signals }
+
+\begin{figure}
+\includegraphics[width=0.45\linewidth]{img/small_in_broad_spectrum}
+\includegraphics[width=0.45\linewidth]{img/power_spectrum_0_50}
+	\includegraphics[width=0.49\linewidth]{{img/broad_coherence_15.0_1.0}.pdf}
+	\includegraphics[width=0.49\linewidth]{{img/coherence_15.0_0.5_narrow_both}.pdf}
+	\includegraphics[width=0.49\linewidth]{{img/broad_coherence_10.5_1.0_200}.pdf}
+	\includegraphics[width=0.49\linewidth]{{img/coherence_10.5_0.5_narrow_both}.pdf}
+	\caption{Coherence for broad and narrow frequency range inputs. a) Broad spectrum. 
+	At the frequency of the firing rate (91Hz, marked by the black bar) and its first 
+	harmonic (182Hz) the coding fraction breaks down. For the weak noise level (blue), 
+	population sizes n=4 and n=4096 show indistinguishable coding fraction. 
+	In case of a small population size, coherence is higher for weak noise (blue) than 
+	for strong noise (green) in  the frequency range up to about 50\hertz. For higher
+	frequencies coherence is unchanged. For the case of the larger population size and the 
+	greater noise strength there is a huge increase in the coherence for all frequencies. 
+	b) Coherence for two narrowband inputs with different frequency ranges. 
+	Low frequency range: coherence for 
+	slow parts of the signal is close to 1 for weak noise. SSR works mostly on the 
+	higher frequencies (here >40\hertz). High frequency range: At 182Hz (first harmonic 
+	of the firing frequency) there is a very sharp decrease in coding fraction, 
+	especially for the weak noise condition (blue). Increasing the noise makes the drop 
+	less clear. For weak noise (blue) there is another break down at 182-(200-182)Hz. 
+	Stronger noise seems to make this sharp drop disappear. Again, the effect of SSR 
+	is most noticeable for the higher frequencies in the interval.}
+	\label{fig:coherence_narrow_15.0}
+\end{figure}
+
+We want to know how  good encoding works for different frequency intervals in the signal.
+When we take out a narrower frequency interval from a broadband signal, the other 
+frequencies in the signal serve as common noise to the neurons encoding the signal.
+In many cases we only care about a certain frequency band in a signal of much wider bandwidth.
+In figure \ref{fig:coherence_narrow_15.0} A we can see that SSR has very different
+effects on certain frequencies inside the signal than on others. In blue we see the
+case of very weak noise (\(10^{-6} \milli\volt\squared\per\hertz\)). Increasing the
+population size from 4 neurons to 2048 neurons has practically no effect. Around
+the average firing rate of the neurons, coherence becomes almost zero.  When
+we keep population size at 4 neurons, but 
+add more noise to the neurons (green, \(2\cdot10^{-3} \milli\volt\squared\per\hertz\)),
+encoding of the low frequencies (up to about 50\hertz) becomes worse, while
+encoding of the higher frequencies stays unchanged. When we increase the population
+size to 2048 neurons we have almost perfect encoding for frequencies up to 50\hertz.
+Coherence is still reduced around the average firing rate of the neurons, but at
+a much higher level than before. For higher frequencies coherence becomes higher again.
+In summary, the high frequency bands inside the broadband stimulus experience a 
+much greater increase in encoding quality than the low frequency bands,
+which were already encoded quite well. 
+ 
+
+\begin{figure}
+\includegraphics[width=0.45\linewidth]{img/broadband_optimum_newcolor.pdf}
+\includegraphics[width=0.45\linewidth]{img/smallband_optimum_newcolor.pdf}
+\centering
+\includegraphics[width=0.9\linewidth]{img/max_cf_smallbroad.pdf}
+\caption{
+ A: Input signal spectrum of a broadband signal. The colored area marks the frequency ranges considered here. 
+ B: Two narrowband signals (red and blue). The broadband signal from A (grey) is shown again for comparison.
+ C and D: Best amount of noise for different number of neurons. The dashed lines show where coding fraction still is at least 95\% from the maximum. The width of the peaks is much larger for the narrowband signals which encompasses the entire width of the high-frequency interval peak. 
+Optimum noise values for a fixed number of neurons are always higher for the broadband signal than for narrowband signals. 
+In the broadband case, the optimum amount of noise is larger for the high-frequency interval than for the low-frequency interval and vice-versa for the narrowband case. %The optimal noise values have been fitted with a function of square root of the population size N, $f(N)=a+b\sqrt{N}$. We observe that the optimal noise value grows with the square root of population size.
+E and F: Coding fraction as a function of noise for a fixed population size (N=512). Red dots show the maximum, the red line where coding fraction is at least 95\% of the maximum value.
+G: An increase in population size leads to a higher coding fraction especially for broader bands and higher frequency intervals. Coding fraction is 
+larger for the narrowband signal than in the equivalent broadband interval for all neural population sizes considered here. The coding fraction for the low frequency intervals is always larger than for the high frequency interval.
+Signal mean $\mu=15.0\milli\volt$, signal amplitude $\sigma=1.0\milli\volt$ and $\sigma=0.5\milli\volt$ respectively.}
+\label{smallbroad}
+\end{figure}
+
+\subsection*{Narrowband Signals vs Broadband Signals}
+
+
+
+
+In nature, often an external stimulus covers a frequency range 
+that starts at high frequencies, so that only using broadband white noise signals 
+as input appears to be insufficient to describe realistic scenarios.
+%, with bird songs\citep{nottebohm1972neural} and ???\footnote{chirps, in a way?}. 
+%We see that in many animals receptor neurons have adapted to these signals. For example, it was found that electroreceptors in weakly electric fish have band-pass properties\citep{bastian1976frequency}. 
+Therefore, we investigate the coding of narrowband signals in the ranges 
+described earlier (0--50Hz, 150--200Hz). Comparing the results from coding of 
+broadband and coding of narrowband signals, we see several differences. 
+
+For both low and high frequency signals, the narrowband signal 
+can be resolved better than the broadband signal for any amount of noise and (figure \ref{smallbroad}, bottom left).
+That coding fractions are higher when we use narrowband signals can be
+explained by the fact that the additional 
+frequencies in the broadband signal are now absent. In the broadband signal
+they are a form of "noise" that is common to all the input neurons.
+Similar to what we saw for the broadband signal, 
+the peak of the low frequency input is still much more broad than the peak of the high frequency input.
+To encode low frequency signals the exact strength of the noise is not as 
+important as it is for the high frequency signals which can be seen from the wider peaks.
+
+\subsection{Discussion}
+The usefulness of noise on information encoding of subthreshold signals by single neurons has been well investigated. However, the encoding of supra-threshold signals by populations of neurons has received comparatively little attention and different effects play a role for suprathreshold signals than for subthreshold signals \citep{Stocks2001}. This paper delivers an important contribution for the understanding of suprathreshold stochastic resonance (SSR). We simulate populations of leaky integrate-and-fire neurons to answer the question how population size influences the optimal noise strength  for linear encoding of suprathreshold signals. We are able to show that this optimal noise is well described as a function of the square root of population size. This relationship is independent of frequency properties of the input signal and holds true for narrowband and broadband signals.
+
+In this paper, we show that SSR works in LIF-neurons for a variety of signals of different bandwidth and frequency intervals. We show that signal-to-noise ratio is for signals above a certain strength sufficient to describe the optimal noise strength in the population, but that the actual coding fraction depends on the absolute value of signal strength. %We furthermore show that increasing signal strength does not always increase the coding fraction.
+
+We contrast how well the low and high frequency parts of a broadband signal can be encoded. We take an input signal with $f_{cutoff} = \unit{200}\hertz$ and analyse the coding fraction for the frequency ranges 0 to \unit{50}\hertz\usk and 150 to \unit{200}\hertz\usk separately. The maximum value of the coding fraction is lower for the high frequency interval compared to the low frequency interval. This means that inside broadband signals higher frequencies intervals appear more difficult to encode for each level of noise and population size. The low frequency interval has a wider peak (defined as 95\% coding fraction of its coding fraction maximum value), which means around the optimal amount of noise there is a large area where coding fraction is still good. The noise optimum for the low frequency parts of the input is lower than the optimum for the high frequency interval (Fig. \ref{highlowcoherence}). In both cases, the optimal noise value appears to grow with the square root of population size.
+
+In general, narrowband signals can be encoded better than broadband signals.
+narrowband vs broadband
+
+Another main finding of this paper is the discovery of frequency dependence of SSR.
+We can see from the shape of the coherence between the signal and the output of the simulated 
+neurons, SSR works mostly for the higher frequencies in the signal. As the lower frequency
+components are in many cases already encoded really well, the addition of noise
+helps to flatten the shape of the coherence curve. In the case of weak noise, often there
+are border effects which disappear with increasing strength of the noise.
+In addition, for weak noise there are often visible effects from the firing rate of the neurons, in so far that the encoding
+around those frequencies is worse than for the surrounding frequencies. Generally
+this effect becomes less pronounced when we add more noise to the simulation, but
+we found a very striking exception in the case of narrowband signals. 
+Whereas for a firing rate of about 
+91\hertz\usk  the coding fraction of the encoding of a signal in the 0-50\hertz\usk band is 
+better than for the encoding of a signal in the 150-200\hertz\usk band. However, this is 
+not the case if the neurons have a firing rate about 34\hertz. 
+We were thus able to show that the firing rate on the neurons in the simulation is of 
+critical importance to the encoding of the signal.
+

calculation.tex

@@ -0,0 +1,46 @@
+\section*{Limit case of large populations}
+
+\subsection*{For large population sizes and strong noise, coding fraction becomes a function of their quotient}
+
+For the linear response regime of large noise, we can estimate the coding fraction. From Beiran et al. 2018 we know the coherence in linear response is given as
+
+\eq{
+C_N(\omega) = \frac{N|\chi(\omega)|^2 S_{ss}}{S_{x_ix_i}(\omega)+(N1)|\chi(\omega)|^2S_{ss}}
+\label{eq:linear_response}
+}
+
+where \(C_1(\omega)\) is the coherence function for a single LIF neuron. Generally, the single-neuron coherence is given by \citep{??}
+
+\eq{
+C_1(\omega)=\frac{r_0}{D} \frac{\omega^2S_{ss}(\omega)}{1+\omega^2}\frac{\left|\mathcal{D}_{i\omega-1}\big(\frac{\mu-v_T}{\sqrt{D}}\big)-e^{\Delta}\mathcal{D}_{i\omega-1}\big(\frac{\mu-v_R}{\sqrt{D}}\big)\right|^2}{\left|{\cal D}_{i\omega}(\frac{\mu-v_T}{\sqrt{D}})\right|^2-e^{2\Delta}\left|{\cal D}_{i\omega}(\frac{\mu-v_R}{\sqrt{D}})\right|^2}
+\label{eq:single_coherence}
+}
+
+where \(r_0\) is the firing rate of the neuron,
+\[r_0 = \left(\tau_{ref} + \sqrt{\pi}\int_\frac{\mu-v_r}{\sqrt{2D}}^\frac{\mu-v_t}{\sqrt{2D}} dz e^{z^2} \erfc(z) \right)^{-1}\]. 
+In the limit of large noise (calculation in the appendix) this equation evaluates to:
+
+\eq{
+C_1(\omega) = \sqrt{\pi}D^{-1}
+\frac{S_{ss}(\omega)\omega^2/(1+\omega^2)}{2 \sinh\left(\frac{\omega\pi}{2}\right)\Im\left( \Gamma\left(1+\frac{i\omega}{2}\right)\Gamma\left(\frac12-\frac{i\omega}{2}\right)\right)}
+\label{eq:simplified_single_coherence}
+}
+
+From eqs.\ref{eq:linear_response} and \ref{eq:simplified_single_coherence} it follows that in the case \(D \rightarrow \infty\) the coherence, and therefore the coding fraction, of the population of LIF neurons is a function of \(D^{-1}N\). We plot the approximation as a function of \(\omega\) (fig. \ref{d_n_ratio}). In the limit of small frequencies the approximation matches the exact equation very well, though not for higher frequencies. We can verify this in our simulations by plotting coding fraction as a function of \(\frac{D}{N}\). We see (fig. \ref{d_n_ratio}) that in the limit of large D, the curves actually lie on top of each other. This is however not the case (fig. \ref{d_n_ratio}) for stimuli with a large cutoff frequency \(f_c\), as expected by our evaluation of the approximation as a function of the frequency.
+
+
+\begin{figure}
+\centering 
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_10.5_0.5_10_detail}.pdf}
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_15.0_0.5_50_detail}.pdf}
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_15.0_1.0_200_detail}.pdf}
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_10.5_0.5_10_detail}.pdf}
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_15.0_0.5_50_detail}.pdf}
+ \includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_15.0_1.0_200_detail}.pdf}
+ \label{d_n_ratio}
+ \caption{Top row: Coding fraction as a function of noise.
+ Bottom row: Coding fraction as a function of the ratio between noise strength and population size. For strong noise, coding fraction is a function of this ratio. 
+ Left: signal mean 10.5mV, signal amplitude 0.5mV, $f_{c}$ 10Hz.
+ Middle: signal mean 15.0mV, signal amplitude 0.5mV, $f_{c}$ 50Hz.
+ Right: signal mean 15.0mV, signal amplitude 1.0mV, $f_{c}$ 200Hz.}
+\end{figure}

citations.bib

@@ -0,0 +1,724 @@
+@article{goldberg1984relation,
+  title={Relation between discharge regularity and responses to externally applied galvanic currents in vestibular nerve afferents of the squirrel monkey},
+  author={Goldberg, JM and Smith, CHARLES E and Fernandez, C},
+  journal={Journal of neurophysiology},
+  volume={51},
+  number={6},
+  pages={1236--1256},
+  year={1984},
+  publisher={American Physiological Society Bethesda, MD}
+}
+@article{benzi1981mechanism,
+  title={The mechanism of stochastic resonance},
+  author={Benzi, Roberto and Sutera, Alfonso and Vulpiani, Angelo},
+  journal={Journal of Physics A: mathematical and general},
+  volume={14},
+  number={11},
+  pages={L453},
+  year={1981},
+  publisher={iOP Publishing}
+}
+@article{palanca2015vivo,
+author = {Palanca-Castan, Nicolas and K{\"{o}}ppl, Christine},
+journal = {Brain, behavior and evolution},
+number = {4},
+pages = {271--286},
+publisher = {Karger Publishers},
+title = {{In vivo recordings from low-frequency nucleus laminaris in the barn owl}},
+volume = {85},
+year = {2015}
+}
+@article{Chapeau-blondeau1996,
+author = {Chapeau-blondeau, Franqois and Godivier, Xavier and Chambet, Nicolas},
+file = {:home/huben/Desktop/PhysRevE.53.1273.pdf:pdf},
+number = {1},
+pages = {3--5},
+title = {s(t)=+„5(t},
+volume = {53},
+year = {1996}
+}
+@article{walz2014static,
+author = {Walz, Henriette and Grewe, Jan and Benda, Jan},
+journal = {Journal of Neurophysiology},
+number = {4},
+pages = {752--765},
+publisher = {Am Physiological Soc},
+title = {{Static frequency tuning accounts for changes in neural synchrony evoked by transient communication signals}},
+volume = {112},
+year = {2014}
+}
+@article{Chapeau-blondeau1996,
+author = {Chapeau-blondeau, Franqois and Godivier, Xavier and Chambet, Nicolas},
+file = {:home/huben/Desktop/PhysRevE.53.1273.pdf:pdf},
+number = {1},
+pages = {3--5},
+title = {s(t)=+„5(t},
+volume = {53},
+year = {1996}
+}
+@article{lindner2016mechanisms,
+author = {Lindner, Benjamin},
+journal = {IEEE Transactions on Molecular, Biological and Multi-Scale Communications},
+publisher = {IEEE},
+title = {{Mechanisms of information filtering in neural systems}},
+year = {2016}
+}
+@article{krahe2002stimulus,
+author = {Krahe, R{\"{u}}diger and Kreiman, Gabriel and Gabbiani, Fabrizio and Koch, Christof and Metzner, Walter},
+journal = {Journal of Neuroscience},
+number = {6},
+pages = {2374--2382},
+publisher = {Soc Neuroscience},
+title = {{Stimulus encoding and feature extraction by multiple sensory neurons}},
+volume = {22},
+year = {2002}
+}
+@article{ahn2014heterogeneity,
+author = {Ahn, Jheeyae and Kreeger, Lauren J and Lubejko, Susan T and Butts, Daniel A and MacLeod, Katrina M},
+journal = {Journal of neurophysiology},
+number = {11},
+pages = {2320--2331},
+publisher = {Am Physiological Soc},
+title = {{Heterogeneity of intrinsic biophysical properties among cochlear nucleus neurons improves the population coding of temporal information}},
+volume = {111},
+year = {2014}
+}
+@article{Krahe2008,
+abstract = {Multiple topographic representations of sensory space are common in the nervous system and presumably allow organisms to separately process particular features of incoming sensory stimuli that vary widely in their attributes. We compared the response properties of sensory neurons within three maps of the body surface that are arranged strictly in parallel to two classes of stimuli that mimic prey and conspecifics, respectively. We used information-theoretic approaches and measures of phase locking to quantify neuronal responses. Our results show that frequency tuning in one of the three maps does not depend on stimulus class. This map acts as a low-pass filter under both conditions. A previously described stimulus-class-dependent switch in frequency tuning is shown to occur in the other two maps. Only a fraction of the information encoded by all neurons could be recovered through a linear decoder. Particularly striking were low-pass neurons the information of which in the high-frequency range could not be decoded linearly. We then explored whether intrinsic cellular mechanisms could partially account for the differences in frequency tuning across maps. Injection of a Ca2+ chelator had no effect in the map with low-pass characteristics. However, injection of the same Ca2+ chelator in the other two maps switched the tuning of neurons from band-pass/high-pass to low-pass. These results show that Ca2+-dependent processes play an important part in determining the functional roles of different sensory maps and thus shed light on the evolution of this important feature of the vertebrate brain.},
+author = {Krahe, R{\"{u}}diger and Bastian, Joseph and Chacron, Maurice J},
+doi = {10.1152/jn.90300.2008},
+issn = {0022-3077},
+journal = {Journal of Neurophysiology},
+number = {2},
+pages = {852--867},
+publisher = {American Physiological Society},
+title = {{Temporal Processing Across Multiple Topographic Maps in the Electrosensory System}},
+url = {http://jn.physiology.org/content/100/2/852},
+volume = {100},
+year = {2008}
+}
+@article{walz2014static,
+author = {Walz, Henriette and Grewe, Jan and Benda, Jan},
+file = {:home/huben/Documents/Paper/Walz2014.pdf:pdf},
+journal = {Journal of Neurophysiology},
+number = {4},
+pages = {752--765},
+publisher = {Am Physiological Soc},
+title = {{Static frequency tuning accounts for changes in neural synchrony evoked by transient communication signals}},
+volume = {112},
+year = {2014}
+}
+@article{Walz2014,
+author = {Walz, Henriette and Grewe, Jan and Benda, Jan},
+doi = {10.1152/jn.00576.2013},
+file = {:home/huben/Documents/Paper/Walz2014.pdf:pdf},
+isbn = {2154253113},
+pages = {752--765},
+title = {{Static frequency tuning accounts for changes in neural synchrony evoked by transient communication signals}},
+year = {2014}
+}
+@article{white2000channel,
+author = {White, John A and Rubinstein, Jay T and Kay, Alan R},
+journal = {Trends in Neurosciences},
+number = {3},
+pages = {131--137},
+publisher = {Elsevier},
+title = {{Channel noise in neurons}},
+volume = {23},
+year = {2000}
+}
+@article{faisal2008noise,
+author = {Faisal, A Aldo and Selen, Luc P J and Wolpert, Daniel M},
+file = {:home/huben/Documents/Paper/Faisal2008.pdf:pdf},
+journal = {Nature Reviews Neuroscience},
+number = {4},
+pages = {292--303},
+publisher = {Nature Publishing Group},
+title = {{Noise in the nervous system}},
+volume = {9},
+year = {2008}
+}
+@article{fourcaud2003spike,
+author = {Fourcaud-Trocm{\'{e}}, Nicolas and Hansel, David and {Van Vreeswijk}, Carl and Brunel, Nicolas},
+journal = {The Journal of Nneuroscience},
+number = {37},
+pages = {11628--11640},
+publisher = {Soc Neuroscience},
+title = {{How spike generation mechanisms determine the neuronal response to fluctuating inputs}},
+volume = {23},
+year = {2003}
+}
+@article{krahe2004burst,
+author = {Krahe, R{\"{u}}diger and Gabbiani, Fabrizio},
+journal = {Nature Reviews Neuroscience},
+number = {1},
+pages = {13--23},
+publisher = {Nature Publishing Group},
+title = {{Burst firing in sensory systems}},
+volume = {5},
+year = {2004}
+}
+@article{grewe2017synchronous,
+  title={Synchronous spikes are necessary but not sufficient for a synchrony code in populations of spiking neurons},
+  author={Grewe, Jan and Kruscha, Alexandra and Lindner, Benjamin and Benda, Jan},
+  journal={Proceedings of the National Academy of Sciences},
+  volume={114},
+  number={10},
+  pages={E1977--E1985},
+  year={2017},
+  publisher={National Acad Sciences}
+}
+@article{schreiber2002energy,
+author = {Schreiber, Susanne and Machens, Christian K and Herz, Andreas V M and Laughlin, Simon B},
+journal = {Neural Computation},
+number = {6},
+pages = {1323--1346},
+publisher = {MIT Press},
+title = {{Energy-efficient coding with discrete stochastic events}},
+volume = {14},
+year = {2002}
+}
+@article{neiman2011temporal,
+abstract = {The manner in which information is encoded in neural signals is a major issue in Neuroscience. A common distinction is between rate codes, where information in neural responses is encoded as the number of spikes within a specified time frame (encoding window), and temporal codes, where the position of spikes within the encoding window carries some or all of the information about the stimulus. One test for the existence of a temporal code in neural responses is to add artificial time jitter to each spike in the response, and then assess whether or not information in the response has been degraded. If so, temporal encoding might be inferred, on the assumption that the jitter is small enough to alter the position, but not the number, of spikes within the encoding window. Here, the effects of artificial jitter on various spike train and information metrics were derived analytically, and this theory was validated using data from afferent neurons of the turtle vestibular and paddlefish electrosensory systems, and from model neurons. We demonstrate that the jitter procedure will degrade information content even when coding is known to be entirely by rate. For this and additional reasons, we conclude that the jitter procedure by itself is not sufficient to establish the presence of a temporal code.},
+author = {Neiman, Alexander B and Russell, David F and Rowe, Michael H},
+doi = {10.1371/journal.pone.0027380},
+journal = {PLOS ONE},
+number = {11},
+pages = {1--13},
+publisher = {Public Library of Science},
+title = {{Identifying Temporal Codes in Spontaneously Active Sensory Neurons}},
+url = {http://dx.doi.org/10.1371{\%}2Fjournal.pone.0027380},
+volume = {6},
+year = {2011}
+}
+@article{gabbiani1996coding,
+author = {Gabbiani, Fabrizio},
+journal = {Network: Computation in Neural Systems},
+number = {1},
+pages = {61--85},
+publisher = {Citeseer},
+title = {{Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons}},
+volume = {7},
+year = {1996}
+}
+@article{gammaitoni1998resonance,
+author = {Gammaitoni, Luca and H{\"{a}}nggi, Peter and Jung, Peter and Marchesoni, Fabio},
+doi = {10.1103/RevModPhys.70.223},
+journal = {Rev. Mod. Phys.},
+number = {1},
+pages = {223--287},
+publisher = {American Physical Society},
+title = {{Stochastic resonance}},
+url = {http://link.aps.org/doi/10.1103/RevModPhys.70.223},
+volume = {70},
+year = {1998}
+}
+@article{davtyan2016protein,
+author = {Davtyan, Aram and Platkov, Max and Gruebele, Martin and Papoian, Garegin A},
+doi = {10.1002/cphc.201501125},
+issn = {1439-7641},
+journal = {ChemPhysChem},
+keywords = {F{\"{o}}rster resonance energy transfer,brownian dynamics,molecular dynamics,protein folding,stochastic resonance},
+number = {9},
+pages = {1305--1313},
+title = {{Stochastic Resonance in Protein Folding Dynamics}},
+url = {http://dx.doi.org/10.1002/cphc.201501125},
+volume = {17},
+year = {2016}
+}
+@article{VanderGroen2016,
+abstract = {Random noise enhances the detectability of weak signals in nonlinear systems, a phenomenon known as stochastic resonance (SR). Though counterintuitive at first, SR has been demonstrated in a variety of naturally occurring processes, including human perception, where it has been shown that adding noise directly to weak visual, tactile, or auditory stimuli enhances detection performance. These results indicate that random noise can push subthreshold receptor potentials across the transfer threshold, causing action potentials in an otherwise silent afference. Despite the wealth of evidence demonstrating SR for noise added to a stimulus, relatively few studies have explored whether or not noise added directly to cortical networks enhances sensory detection. Here we administered transcranial random noise stimulation (tRNS; 100–640 Hz zero-mean Gaussian white noise) to the occipital region of human participants. For increasing tRNS intensities (ranging from 0 to 1.5 mA), the detection accuracy of a visual stimuli changed according to an inverted-U-shaped function, typical of the SR phenomenon. When the optimal level of noise was added to visual cortex, detection performance improved significantly relative to a zero noise condition (9.7 ± 4.6{\%}) and to a similar extent as optimal noise added to the visual stimuli (11.2 ± 4.7{\%}). Our results demonstrate that adding noise to cortical networks can improve human behavior and that tRNS is an appropriate tool to exploit this mechanism.SIGNIFICANCE STATEMENT Our findings suggest that neural processing at the network level exhibits nonlinear system properties that are sensitive to the stochastic resonance phenomenon and highlight the usefulness of tRNS as a tool to modulate human behavior. Since tRNS can be applied to all cortical areas, exploiting the SR phenomenon is not restricted to the perceptual domain, but can be used for other functions that depend on nonlinear neural dynamics (e.g., decision making, task switching, response inhibition, and many other processes). This will open new avenues for using tRNS to investigate brain function and enhance the behavior of healthy individuals or patients.},
+author = {van der Groen, Onno and Wenderoth, Nicole},
+journal = {The Journal of Neuroscience},
+number = {19},
+pages = {5289 LP -- 5298},
+title = {{Transcranial Random Noise Stimulation of Visual Cortex: Stochastic Resonance Enhances Central Mechanisms of Perception}},
+url = {http://www.jneurosci.org/content/36/19/5289.abstract},
+volume = {36},
+year = {2016}
+}
+@article{shapira2016sound,
+author = {Shapira, Einat and Pujol, R{\'{e}}my and Plaksin, Michael and Kimmel, Eitan},
+journal = {Physics in Medicine},
+publisher = {Elsevier},
+title = {{Sound-induced motility of outer hair cells explained by stochastic resonance in nanometric sensors in the lateral wall}},
+year = {2016}
+}
+@article{mileva2016short,
+author = {Mileva, G R and Kozak, I J and Lewis, J E},
+file = {:home/huben/Desktop/1-s2.0-S0306452216000336-main.pdf:pdf},
+journal = {Neuroscience},
+pages = {1--11},
+publisher = {Elsevier},
+title = {{Short-term synaptic plasticity across topographic maps in the electrosensory system}},
+volume = {318},
+year = {2016}
+}
+@article{shimokawa1999stochastic,
+author = {Shimokawa, T and Rogel, A and Pakdaman, K and Sato, S},
+journal = {Physical Review E},
+number = {3},
+pages = {3461},
+publisher = {APS},
+title = {{Stochastic resonance and spike-timing precision in an ensemble of leaky integrate and fire neuron models}},
+volume = {59},
+year = {1999}
+}
+@incollection{benda2013neural,
+author = {Benda, Jan and Grewe, Jan and Krahe, R{\"{u}}diger},
+booktitle = {Animal Communication and Noise},
+pages = {331--372},
+publisher = {Springer},
+title = {{Neural noise in electrocommunication: from burden to benefits}},
+year = {2013}
+}
+@article{hupe2008effect,
+author = {Hup{\'{e}}, Ginette J and Lewis, John E and Benda, Jan},
+journal = {Journal of Physiology-Paris},
+number = {4},
+pages = {164--172},
+publisher = {Elsevier},
+title = {{The effect of difference frequency on electrocommunication: chirp production and encoding in a species of weakly electric fish, Apteronotus leptorhynchus}},
+volume = {102},
+year = {2008}
+}
+@article{collins1995stochastic,
+  title={Stochastic resonance without tuning},
+  author={Collins, JJ and Chow, Carson C and Imhoff, Thomas T},
+  journal={Nature},
+  volume={376},
+  number={6537},
+  pages={236},
+  year={1995},
+  publisher={Nature Publishing Group}
+}
+@article{collins1995aperiodic,
+  title={Aperiodic stochastic resonance in excitable systems},
+  author={Collins, JJ and Chow, Carson C and Imhoff, Thomas T},
+  journal={Physical Review E},
+  volume={52},
+  number={4},
+  pages={R3321},
+  year={1995},
+  publisher={APS}
+}
+
+@article{lindner2002maximizing,
+author = {Lindner, Benjamin and Schimansky-Geier, Lutz and Longtin, Andr{\'{e}}},
+file = {:home/huben/Desktop/PhysRevE.66.031916.pdf:pdf},
+journal = {Physical Review E},
+number = {3},
+pages = {31916},
+publisher = {APS},
+title = {{Maximizing spike train coherence or incoherence in the leaky integrate-and-fire model}},
+volume = {66},
+year = {2002}
+}
+@article{Chapeau-blondeau1996,
+author = {Chapeau-blondeau, Franqois and Godivier, Xavier and Chambet, Nicolas},
+file = {:home/huben/Desktop/PhysRevE.53.1273.pdf:pdf},
+number = {1},
+pages = {3--5},
+title = {s(t)=+„5(t},
+volume = {53},
+year = {1996}
+}
+@article{borst1999information,
+author = {Borst, Alexander and Theunissen, Fr{\'{e}}d{\'{e}}ric E},
+journal = {Nature Neuroscience},
+number = {11},
+pages = {947--957},
+publisher = {Nature Publishing Group},
+title = {{Information theory and neural coding}},
+volume = {2},
+year = {1999}
+}
+@article{krahe2014neural,
+author = {Krahe, R{\"{u}}diger and Maler, Leonard},
+file = {:home/huben/Downloads/1-s2.0-S0959438813001724-main.pdf:pdf},
+journal = {Current opinion in Neurobiology},
+pages = {13--21},
+publisher = {Elsevier},
+title = {{Neural maps in the electrosensory system of weakly electric fish}},
+volume = {24},
+year = {2014}
+}
+@article{stocks2000suprathreshold,
+author = {Stocks, N G},
+file = {:home/huben/Desktop/PhysRevLett.84.2310.pdf:pdf},
+journal = {Physical Review Letters},
+number = {11},
+pages = {2310},
+publisher = {APS},
+title = {{Suprathreshold stochastic resonance in multilevel threshold systems}},
+volume = {84},
+year = {2000}
+}
+@article{stocks2001information,
+  title={Information transmission in parallel threshold arrays: Suprathreshold stochastic resonance},
+  author={Stocks, NG},
+  journal={Physical Review E},
+  volume={63},
+  number={4},
+  pages={041114},
+  year={2001},
+  publisher={APS}
+}
+@article{stocks2002application,
+  title={The application of suprathreshold stochastic resonance to cochlear implant coding},
+  author={Stocks, NG and Allingham, D and Morse, RP},
+  journal={Fluctuation and noise letters},
+  volume={2},
+  number={03},
+  pages={L169--L181},
+  year={2002},
+  publisher={World Scientific}
+}
+
+@article{beiran2018coding,
+  title={Coding of time-dependent stimuli in homogeneous and heterogeneous neural populations},
+  author={Beiran, Manuel and Kruscha, Alexandra and Benda, Jan and Lindner, Benjamin},
+  journal={Journal of computational neuroscience},
+  volume={44},
+  number={2},
+  pages={189--202},
+  year={2018},
+  publisher={Springer}
+}
+
+@article{stocks2001generic,
+  title={Generic noise-enhanced coding in neuronal arrays},
+  author={Stocks, NG and Mannella, Riccardo},
+  journal={Physical Review E},
+  volume={64},
+  number={3},
+  pages={030902},
+  year={2001},
+  publisher={APS}
+}
+@article{nottebohm1972neural,
+  title={Neural lateralization of vocal control in a passerine bird. II. Subsong, calls, and a theory of vocal learning},
+  author={Nottebohm, Fernando},
+  journal={Journal of Experimental Zoology},
+  volume={179},
+  number={1},
+  pages={35--49},
+  year={1972},
+  publisher={Wiley Online Library}
+}
+@article{bastian1976frequency,
+  title={Frequency response characteristics of electroreceptors in weakly electric fish (Gymnotoidei) with a pulse discharge},
+  author={Bastian, Joseph},
+  journal={Journal of Comparative Physiology},
+  volume={112},
+  number={2},
+  pages={165--180},
+  year={1976},
+  publisher={Springer}
+}
+
+@article{gabbiani1996stimulus,
+author = {Gabbiani, Fabrizio and Metzner, Walter and Wessel, Ralf and Koch, Christof and Others},
+journal = {Nature},
+number = {6609},
+pages = {564--567},
+title = {{From stimulus encoding to feature extraction in weakly electric fish}},
+volume = {384},
+year = {1996}
+}
+@article{douglass1993noise,
+  title={Noise enhancement of information transfer in crayfish mechanoreceptors by stochastic resonance},
+  author={Douglass, John K and Wilkens, Lon and Pantazelou, Eleni and Moss, Frank},
+  journal={Nature},
+  volume={365},
+  number={6444},
+  pages={337},
+  year={1993},
+  publisher={Nature Publishing Group}
+}
+@article{levin1996broadband,
+  title={Broadband neural encoding in the cricket cereal sensory system enhanced by stochastic resonance},
+  author={Levin, Jacob E and Miller, John P},
+  journal={Nature},
+  volume={380},
+  number={6570},
+  pages={165},
+  year={1996},
+  publisher={Nature Publishing Group}
+}
+
+@article{gabbiani1996codingLIF,
+  title={Coding of time-varying signals in spike trains of integrate-and-fire neurons with random threshold},
+  author={Gabbiani, Fabrizio and Koch, Christof},
+  journal={Neural Computation},
+  volume={8},
+  number={1},
+  pages={44--66},
+  year={1996},
+  publisher={MIT Press}
+}
+@article{Gjorgjieva2014,
+author = {Gjorgjieva, Julijana and Sompolinsky, Haim and Meister, Markus},
+doi = {10.1523/JNEUROSCI.1032-14.2014},
+file = {:home/huben/Downloads/12127.full.pdf:pdf},
+journal = {Journal of Neuroscience,},
+keywords = {efficient coding,off,on,optimality,parallel pathways,retina,sensory processing},
+number = {36},
+pages = {12127--12144},
+title = {{Benefits of Pathway Splitting in Sensory Coding}},
+volume = {34},
+year = {2014}
+}
+@article{Maler,
+author = {Maler, Leonard},
+doi = {10.1016/j.conb.2013.08.013},
+file = {:home/huben/Downloads/1-s2.0-S0959438813001724-main.pdf:pdf},
+title = {{Neural maps in the electrosensory system of weakly electric fish ¨}},
+year = {2014}
+}
+@article{Huang2016,
+author = {Huang, Chengjie G. and Chacron, Maurice J.},
+doi = {10.1523/JNEUROSCI.1433-16.2016},
+file = {:home/huben/Downloads/parallelcodingchacron2016.pdf:pdf},
+keywords = {adaptation,electrosensory,envelope,features,have no or significant,in the same sensory,influence on their,neural coding,neuron type can either,off-type response to first-order,responses to second-order stimulus,significance statement,sk channels,stimulus attributes has no,we demonstrate that heterogeneities,weakly electric fish,while an on- or},
+number = {38},
+pages = {9859--9872},
+title = {{Optimized Parallel Coding of Second-Order Stimulus Features by Heterogeneous Neural Populations}},
+volume = {36},
+year = {2016}
+}
+@article{stocks2001suprathreshold,
+  title={Suprathreshold stochastic resonance: an exact result for uniformly distributed signal and noise},
+  author={Stocks, NG},
+  journal={Physics Letters A},
+  volume={279},
+  number={5-6},
+  pages={308--312},
+  year={2001},
+  publisher={Elsevier}
+}
+@article{nowak1997influence,
+  title={Influence of low and high frequency inputs on spike timing in visual cortical neurons.},
+  author={Nowak, Lionel G and Sanchez-Vives, Maria V and McCormick, David A},
+  journal={Cerebral cortex (New York, NY: 1991)},
+  volume={7},
+  number={6},
+  pages={487--501},
+  year={1997}
+}
+@article{mainen1995reliability,
+  title={Reliability of spike timing in neocortical neurons},
+  author={Mainen, Zachary F and Sejnowski, Terrence J},
+  journal={Science},
+  volume={268},
+  number={5216},
+  pages={1503--1506},
+  year={1995},
+  publisher={American Association for the Advancement of Science}
+}
+@article{mcdonnell2002characterization,
+  title={A characterization of suprathreshold stochastic resonance in an array of comparators by correlation coefficient},
+  author={Mcdonnell, Mark D and Abbott, Derek and Pearce, Charles EM},
+  journal={Fluctuation and Noise Letters},
+  volume={2},
+  number={03},
+  pages={L205--L220},
+  year={2002},
+  publisher={World Scientific}
+}
+@article{bulsara1996threshold,
+  title={Threshold detection of wideband signals: A noise-induced maximum in the mutual information},
+  author={Bulsara, Adi R and Zador, Anthony},
+  journal={Physical Review E},
+  volume={54},
+  number={3},
+  pages={R2185},
+  year={1996},
+  publisher={APS}
+}
+
+@article{Sadeghi2007,
+author = {Sadeghi, Soroush G and Chacron, Maurice J and Taylor, Michael C and Cullen, Kathleen E},
+doi = {10.1523/JNEUROSCI.4690-06.2007},
+file = {:home/huben/Downloads/31{\_}sadeghi{\_}chacron{\_}taylor{\_}cullen{\_}2007.pdf:pdf},
+keywords = {detection threshold,heterogeneity,information theory,regular afferents,spike timing,vestibular afferents},
+number = {4},
+pages = {771--781},
+title = {{Neural Variability, Detection Thresholds, and Information Transmission in the Vestibular System}},
+journal = {Journal of Neuroscience,},
+volume = {27},
+year = {2007}
+}
+@article{hoch2003optimal,
+  title={Optimal noise-aided signal transmission through populations of neurons},
+  author={Hoch, Thomas and Wenning, Gregor and Obermayer, Klaus},
+  journal={Physical Review E},
+  volume={68},
+  number={1},
+  pages={011911},
+  year={2003},
+  publisher={APS}
+}
+
+@article{mcdonnell2006optimal,
+  title={Optimal information transmission in nonlinear arrays through suprathreshold stochastic resonance},
+  author={McDonnell, Mark D and Stocks, Nigel G and Pearce, Charles EM and Abbott, Derek},
+  journal={Physics Letters A},
+  volume={352},
+  number={3},
+  pages={183--189},
+  year={2006},
+  publisher={Elsevier}
+}
+
+@article{mcdonnell2007optimal,
+  title={Optimal stimulus and noise distributions for information transmission via suprathreshold stochastic resonance},
+  author={McDonnell, Mark D and Stocks, Nigel G and Abbott, Derek},
+  journal={Physical Review E},
+  volume={75},
+  number={6},
+  pages={061105},
+  year={2007},
+  publisher={APS}
+}
+
+@article{Borst1999,
+author = {Borst, Alexander},
+file = {:home/huben/Downloads/nn1199{\_}947.pdf:pdf},
+pages = {13--16},
+title = {{Information theory and neural coding}},
+year = {1999}
+}
+@book{Walz2016,
+author = {Walz, Henriette and Maler, Leonard and Longtin, Andre and Benda, Jan},
+file = {:home/huben/Downloads/punitModel.pdf:pdf},
+isbn = {2154253113},
+title = {{A simple neuron model of spike generation accurately describes frequency tuning of an electroreceptor population.}},
+year = {2016}
+}
+@article{Stocks2000,
+  title={Suprathreshold stochastic resonance in multilevel threshold systems},
+  author={Stocks, NG},
+  journal={Physical Review Letters},
+  volume={84},
+  number={11},
+  pages={2310},
+  year={2000},
+  publisher={APS}
+}
+@article{Strong1998,
+author = {Strong, S P and Koberle, Roland and Bialek, William},
+file = {:home/huben/Desktop/PhysRevLett.80.197.pdf:pdf},
+pages = {197--200},
+title = {{Entropy and Information in Neural Spike Trains}},
+year = {1998}
+}
+@article{deweese1995information,
+  title={Information flow in sensory neurons},
+  author={DeWeese, M and Bialek, W},
+  journal={Il Nuovo Cimento D},
+  volume={17},
+  number={7-8},
+  pages={733--741},
+  year={1995},
+  publisher={Springer}
+}
+@article{bialek1993bits,
+  title={Bits and brains: Information flow in the nervous system},
+  author={Bialek, William and DeWeese, Michael and Rieke, Fred and Warland, David},
+  journal={Physica A: Statistical Mechanics and its Applications},
+  volume={200},
+  number={1-4},
+  pages={581--593},
+  year={1993},
+  publisher={Elsevier}
+}
+
+@article{wiesenfeld1995stochastic,
+  title={Stochastic resonance and the benefits of noise: from ice ages to crayfish and SQUIDs},
+  author={Wiesenfeld, Kurt and Moss, Frank},
+  journal={Nature},
+  volume={373},
+  number={6509},
+  pages={33},
+  year={1995},
+  publisher={Nature Publishing Group}
+}
+
+@article{Chapeau-blondeau1996,
+author = {Chapeau-blondeau, Franqois and Godivier, Xavier and Chambet, Nicolas},
+file = {:home/huben/Desktop/PhysRevE.53.1273.pdf:pdf},
+number = {1},
+pages = {3--5},
+title = {s(t)=+„5(t},
+volume = {53},
+year = {1996}
+}
+@article{Lindner2002,
+author = {Lindner, Benjamin and Schimansky-geier, Lutz},
+doi = {10.1103/PhysRevE.66.031916},
+file = {:home/huben/Desktop/PhysRevE.66.031916.pdf:pdf},
+pages = {1--6},
+title = {{Maximizing spike train coherence or incoherence in the leaky integrate-and-fire model}},
+year = {2002}
+}
+
+@article{Farkhooi2009,
+author = {Farkhooi, Farzad and Strube-Bloss, Martin F and Nawrot, Martin P},
+journal = {Physical Review E},
+month = {feb},
+number = {2},
+pages = {21905},
+publisher = {American Physical Society},
+title = {{Serial correlation in neural spike trains: Experimental evidence, stochastic modeling, and single neuron variability}},
+url = {http://link.aps.org/doi/10.1103/PhysRevE.79.021905},
+volume = {79},
+year = {2009}
+}
+@article{Mileva2016,
+author = {Mileva, G R and Kozak, I J and Lewis, J E},
+doi = {10.1016/j.neuroscience.2016.01.014},
+file = {:home/huben/Desktop/1-s2.0-S0306452216000336-main.pdf:pdf},
+issn = {0306-4522},
+journal = {NEUROSCIENCE},
+keywords = {facilitation,synaptic depression,weakly electric},
+pages = {1--11},
+publisher = {IBRO},
+title = {{Short-term synaptic plasticity across topographic maps in the electrosensory system}},
+url = {http://dx.doi.org/10.1016/j.neuroscience.2016.01.014},
+volume = {318},
+year = {2016}
+}
+@article{Maler2009,
+abstract = {The electric fish Apteronotus leptorhynchus emits a high-frequency electric organ discharge (EOD) sensed by specialized electroreceptors (P-units) distributed across the fish's skin. Objects such as prey increase the amplitude of the EOD over the underlying skin and thus cause an increase in P-unit discharge. The resulting localized intensity increase is called the electric image and is detected by its effect on the P-unit population; the electric image peak value and the extent to its spreads are cues utilized by these fish to estimate the location and size of its prey. P-units project topographically to three topographic maps in the electrosensory lateral line lobe (ELL): centromedial (CMS), centrolateral (CLS), and lateral (LS) segments. In a companion paper I have calculated the receptive fields (RFs) in these maps: RFs were small in CMS and very large in LS, with intermediate values in CLS. Here I use physiological data to create a simple model of the RF structure within the three ELL maps and to compute the response of these model maps to simulated prey. The Fisher information (FI) method was used to compute the optimal estimates possible for prey localization across the three maps. The FI predictions were compared with behavioral studies on prey detection. These comparisons were used to frame alternative hypotheses on the functions of the three maps and on the constraints that RF size and synaptic strength impose on weak signal detection and estimation.},
+author = {Maler, Leonard},
+doi = {10.1002/cne.22120},
+file = {:home/huben/Desktop/Maler2009a-2.pdf:pdf},
+issn = {00219967},
+journal = {Journal of Comparative Neurology},
+keywords = {Electric fish,Electrosensory lateral line lobe,Fisher information,Receptive field,Stimulus estimation,Topographic maps},
+number = {5},
+pages = {394--422},
+pmid = {19655388},
+title = {{Receptive field organization across multiple electrosensory maps. II. Computational analysis of the effects of receptive field size on prey localization}},
+volume = {516},
+year = {2009}
+}
+@article{Cumming2007,
+abstract = {Error bars commonly appear in figures in publications, but experimental biologists are often unsure how they should be used and interpreted. In this article we illustrate some basic features of error bars and explain how they can help communicate data and assist correct interpretation. Error bars may show confidence intervals, standard errors, standard deviations, or other quantities. Different types of error bars give quite different information, and so figure legends must make clear what error bars represent. We suggest eight simple rules to assist with effective use and interpretation of error bars.},
+archivePrefix = {arXiv},
+arxivId = {Error bars in experimental biology},
+author = {Cumming, G. and Fidler, F. and Vaux, D.L.},
+doi = {10.1083/jcb.200611141},
+eprint = {Error bars in experimental biology},
+isbn = {0021-9525 (Print)},
+issn = {0021-9525},
+journal = {The Journal of Cell Biology},
+number = {1},
+pages = {7--11},
+pmid = {17420288},
+title = {{Error bars in experimental biology}},
+url = {http://www.jcb.org/cgi/doi/10.1083/jcb.200611141},
+volume = {177},
+year = {2007}
+}

firing_rate.tex

@@ -0,0 +1,184 @@
+\section*{High and low firing rates}
+
+One key factor that determines how well a neuron can encode a given signal is
+the firing rate of the neuron. Though it has been shown to be possible for
+neurons to encode signals with frequencies above their firing rate \citep{knight1972},
+in general higher firing rates lead to a better encoding of the signal. 
+In our simulations the firing rate of the neurons depends on the noise added
+to the neurons. The effect can be seen in figure \ref{firingrates}, which shows
+average firing rate as a function of the mean input for different noise strengths.
+The largest differences can be seen for inputs with a mean around the firing threshold
+(10\milli\volt) and below. While for weak noise (gray) there is an obvious non-linearity
+at the firing threshold, increasing noise strength linearizes the average firing rate.
+This illustrates subthreshold SR quite well, as the noise induces firing of the neurons
+for signals which would otherwise be too weak to elicit a response.
+For stronger mean inputs like the 15\milli\volt\usk (second vertical black line) 
+we use in our simulations firing rate is roughly linear with input strength and 
+is not sensitive to changes in the noise strength. Therefore, the effect of noise
+on the average firing rate of the neurons plays at most a weak role in the
+explanation of SSR. However, the mean input strength is very important because of 
+its effects on the average firing rate, as we will show below.\footnote{could use a plot comparing high/low directly}
+We found that in our simulations, the amplitude of the signal has a negligible 
+influence on the firing rate.
+
+
+
+
+Previously we looked at the optimum noise value for a given a population size. Now we look
+at optimal population sizes for a given noise value: We define a
+"maximum" coding fraction as the coding fraction at a population size of 4096 neurons,
+if the coding faction at this population size is no more than 2\% greater than the coding
+fraction for a population of 2048 neurons. This ensures that coding
+fraction has reasonably converged at this point. To see why this is important,
+compare the yellow line (noise strength \(10^{-3}\milli\volt\square\per\hertz\)) 
+in figure \ref{CodingFrac} B to the other lines in that figure.
+The coding fraction is still rising with increasing population size and there is no reliable
+way to estimate for which population size it will ultimately converge and what the maximum coding
+fraction will be. We are only using the noise strengths for which coding fraction has
+converged in the following analysis.
+Then, we define the "optimum" population size as the size where coding fraction is 95\% of the maximum
+coding fraction, using linear interpolation between the different population sizes. Exponential interpolation
+yielded essentially the same results. We call this population size "optimal" because we assume that, for efficiency reasons, 
+population size should be as small as possible while having very good encoding capabilities.  
+
+\subsection*{Strong average input (high firing rates)}
+
+
+
+\begin{figure}
+	\centering
+	\includegraphics[width=0.32\linewidth]{{img/popsize_15.0_1.0}.pdf}
+	\includegraphics[width=0.32\linewidth]{{img/max_cf_15.0_1.0}.pdf}	
+	\includegraphics[width=0.32\linewidth]{{img/improvement_15.0_1.0}.pdf}
+	
+	\includegraphics[width=0.32\linewidth]{{img/popsize_10.5_1.0}.pdf}
+	\includegraphics[width=0.32\linewidth]{{img/improvement_10.5_1.0}.pdf}
+    \includegraphics[width=0.32\linewidth]{{img/max_cf_10.5_1.0}.pdf}	
+	\caption{An overview of the effect of different noise strengths on maximum coding fraction and population size. We only considered noise values where the difference in coding fraction between population sizes n=4096 and n=2048 is less than 2\%. Average firing rate of the neurons was about 91\hertz. Input strengths where chosen so taht the power of the signal in the corresponding bands is the same (1.0\milli\volt for the broadband and 0.5\milli\volt for the narrowband signals. Top to bottom:  a) Minimum population size needed to have coding fraction be at least 95\% of the maximum as a function of noise. Optimal population size grows with increasing noise. Optimal population size is larger for the narrowband signals (dots) than for the broadband signal (crosses). For weak noise and narrowband signals, there is little difference in the optimal population size for the high frequency interval
+	(brown) and the low frequency interval (blue). As noise becomes stronger, optimal population size 
+	is larger for the higher interval. For the broadband signal, optimum population size is always largest for the higher frequency interval, then for the broadband signal and finally the low frequency narrowband signal. That minimum population size is larger for the narrowband signals than
+	for the broadband signal can be explained by the fact that the maximum coding fraction (b) is higher for the narrowband signals. b) Maximum coding fraction is higher for lower intervals and narrower bands. In the case of the narrowband signal and the lwoer interval, maximum coding fraction is close to 1 for all noise strengths. For weak noise, the low interval in the
+	broadband signal and the higher narrowband signal have very similar maximum coding fractions. With increasing noise, the maximum coding fraction rises much 
+	faster for the high frequency narrowband signal. The broadband signal and the high frequency interval inside that signal have very low maximum coding fraction for weak noise. Increasing the strength of the noise at some point coding fraction begins to increase rapidly. c) Frequency band, not signal bandwidth appears to be the main factor in the relative improvement of coding fraction with increasing population size. For the slow narrowband signal the improvement is mostly not because the maximum becomes higher, as it changes very little. Instead the reason is that a single neuron has a diminished ability to encode the signal with increasing noise levels (see fig. \ref{CodingFrac} B and C). For the whole broadband signal and the low frequency interval within, for weak noise there is almost no improvement in coding fraction through increasing population size. Interestingly, with stronger noise the relative improvement for the low
+	frequency interval is the same for both the narrowband and the broadband signal, even though maximum coding fraction is very different, at least for intermediate noise strength (\(10^{-4}\) to \(10^{-3}\)\milli\volt\squared\per\hertz).
+		}
+	\label{fig:popsizenarrow15}
+\end{figure}
+
+First, we consider the case of strong input (15 \milli\volt) which leads to a an average firing
+rate of about 91\hertz. As before, we see great differences for the different frequency intervals. In figure \ref{fig:popsizenarrow15} A
+we see population sizes necessary to reach an encoding quality close to the maximum. 
+To encode the high frequency parts of the spectrum much larger populations are required
+than for the low frequency parts. As expected, the 
+size of the optimal population increases with increasing noise strength. To encode the broadband
+signal over its entire spectrum, a population size between the population sizes 
+for the narrowband intervals is optimal. This can
+be understood because the broadband signal contains both the "easy" and the "difficult" intervals
+and should therefore fall in between. The same is true for the maximum coding fraction (figure \ref{fig:popsizenarrow15} C):
+Again, the broadband signal falls in between the intervals. As we could see before (figure \ref{smallbroad}), 
+the lower frequency interval is easier to encode than the higher frequency interval
+and the whole broadband signal. 
+We also considered the relative effect of 
+increasing population size. To quantify this, we divided the maximum coding fraction by 
+the coding fraction of a single neuron. Figure \ref{fig:popsizenarrow15} E shows that 
+the increase in coding fraction gained by increasing population size
+is larger for the higher frequency interval. In contrast, a single neuron can 
+encode the broadband signal or the low frequency interval about as well as a 
+larger population can.
+That relative improvement increases for stronger noise 
+is a consequence of the reduction in the encoding 
+capability of a single neuron (compare figure \ref{CodingFrac} C).
+
+For the narrowband signals we see a similar picture: Except for very weak noise, 
+optimal population size to encode the high frequency signal is larger than that
+for the low frequency signal (figure \ref{fig:popsizenarrow15} B). Optimal
+population size for encoding of the low frequency signal now starts at a much 
+higher level than before. 
+Maximum coding fraction
+again is larger for the low frequency signal (figure \ref{fig:popsizenarrow15} D). 
+For the high frequency signal, coding fraction
+is at a much higher level than it is for the high frequency interval in the broadband signal
+for all noise strengths.
+With increasing noise strength the coding fraction increases rapidly and almost reaches the level achieved
+for the low frequency signal for the population sizes considered here.  
+The relative increase in coding fraction
+(figure \ref{fig:popsizenarrow15} F) is similar to what
+we saw for the broadband signal. For high frequencies the increase in
+coding fraction from a single neuron to a population of neurons becomes apparent
+even at a relatively low noise strength. Whereas for the low frequency
+signal the relative improvement starts at much higher noise levels and is
+mostly explained by the decrease in encoding capabilities of a single neuron.
+
+For both narrowband and broadband signals, the results here qualitatively do not depend 
+on the input amplitude. See \textit{appendix} for more information.
+
+\subsection*{Weak average input (low firing rates)}
+
+\begin{figure}
+
+	\includegraphics[width=0.49\linewidth]{{img/0to50_broad_small_coherence_10.5_0.5}.pdf}
+
+	\caption{Coherence curves for broad and narrow frequency range inputs. The average firing rate of the cells is marked with a black vertical line. a) Broad spectrum. Coherence for low frequencies is much lower than for the case of higher mean input (fig. \ref{fig:coherence_narrow_15.0} A). For the weak noise level (blue), population sizes n=4 and n=4096 show indistinguishable coding fraction. In case of a small population size, coherence is higher for stronger noise (green), contrary to what we have seen before. This can not be explained by an increase in firing rate, as the difference in firing rate is only about 1\%.  b) Narrow band inputs for two frequency ranges. Low frequency range: coherence for slow parts of the signal is similar to those in the broadband signal. High frequency range: In contrast to the case of higher mean input (fig. \ref{fig:coherence_narrow_15.0}) the high-frequency signal is encoded better than the low-frequency signal. Even for comparatively weak noise (blue), an increase in population size offers better encoding.
+}
+	\label{fig:coherence_narrow_10.5}
+\end{figure}
+
+We also consider the case of a weaker mean input (10.5\milli\volt) which leads
+to lower average firing rates (about 34\hertz). Results can be seen in figure
+\ref{fig:popsizenarrow10}. For the broadband signal, in general 
+results are similar to the results in the strong average input case. 
+We again see (fig. \ref{fig:popsizenarrow10} A) that optimum population size is
+larger for the high frequency interval than for the low frequency interval, with
+the entire broadband signal somewhere in between. Now the optimal population sizes
+are much closer than in the case of high average firing rates. The values being
+closer together is also true for the maximum coding fraction 
+(fig. \ref{fig:popsizenarrow10} C). The value for the low frequency interval
+  is now much lower for very weak noise than for the high mean input.
+The curves for both intervals and the broadband signal now look very similar
+to each other.
+Maximum coding fraction for the broadband signal and the high
+frequency interval are almost equal for noise strengths greater
+than \(10^{-3}\milli\volt\squared\per\hertz\).
+Relative improvement (fig. \ref{fig:popsizenarrow10} E) is very similar for all
+intervals. For increasing noise strength relative improvement starts to show
+the pattern we have seen before, with improvement being greatest for
+the high frequency interval, followed by the broadband signal and then the
+low frequency interval. 
+
+For the narrowband signals we see some striking differences to what we saw
+for the high mean input. Optimal population sizes are now very different for the low frequency
+signal and the high frequency signal for all noise strengths (fig. \ref{fig:popsizenarrow10} B).
+The high frequency interval signal again needs a larger population for
+encoding being close to optimal. The optimal population size for
+the low frequency signal is now very similar to the optimal population
+size for the low frequency interval in the broadband signal. 
+This contrasts with the high input case (fig. \ref{fig:popsizenarrow15} A \& B). For
+weak noise the optimal population size was much larger for the narrowband signal 
+than for the interval in the broadband signal.
+
+A striking change happens for the maximum coding fraction (fig. \ref{fig:popsizenarrow10} D).
+As opposed to every case we have looked at before, now the higher frequency
+signal appears to be more easily encoded than the low frequency signal.
+This can be explained by looking at the coherence curves of the two signals 
+(fig. \ref{fig:coherence_narrow_10.5} B). As the firing rate of the neurons
+is inside the frequency range of the low frequency signal, encoding
+of those frequencies is suppressed. This is not the case for the
+high frequency signal, where the coherence behaves similarly to
+what we have seen for the strong input case (fig. \ref{fig:coherence_narrow_15.0} B).
+Maximum coding fraction for the low frequency signal is very similar to
+the maximum coding fraction for the low frequency interval of the broadband signal.
+This is also different to what we have seen before, as for every other case 
+coding fraction can get much higher for the narrowband signals. 
+
+Relative improvement is higher for the high frequency signal 
+than for the low frequency signal (fig. \ref{fig:popsizenarrow10} F).
+For the high frequency signal,  the improvement is greater
+than for the equivalent interval in the broadband signal for weak noise. 
+Relative improvement is also greater
+than it was for high average input (fig. \ref{fig:popsizenarrow15} E \& F).		
+Here, even for weak noise increasing the population size has a large effect.
+In other words, to encode the high frequency signal in the case of a 
+low average firing rate, population size is very critical to the quality
+of the encoding. For the low frequency signal there is no large difference
+in relative improvement to the other cases.
+

fish_bands.tex

@@ -0,0 +1,130 @@
+\section*{Electric fish as a real world model system}
+
+To put the results from our simulations into a real world context, we chose the 
+weakly electric fish \textit{Apteronotus leptorhynchus} as a model system.
+\lepto\ uses an electric organ to produce electric fields which it
+uses for orientation, prey detection and communication. Distributed over the skin
+of \lepto\ are electroreceptors which produce action potentials
+in response to electric signals. 
+
+These receptor cells ("p-units") are analogous to the 
+simulated neurons we used in our simulations because they do not receive any
+input other than the signal they are encoding. Individual cells fire independently
+of each other and there is no feedback. 
+
+
+
+
+\subsection*{Results}
+
+Figure \ref{fig:ex_data} A,B and C show three examples for coherence from intracellular
+measurements in \lepto\. Each cell was exposed to up to 128 repetitions of the
+same signal. The response was then averaged over different numbers of trials to 
+simulate different population sizes of homogeneous cells. We can see that an increase
+in population size leads to higher coherence. Similar to what we saw in the simulations,
+around the average firing rate of the cell (marked by the red vertical lines), coherence
+decreases sharply. We then aggregated the results for 31 different cells (50 experiments total,
+as some cells were presented with the stimulus more than once). 
+Figure \ref{ex_data} D shows that the increase is largest inside the
+high frequency intervals. As we could see in our simulations (figures \ref{fig:popsizenarrow15} C
+and \ref{fig:popsizenarrow10} C), the ratio of coding fraction in a large population
+to the coding fraction in a single cell is larger for higher frequencies.
+
+%simulation plots are from 200hz/nice coherence curves.ipynb
+\begin{figure}
+	\centering
+	\includegraphics[width=0.49\linewidth]{img/fish/coherence_example.pdf}
+	\includegraphics[width=0.49\linewidth]{img/fish/coherence_example_narrow.pdf}
+	\includegraphics[width=0.49\linewidth]{{img/coherence/broad_coherence_15.0_1.0_different_popsizes_0.001}.pdf}
+	\includegraphics[width=0.49\linewidth]{{img/coherence/coherence_15.0_0.5_narrow_both_different_popsizes_0.001}.pdf}
+	\label{fig:ex_data}
+	\caption{A,B,C: examples of coherence in the p-Units of \lepto. Each plot shows
+	the coherence of the response of a single cell to a stimulus for different numbers of trials. 
+	Like in the simulations, increased population sizes lead to a higher coherence.
+	D: Encoding of higher frequency intervals profits more from an increase in 
+	population size than encoding of lower frequency intervals. 
+	The ratio of the coding fraction for the largest number of trials divided by
+	the coding fraction for a single trial for each of six different frequency
+	intervals. Shown here are the data  	for all 50 experiments (31 different cells). 
+	The orange line signifies the median value for all cells. The box
+	extends over the 2nd and 3rd quartile. }
+\end{figure}
+
+
+\begin{figure}
+ \centering
+ broad
+ 
+ \includegraphics[width=0.48\linewidth]{img/fish/cf_curves/cfN_broad_0.pdf}
+ \includegraphics[width=0.48\linewidth]{img/fish/cf_curves/cfN_broad_1.pdf}
+ \includegraphics[width=0.48\linewidth]{img/fish/cf_curves/cfN_broad_2.pdf}
+ \includegraphics[width=0.48\linewidth]{img/fish/cf_curves/cfN_broad_3.pdf}
+\end{figure}
+
+%box_script.py, quot_sigma() und quot_sigma_narrow()
+\begin{figure}
+ \centering
+ broad
+ 
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_0_50.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_50_100.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_100_150.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_150_200.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_200_250.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_broad_250_300.pdf}
+ 
+ 
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_0_50.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_50_100.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_100_150.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_150_200.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_200_250.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_broad_250_300.pdf}
+  
+ narrow
+ 
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_narrow_0_50.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_narrow_50_100.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_narrow_150_200.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_narrow_250_300.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/sigma_cf_quot_narrow_350_400.pdf}
+ 
+  \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_narrow_0_50.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_narrow_50_100.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_narrow_150_200.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_narrow_250_300.pdf}
+ \includegraphics[width=0.16\linewidth]{img/fish/scatter/check_fr_quot_narrow_350_400.pdf}
+ 
+\end{figure}
+
+
+
+\begin{figure}
+\centering
+\includegraphics[width=0.4\linewidth]{img/fish/diff_box.pdf}
+\includegraphics[width=0.4\linewidth]{img/fish/diff_box_narrow.pdf}
+ \includegraphics[width=0.4\linewidth]{img/relative_coding_fractions_box.pdf}
+ \notedh{needs figure 3.6 from yue and equivalent}
+\end{figure}
+
+\begin{figure}
+ \includegraphics[width=0.49\linewidth]{img/fish/ratio_narrow.pdf}
+ \includegraphics[width=0.49\linewidth]{img/fish/broad_ratio.pdf}
+ \label{freq_delta_cf}
+ \caption{This is about frequency and how it determines $delta_cf$. In other paper I have used $quot_cf$.}
+\end{figure}
+
+\subsection{Discussion}
+
+We also confirmed that the results from the theory part of the paper play a role in a 
+real world example. Inside the brain of the weakly electric fish 
+\textit{Apteronotus leptorhynchus} pyramidal cells in different areas 
+are responsible for encoding different frequencies. In each of those areas,
+cells integrate over different numbers of the same receptor cells. 
+Artificial populations consisting of different trials of the same receptor cell
+show what we have seen in our simulations: Larger populations help
+especially with the encoding of high frequency signals. These results
+are in line with what is known about the pyramidal cells of \lepto:
+The cells which encode high frequency signals best are the cells which
+integrate over the largest number of neurons.
+

fish_methods.tex

@@ -0,0 +1,65 @@
+\subsection*{Electrophysiology}
+
+We recorded electrophysiological data from X cells from Y different fish.
+
+\textit{Surgery}. Twenty-two E. virescens (10 to 21 cm) were used for
+single-unit recordings. Recordings of electroreceptors were made
+from the anterior part of the lateral line nerve.
+Fish were initially anesthetized with 150 mg/l MS-222 (PharmaQ,
+Fordingbridge, UK) until gill movements ceased and were then
+respirated with a constant flow of water through a mouth tube,
+containing 120 mg/l MS-222 during the surgery to sustain anesthesia.
+The lateral line nerve was exposed dorsal to the operculum. Fish were
+fixed in the setup with a plastic rod glued to the exposed skull bone.
+The wounds were locally anesthetized with Lidocainehydrochloride
+2\% (bela-pharm, Vechta, Germany) before the nerve was exposed.
+Local anesthesia was renewed every 2 h by careful application of
+Lidocaine to the skin surrounding the wound.
+After surgery, fish were immobilized with 0.05 ml 5 mg/ml tubocurarine (Sigma-Aldrich, Steinheim, Germany) injected into the trunk
+muscles. 
+\sout{Since tubocurarine suppresses all muscular activity, it also
+suppresses the activity of the electrocytes of the electric organ and thus
+strongly reduces the EOD of the fish. We therefore mimicked the EOD
+by a sinusoidal signal provided by a sine-wave generator (Hameg HMF
+2525; Hameg Instruments, Mainhausen, Germany) via silver electrodes
+in the mouth tube and at the tail. The amplitude and frequency of the
+artificial field were adjusted to the fish’s own field as measured before
+surgery.} After surgery, fish were transferred into the recording tank of the
+setup filled with water from the fish’s housing tank not containing
+MS-222. Respiration was continued without anesthesia. The animals
+were submerged into the water so that the exposed nerve was just above
+the water surface. Electroreceptors located on the parts above water
+surface did not respond to the stimulus and were excluded from analysis.
+Water temperature was kept at 26°C.\footnote{From St\"ockl et al. 2014}
+
+\textit{Recording. }Action potentials from electroreceptor afferents were
+recorded intracellularly with sharp borosilicate microelectrodes
+(GB150F-8P; Science Products, Hofheim, Germany), pulled to a resistance between 20 and 100 M and filled with a 1 M KCl solution.
+Electrodes were positioned by microdrives (Luigs-Neumann, Ratingen,
+Germany). As a reference, glass microelectrodes were used. They were
+placed in the tissue surrounding the nerve, adjusted to the isopotential line
+of the recording electrode. The potential between the micropipette and the
+reference electrode was amplified (SEC-05X; npi electronic) and lowpass filtered at 10 kHz. Signals were digitized by a data acquisition board
+(PCI-6229; National Instruments) at a sampling rate of 20 kHz. Spikes
+were detected and identified online based on the peak-detection algorithm
+proposed by Todd and Andrews (1999).
+The EOD of the fish was measured between the head and tail via
+two carbon rod electrodes (11 cm long, 8-mm diameter). The potential
+at the skin of the fish was recorded by a pair of silver wires, spaced
+1 cm apart, which were placed orthogonal to the side of the fish at
+two-thirds body length. The residual EOD potentials were recorded
+and monitored with a pair of silver wire electrodes placed in a piece
+of tube that was put over the tip of the tail. These EOD voltages were
+amplified by a factor of 1,000 and band-pass filtered between 3 Hz and
+1.5 kHz (DPA-2FXM; npi electronics).
+Stimuli were attenuated (ATN-01M; npi electronics), isolated from
+ground (ISO-02V; npi electronics), and delivered by two carbon rod
+electrodes (30-cm length, 8-mm diameter) placed on either side of the
+fish parallel to its longitudinal axis. Stimuli were calibrated to evoke
+defined AM measured close to the fish. Spike and EOD detection,
+stimulus generation and attenuation, as well as preanalysis of the
+data were performed online during the experiment within the
+RELACS software version 0.9.7 using the efish plugin-set (by J.
+Benda: http://www.relacs.net).\footnote{From St\"ockl et al. 2014}
+
+\textit{Stimulation.} White noise stimuli with a cutoff frequency of 300{\hertz} defined an AM of the fish's signal. The stimulus was combined with the fish's own EOD in a way that the desired AM could be measured near the fish. Amplitude of the AM was 10\% (?) of the amplitude of the EOD. Stimulus duration was between 2s and 10s, with a time resolution of X.