dennis/role_of_noise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Change first half of sigma part

Browse Source
This commit is contained in:
Dennis Huben 2024-09-02 16:00:42 +02:00
parent 4da718524e
commit f35fa7cfcb
1 changed files with 282 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

300
main.tex
View File

@ -99,7 +99,7 @@ If noise is weak, the neurons behave regularly and similar to each other (A). Fo
\label{example_spiketrains}
\end{figure}
\subsection*{Analysis}
\subsection{Analysis}
For each combination of parameters, a histogram of the output spikes from all neurons or a subset of the neurons was created.
The coherence $C(f)$ was calculated \citep{lindner2016mechanisms} in frequency space as the fraction between the squared cross-spectral density $|S_{sx}^2|$ of input signal $s(t) = \sigma I_{t}$ and output spikes x(t), $S_{sx}(f) = \mathcal{F}\{ s(t)*x(t) \}(f) $, divided by the product of the power spectral densities of input ($S_{ss}(f) = |\mathcal{F}\{s(t)\}(f)|^2 $) and output ($S_{xx}(f) = |\mathcal{F}\{x(t)\}(f)|^2$), where $\mathcal{F}\{ g(t) \}(f)$ is the Fourier transform of g(t).
\begin{equation}C(f) = \frac{|S_{sx}(f)|^2}{S_{ss}(f) S_{xx}(f)}\label{coherence}\end{equation}
@ -132,11 +132,11 @@ For the weak signals ($\sigma = 0.1mV$) combined with the strongest noise ($D =
\subsection{Simulations with more neurons}
\subsection*{Noise makes neurons' responses different from each other}
\subsection{Noise makes neurons' responses different from each other}
If noise levels are low (fig. \ref{example_spiketrains} a)), neurons within a population with behave very similarly to each other. There is little variation in the spike responses of the neurons to a signal, and recreating the signal is difficult. If the strength of the noise is increasing, at some point the coding fraction will also begin increasing. The signal recreation will become better as the responses of the different neurons begin to deviate from each other. When noise strength is increased even further at some point a peak coding fraction is reached. This point is the optimal noise strength for the given parameters (fig. \ref{example_spiketrains} b)). If the strength of the noise is increased beyond this point, the responses of the neurons will be determined more by random fluctuations and less by the actual signal, making reconstruction more difficult (fig. \ref{example_spiketrains} c)). At some point, signal encoding breaks down completely and coding fraction goes to 0.
\subsection*{Large population size is only useful if noise is strong}
\subsection{Large population size is only useful if noise is strong}
We see that an increase in population size leads to a larger coding fraction until it hits a limit which depends on noise. For weak noise the increase in conding fraction with an increase in population size is low or non-existent. This can be seen in figure \ref{cf_limit} c) where the red ($10^{-5}\frac{mV^2}{Hz}$) and orange ($10^{-4}\frac{mV^2}{Hz}$) curves (relatively weak noise) saturate for relatively small population size (about 8 neurons and 32 neurons respectively).
An increase in population size also leads to the optimum noise level moving towards stronger noise (green dots in figure \ref{cf_limit} a)). A larger population can exploit the higher noise levels better. Within the larger population the precision of the individual neurons becomes less important. After the optimum noise where peak coding fraction is reached, an increase in noise strength leads to a reduction in coding fraction. If the noise is very strong, coding fraction can reach approximately 0. This happens earlier (for weaker noise) in smaller populations than in larger populations. Together those facts mean that for a given noise level and population size, coding fraction might already be declining; whereas for larger populations, coding fraction can still be increasing. A given amount of noise can lead to a very low coding fraction in a small population, but to a greater coding fraction in a larger population. (figure \ref{cf_limit} c), blue and purple curves). The noise levels that work best for large populations are in general performing very bad in small populations. If coding fraction is supposed to reach its highest values and needs large populations to do so, the necessary noise strength will be at a level, where basically no encoding will happen in a single neurons or small populations.
@ -151,7 +151,7 @@ An increase in population size also leads to the optimum noise level moving towa
\notedh{langsames signal hier nehmen(!?)}}
\end{figure}
\subsection*{Influence of the input is complex}
\subsection{Influence of the input is complex}
Two very important variables are the mean strength of the signal, equivalent to the baseline firing rate of the neurons and the strength of the signal. A higher baseline firing rate leads to a larger coding fraction. In our terms that means that a mean signal strength $\mu$ that is much above the signal will lead to higher coding fractions than if the signal strength is close to the threshold (see figure \ref{cf_limit} b), orange curves are above the green curves). The influence of the signal amplitude $\sigma$ is more complex. In general, at small population sizes, larger amplitudes appear to work better, but with large populations they might perform as well or even better than stronger signals (figure \ref{cf_limit} c), dashed curves vs solid curves.)
\begin{figure}
@ -168,7 +168,7 @@ C: Peak coding fraction as a function of population size for different input par
\subsection*{Slow signals are more easily encoded}
\subsection{Slow signals are more easily encoded}
To encode a signal well, neurons in a population need to keep up with the rising and falling of the signal.
Signals that change fast are harder to encode than signals which change more slowly. When a signal changes more gradually, the neurons can slowly adapt their firing rate. A visual example can be see in figure \ref{freq_raster}. When all other parameters are equal, a signal with a lower frequency is easier to recreate from the firing of the neurons.
In the rasterplots one can see especially for the 50Hz signal (bottom left) that the firing probability of each neuron follows the input signal. When the input is low, almost none of the neurons fire. The result are the ``stripes'' we can see in the rasterplot. The stripes have a certain width which is determined by the signal frequency and the noise level. When the signal frequency is low, the width of the stripes can't be seen in a short snapshot. For the 50Hz signal in this example we can clearly see a break in the firing activity of the neurons at around 25ms. The slower changes in the signal allow for the reconstruction to follow the original signal more closely.
@ -191,7 +191,7 @@ Something similar can be said for the 1Hz signal. Because the peaks are about 1s
\subsection*{Fast signals are harder to encode - noise can help with that}
\subsection{Fast signals are harder to encode - noise can help with that}
For low frequency signals, the coding fraction is almost always at least as large as the coding is for signals with higher frequency. For the parameters we have used there is very little difference in coding fraction for a random noise signal with frequencies of 1Hz and 10Hz respectively (figure \ref{cf_for_frequencies}, bottom row).
For all signal frequencies and amplitudes a signal mean much larger than the threshold ($\mu = 15.0mV$, with the threshold at $10.0mV$) results in a higher coding fraction than the signal mean closer to the threshold ($\mu = 10.5 mV$). Firing rates of the neurons is much higher at the large input: about 90 Hz vs. 30 Hz for the lower signal mean.
We also find that for the signal mean which is further away from the threshold for the loss of coding fraction from the 10Hz signal to the 50Hz signal is smaller than for the lower signal mean. This is partially explained by the firing rate of the neurons: Around the firing rate the signal encoding is weaker (see figure \ref{CodingFrac}. In general, an increase in signal frequency and bandwidth leads to a decrease in the maximum achievable coding fraction. This decrease is smaller if the noise is stronger. In some conditions, a 50 Hz signal can be encoded as well as a 10 Hz signal (fig. \ref{cf_for_frequencies} d)).
@ -211,7 +211,7 @@ For slower signals, coding fraction converges faster in terms of population size
This (convergence speed) is also true for stronger signals as opposed to weaker signals.
For slower signals the maximum value is reached for weaker noise.}
\subsection*{A tuning curve allows calculation of coding fraction for arbitrarily large populations}
\subsection{A tuning curve allows calculation of coding fraction for arbitrarily large populations}
To understand information encoding by populations of neurons it is common practice to use simulations. However, the size of the simulated population is limited by computational power. We demonstrate a way to circumvent these limitations, allowing to make predictions in the limit case of large population size. We use the interpretation of the tuning curve as a kind of averaged population response. To calculate this average, we need relatively few neurons to reproduce the response of an arbitrarily large population of neurons. This allows the necessary computational power to be greatly reduced.
@ -302,14 +302,14 @@ In the case of low cutoff frequency and strong noise we were able to derive a fo
\subsection{Different frequency ranges}
\subsection*{Narrow-/wideband}
\subsection{Narrow-/wideband}
\subsection*{Narrowband stimuli}
\subsection{Narrowband stimuli}
Using the \(f_{cutoff} = 200 \hertz\usk\) signal, we repeated the analysis (fig. \ref{cf_limit}) considering only selected parts of the spectrum. We did so for two "low frequency" (0--8Hz, 0--50Hz) and two "high frequency" (192--200Hz, 150--200Hz) intervals.\notedh{8Hz is not in yet.} We then compared the results to the results we get from narrowband stimuli, with power only in those frequency bands.
To keep the power of the signal inside the two intervals the same as in the broadband stimulus, amplitude of the narrowband signals was less than that of the broadband signal. For the 8Hz intervals, amplitude (i.e. standard deviation) of the signal was 0.2mV, or a fifth of the amplitude of the broadband signal. Because signal power is proportional to the square of the amplitude, this was appropriate for a stimulus with a spectrum 25 times smaller. Similarly, for the 50Hz intervals we used a 0.5mV amplitude, or half of that of the broadband stimulus.
As the square of the amplitude is equal to the integral over the frequency spectrum, for a signal with a quarter of the width we need to half the amplitude to have the same power in the interval defined by the narrowband signals.
\subsection*{Smaller frequency intervals in broadband signals }
\subsection{Smaller frequency intervals in broadband signals }
\begin{figure}
\includegraphics[width=0.45\linewidth]{img/small_in_broad_spectrum}
@ -375,7 +375,7 @@ Signal mean $\mu=15.0\milli\volt$, signal amplitude $\sigma=1.0\milli\volt$ and
\label{smallbroad}
\end{figure}
\subsection*{Narrowband Signals vs Broadband Signals}
\subsection{Narrowband Signals vs Broadband Signals}
In nature, often an external stimulus covers a narrow frequency range
that starts at high frequencies, so that only using broadband white noise signals
@ -426,9 +426,7 @@ critical importance to the encoding of the signal.
\section{Theory}
\subsection{Firing rates}
\subsection*{For large population sizes and strong noise, coding fraction becomes a function of their quotient}
\subsection{For large population sizes and strong noise, coding fraction becomes a function of their quotient}
For the linear response regime of large noise, we can estimate the coding fraction. From Beiran et al. 2018 we know the coherence in linear response is given as
@ -473,11 +471,20 @@ From eqs.\ref{eq:linear_response} and \ref{eq:simplified_single_coherence} it fo
Right: signal mean 15.0mV, signal amplitude 1.0mV, $f_{c}$ 200Hz.}
\end{figure}
\input{firing_rate}
\subsection{Refractory period}
\input{refractory_periods}
We analyzed the effect of non-zero refractory periods on the previous results. We repeated the same simulations as before but added a 1ms or a 5ms refractory period to each of the LIF-neurons. Results are summarized in figure \ref{refractory_periods}.
Results change very little for a refractory period of 1ms, especially for large noise values. For a refractory period of 5ms resulting coding fraction is lower for almost all noise values. Paradoxically, for high frequencies in smallband signals and very small noise, coding fraction actually is larger for 5ms refractory period than for 1ms. \notedh{Needs plots!} In spite of this, coding fraction is still largest for the LIF-ensembles without refractory period.
We also find all other results replicated even with refractory periods of 1ms or 5ms: Figure \ref{refractory_periods} shows that the optimal noise stills grows with \(\sqrt{N}\) for both the 1ms and the 5ms refractory period. We see an increase in the value of the optimum noise with an increase of the refractory period. The achievable coding fraction is lower for the neurons with refractory periods, especially at the maximum. In the limit of large noise, the neurons with 1ms refractory period and the ones with no refractory period also result in similar coding fractions, over a wide range of population sizes. However, this is not true for the neurons with 5ms refractory period.
\begin{figure}
\includegraphics[width=0.8\linewidth]{img/ordnung/refractory_periods_coding_fraction.pdf}
\caption{Repeating the simulations adding a refractory period to the LIF-neurons shows no qualitative changes in the SSR behaviour of the neurons. Coding fraction is lower the longer the refractory period. The SSR peak moves to stronger noise; cells with larger refractory periods need stronger noise to work optimally.}
\label{refractory_periods}
\end{figure}
\section{Electric fish}
@ -485,10 +492,267 @@ From eqs.\ref{eq:linear_response} and \ref{eq:simplified_single_coherence} it fo
\subsection{Methods}
\input{fish_methods}
\subsection*{Electrophysiology}
We recorded electrophysiological data from X cells from Y different fish.
\textit{Surgery}. Twenty-two E. virescens (10 to 21 cm) were used for
single-unit recordings. Recordings of electroreceptors were made
from the anterior part of the lateral line nerve.
Fish were initially anesthetized with 150 mg/l MS-222 (PharmaQ,
Fordingbridge, UK) until gill movements ceased and were then
respirated with a constant flow of water through a mouth tube,
containing 120 mg/l MS-222 during the surgery to sustain anesthesia.
The lateral line nerve was exposed dorsal to the operculum. Fish were
fixed in the setup with a plastic rod glued to the exposed skull bone.
The wounds were locally anesthetized with Lidocainehydrochloride
2\% (bela-pharm, Vechta, Germany) before the nerve was exposed.
Local anesthesia was renewed every 2 h by careful application of
Lidocaine to the skin surrounding the wound.
After surgery, fish were immobilized with 0.05 ml 5 mg/ml tubocurarine (Sigma-Aldrich, Steinheim, Germany) injected into the trunk
muscles.
\sout{Since tubocurarine suppresses all muscular activity, it also
suppresses the activity of the electrocytes of the electric organ and thus
strongly reduces the EOD of the fish. We therefore mimicked the EOD
by a sinusoidal signal provided by a sine-wave generator (Hameg HMF
2525; Hameg Instruments, Mainhausen, Germany) via silver electrodes
in the mouth tube and at the tail. The amplitude and frequency of the
artificial field were adjusted to the fish’s own field as measured before
surgery.} After surgery, fish were transferred into the recording tank of the
setup filled with water from the fish’s housing tank not containing
MS-222. Respiration was continued without anesthesia. The animals
were submerged into the water so that the exposed nerve was just above
the water surface. Electroreceptors located on the parts above water
surface did not respond to the stimulus and were excluded from analysis.
Water temperature was kept at 26°C.\footnote{From St\"ockl et al. 2014}
\textit{Recording. }Action potentials from electroreceptor afferents were
recorded intracellularly with sharp borosilicate microelectrodes
(GB150F-8P; Science Products, Hofheim, Germany), pulled to a resistance between 20 and 100 M and filled with a 1 M KCl solution.
Electrodes were positioned by microdrives (Luigs-Neumann, Ratingen,
Germany). As a reference, glass microelectrodes were used. They were
placed in the tissue surrounding the nerve, adjusted to the isopotential line
of the recording electrode. The potential between the micropipette and the
reference electrode was amplified (SEC-05X; npi electronic) and lowpass filtered at 10 kHz. Signals were digitized by a data acquisition board
(PCI-6229; National Instruments) at a sampling rate of 20 kHz. Spikes
were detected and identified online based on the peak-detection algorithm
proposed by Todd and Andrews (1999).
The EOD of the fish was measured between the head and tail via
two carbon rod electrodes (11 cm long, 8-mm diameter). The potential
at the skin of the fish was recorded by a pair of silver wires, spaced
1 cm apart, which were placed orthogonal to the side of the fish at
two-thirds body length. The residual EOD potentials were recorded
and monitored with a pair of silver wire electrodes placed in a piece
of tube that was put over the tip of the tail. These EOD voltages were
amplified by a factor of 1,000 and band-pass filtered between 3 Hz and
1.5 kHz (DPA-2FXM; npi electronics).
Stimuli were attenuated (ATN-01M; npi electronics), isolated from
ground (ISO-02V; npi electronics), and delivered by two carbon rod
electrodes (30-cm length, 8-mm diameter) placed on either side of the
fish parallel to its longitudinal axis. Stimuli were calibrated to evoke
defined AM measured close to the fish. Spike and EOD detection,
stimulus generation and attenuation, as well as preanalysis of the
data were performed online during the experiment within the
RELACS software version 0.9.7 using the efish plugin-set (by J.
Benda: http://www.relacs.net).\footnote{From St\"ockl et al. 2014}
\textit{Stimulation.} White noise stimuli with a cutoff frequency of 300{\hertz} defined an AM of the fish's signal. The stimulus was combined with the fish's own EOD in a way that the desired AM could be measured near the fish. Amplitude of the AM was 10\% (?) of the amplitude of the EOD. Stimulus duration was between 2s and 10s, with a time resolution of X.
\subsection{How to determine noisiness}
\subsection*{Determining noise in real world}
While in simulations we can control the noise parameter directly, we cannot do so in electrophysiological experiments.
Therefore, we need a way to quantify "noisiness".
One such way is by using the activation curve of the neuron, fitting a function and extracting the parameters from this function.
Stocks (2000) uses one such function to simulate groups of noisy spiking neurons:
\begin{equation}
\label{errorfct}\frac{1}{2}\erfc\left(\frac{\theta-x}{\sqrt{2\sigma^2}}\right)
\end{equation}
where $\sigma$ is the parameter quantifying the noise (figure \ref{Curve_examples}). $\sigma$ determines the steepness of the curve.
A neuron with a $\sigma$ of 0 would be a perfect thresholding mechanism. Firing probability for all inputs below the threshold is 0, and firing probability for all inputs above is 1.
Larger values mean a flatter activation curve. Neurons with such an activation curve can sometimes fire even for signals below the firing threshold, while it will sometimes not fire for inputs above the firing threshold. Its firing behaviour is influenced less by the signal and more affected by noise.
We also tried different other methods of quantifying noise commonly used (citations), but none of them worked as well as the errorfunction fit (fig. \ref{noiseparameters} and \ref{noiseparameters2}).
\subsection*{Methodology}
We calculate the cross correlation between the signal and the discrete output spikes.
The signal values were binned in 50 bins. The result is a discrete Gaussian distribution around 0mV, the mean of the signal, as is expected from the way the signal was created.
We have to account for the delay between the moment we play the signal and when it gets processed in the cell, which can for example depend on the position of the cell on the skin. We can easily reconstruct the delay from the measurements.
The position of the peak of the crosscorrelation is the time shift for which the signal influences the result of the output the most. \notedh{This will be much clearer with a plot.}
Then for every spike we assign the value of the signal at the time of the spike minus the time shift.
The result is a histogram, where each signal value bin has a number of spikes.
This histogram is then normalized by the distribution of the signal. The result is another histogram, whose values are firing frequencies for each signal value. Because those frequencies are just firing probabilities multiplied by time, we can fit a Gaussian error function to those probabilities.
\subsection*{Simulation}
To confirm that the $\sigma$ parameter estimated from the fit is indeed a good measure for the noisiness, we validated it against D, the noise parameter from the simulations. We find that there is a strictly monotonous relationship between the two for different sets of simulation parameters. Other parameters often used to determine noisiness (citations) such as the variance of the spike PSTH, the coefficient of variation (CV) of the interspike interval are not as useful. In figure \ref{noiseparameters} we see why. The variance of the psth is not always monotonous in D and is very flat for low values of D.
%describe what happens to the others
%check Fano-factor maybe?
\begin{figure}
\includegraphics[width=0.45\linewidth]{img/ordnung/base_D_sigma}
\includegraphics[width=0.45\linewidth]{img/dataframe_scatter_D_normalized_psth_1ms_test_tau}
\includegraphics[width=0.45\linewidth]{img/dataframe_scatter_D_psth_5ms_test}
% \includegraphics[width=0.45\linewidth]{img/dataframe_scatter_D_cv_test}
\caption{a)The parameter \(\sigma\) as a function of the noise parameter D in LIF-simulations. There is a strictly monotonous relationship between the two, which allows us to use \(\sigma\) as a susbtitute for D in the analysis of electrophysiological experiments. b-d) different other parameters commonly used to quantify noise. None of these functions is stricly monotonous and therefore none is useful as a substitute for D. b) Peri-stimulus time histogram (PSTH) of the spikes with a bin width of 1ms, normalized by c) PSTH of the spikes with a bin width of 5ms. d) coefficient of variation (cv) of the interspike-intervals.}
\label{noiseparameters}
\end{figure}
\begin{figure}
\centering
%\includegraphics[width=0.45\linewidth]{img/ordnung/base_D_sigma}\\
\includegraphics[width=0.23\linewidth]{img/simulation_sigma_examples/fitcurve_50hz_0.0002noi500s_0_capped.pdf}
\includegraphics[width=0.23\linewidth]{img/simulation_sigma_examples/fitcurve_50hz_0.001noi500s_0_capped.pdf}
\includegraphics[width=0.23\linewidth]{img/simulation_sigma_examples/fitcurve_50hz_0.1noi500s_0_capped.pdf}
\includegraphics[width=0.23\linewidth]{img/ISI_explanation.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_sigma_membrane_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_cv_membrane_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_psth_1ms_membrane_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_psth_5ms_membrane_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_sigma_refractory_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_cv_refractory_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_psth_1ms_refractory_50.pdf}
\includegraphics[width=0.23\linewidth]{img/cv_psth_sigma_compare/dataframe_scatter_labels_D_psth_5ms_refractory_50.pdf}
\caption{a)The parameter \(\sigma\) as a function of the noise parameter D in LIF-simulations. There is a strictly monotonous relationship between the two, which allows us to use \(\sigma\) as a susbtitute for D in the analysis of electrophysiological experiments.
b-e) Left to right: $\sigma$, CV and standard deviation of the psth with two diffrent kernel widths as a function of D for different membrane constants (4ms, 10ms and 16ms). The membrane constant $\tau$ determines how quickly the voltage of a LIF-neuron changes, with lower constants meaning faster changes. Only $\sigma$ does not change its values with different $\tau$. The CV (c)) is not even monotonous in the case of a timeconstant of 4ms, ruling out any potential usefulness.
f-i) Left to right: $\sigma$, CV and standard deviation of the psth with two diffrent kernel widths as a function of D for different refractory periods (0ms, 1ms and 5ms). Only $\sigma$ does not change with different refractory periods.
}
\label{noiseparameters2}
\end{figure}
We tried several different bin sizes (30 to 300 bins) and spike widths. There was little difference between the different parameters (see appendix).
\subsection*{Electrophysiology}
We can see from figure \ref{sigmafits_example} that the fits look very close to the data. Due to the gaussian signal distribution there are fewer samples for very weak and very strong inputs. In these regions the firing rates become somewhat noisy. This is especially noticeable for strong inputs, as there are more spikes there, and therefore large fluctuations, while there is very little spiking anyway for weak inputs.
\begin{figure}
\includegraphics[width=0.4\linewidth]{img/ordnung/cropped_fitcurve_0_2010-08-31-ad-invivo-1_0.pdf}
\includegraphics[width=0.4\linewidth]{img/ordnung/cropped_fitcurve_0_2010-08-11-aq-invivo-1_0.pdf}
% cropped_fitcurve_0_2010-08-31-ad-invivo-1_0.pdf: 0x0 px, 300dpi, 0.00x0.00 cm, bb=
\caption{Histogram of spike count distribution (firing rate) and errorfunction fits. 50 bins represent different values of the Gaussian distributed input signal [maybe histogram in background again]. The value of each of those bins is the number of spikes during the times the signal was in that bin. Each of the values was normalized by the signal distribution. To account for delay, we first calculated the cross-correlation of signal and spike train and took its peak as the delay. The lines show fits according to equation \eqref{errorfct}. Left and right plots show two different cells, one with a relatively narrow distribution and one with a distribution that is more broad, as indicated by the parameter \(\sigma\). Different amounts of bins (30 and 100) showed no difference in resulting parameters. \notedh{Show a plot.} \notedh{Show more than two plots?}}
\label{sigmafits_example}
\end{figure}
\begin{figure}
\includegraphics[width=0.65\linewidth]{img/sigma/cf_N_sigma.pdf}
\caption{Firing rate as a function of signal strength. Examples from experimental data.}
\label{Curve_examples}
\end{figure}
%TODO insert plot with sigma x-axis and delta_cf on y-axis here; also, plot with sigma as function of firing rate, also absoulte cf for different population size as function of sigma.
When we group neurons by their noise and plot coding fraction as a function of population size for averages of the groups, we see results similar to what we see for simulations.
Noisier cells have a lower coding fraction for small populations. For increasing population size, coding fraction increases for all groups, but the increase is much larger for noisy cells. For large population sizes the noisy cells show a better linear encoding of the signal than the more regular cells.
\begin{figure}
\includegraphics[width=0.45\linewidth]{img/ordnung/sigma_popsize_curves_0to300}
\caption{Left: Coding fraction as a function of population size for all recorded neurons. Color are by \(\sigma\) from the fit of the function in equation \ref{errorfct}, so that there are roughly an equal number of neurons in each category. Red: \(\sigma = \) 0 to 0.5, pink: 0.5 to 1.0, purple: 1.0 to 1.5, blue: 1.5 and above. Thick colored lines are average of the neurons in each group. For a population size of 1, coding fraction descreases on average with increasing \(\sigma\). As population sizes increase, coding fraction for weak noise neurons quickly stops increasing. Strong noise neurons show better coding performance for larger popuation sizes (about 8 to 32 neurons). Right [missing]: Increase in coding as a function of sigma. y-axis shows the difference in coding fraction between N=1 and N=32,}
\label{ephys_sigma}
\end{figure}
\subsection*{Determining the strength of noise in a real world example}
While in simulations we can control the noise parameter directly, we cannot do so in electrophysiological experiments.
Therefore, we need a way to quantify the intrinsic noise of the cell from the output of the measured cells. Common measures to quantify noisiness of neuronal spike trains are not directly correlated with intrinsic noise strength (figure \ref{noiseparameters}). An example for such a measure is the coefficient of variation (cv) of the interspike interval (ISI)\citep{white2000channel, goldberg1984relation,nowak1997influence}. The ISI is the time between each consecutive pair of spikes. The coefficient of variation is then defined as the standard deviation of the ISI divided by the mean ISI. Even though it is frequently used, we find that the cv as a function the intrinsic noise in our LIF-simulations is not necessarily monotonously related. In addition, for different membrane constants, which determine how quickly a neuron reacts to inputs, the same intrinsic noise results in widely different cv-values. Refractory periods also have an influence on the cv.
\notedh{their/holt1996 cv2 looks interesting.}
Another measure which has been used before is the standard variation of the peri-stimulus spike histogram
\citep{mainen1995reliability} \notedh{can't find any paper which did something like we did here, even though Schreiber et al. 2003 A new correlation-based measure of spike timing reliability - claim it's frequently done with psth}. This approach also does not work well, as it also depends on the membrane constant and to a lesser extend the refractory period.
The approach used here uses the activation curve of the neuron, fitting a function to it and extracting the parameters from the fitted function. It is assumed that the neurons show Gaussian noise. The mean of the distribution is the activation threshold and the width of the Gaussian is a measure for noise.
The probability of spiking as a function of the input is then the integral over the Gaussian, i.e. an error function.
Stocks (2000) uses one such function to simulate groups of noisy spiking neurons:
\begin{equation}
\label{errorfct}\frac{1}{2}\erfc\left(\frac{\theta-x}{\sqrt{2\sigma^2}}\right)
\end{equation}
where $\sigma$ is the parameter quantifying the noise (figure ?) %\ref{idealizedactivation}). \notejb{$\sigma$ quantifies the noise in units of the stimulus!!! THis is why this approach might work!}
A neuron with a $\sigma$ of 0 would be a perfect thresholding mechanism. Firing probability for all inputs below the threshold is 0, and firing probability for all inputs above is 1. If $\sigma$ is greater than 0, a neuron with such an activation curve will fire even for some signals below the firing threshold, while it will sometimes not fire for inputs above the firing threshold. For large values of $\sigma$ the activation curve becomes flatter, meaning the probability for inputs below the theshold eliciting a spike is large and the probability that an input above the threshold does not lead to firing is also large. The firing behaviour of such a cell is influenced less by the signal, which indicates noisiness.
However, for strong noise $(>10^{-2} \frac{mV^2}{Hz})$, results are not monotonous anymore. This happens at a point where $\sigma$ becomes large. Therefore, we excluded all values of the unit-less \(\sigma\) larger than two from the following analyses.
\subsection*{Methodology}
The signal was binned according to its amplitude. The result is a discrete Gaussian distribution around 0mV, the mean of the signal, as is expected from the way the signal was created.
After accounting for time delays in signal processing, we make a histogram which contains the distribution of spikes according to signal amplitude.
This histogram is then normalized by the distribution of the signal.
The result is another histogram, where values are firing frequencies for each signal value. Because those frequencies are just firing probabilities multiplied with equal time steps, we can fit a Gaussian error function to those probabilities.
\subsection*{Simulation}
To confirm that the $\sigma$ parameter estimated from the fit is indeed a good measure for the noisiness, we validated it against D, the noise parameter from the simulations. We find that there is a strictly monotonous relationship between the two for different sets of simulation parameters.
%Other parameters often used to determine noisiness (citations) such as the variance of the spike PSTH, the coefficient of variation (CV) of the interspike interval are not as useful. In figure \ref{noiseparameters} we see why. The variance of the psth is not always monotonous in D and is very flat for low values of D.
%describe what happens to the others
%check Fano-factor maybe?
We tried several different bin sizes (30 to 300 bins) and spike widths. There was little difference between the different parameters (see appendix).
\section*{-----------------------}
%We can use $\sigma$ instead of D*firing_rate: $\sigma$ makes it ind. of fr!!
\subsection*{Electrophysiology}
We find that the fits match the experimental data very well (figure \ref{sigmafits}). For very weak and very strong inputs, the firing rates themselves become noisy, because the signal only assumes those values rarely. This is especially noticeable for strong inputs, as there are more spikes there, and therefore large fluctuations, while there is very little spiking anyway for weak inputs.
% fish_raster.py on oilbird for the eventplot
% instructions.txt enth\"alt python-Befehle um Verteilungen und scatter zu rekonstruieren
\begin{figure}
\centering
\includegraphics[width=0.4\linewidth]{img/sigma/example_spikes_sigma.pdf}
\includegraphics[width=0.28\linewidth]{img/ordnung/cropped_fitcurve_0_2010-08-11-aq-invivo-1_0.pdf}
\includegraphics[width=0.28\linewidth]{img/sigma/cropped_fitcurve_0_2010-08-31-aj-invivo-1_0.pdf}
\notedh{Daraus ergibt sich nicht direkt eine Intuition, wieso das noisy ist. H\"angt einfach sehr am Eingangssignal; wenn es im eventplot (un-)regelm\"a\ss{}ig w\"are, k\"onnten wir auch einfach cv nehmen...}\\
% \includegraphics[width=0.28\linewidth]{img/ordnung/cropped_fitcurve_0_2010-08-31-ad-invivo-1_0.pdf}
\includegraphics[width=0.4\linewidth]{img/fish/dataframe_scatter_sigma_cv.pdf}
\includegraphics[width=0.4\linewidth]{img/fish/dataframe_scatter_sigma_firing_rate.pdf}
\includegraphics[width=0.32\linewidth]{img/fish/sigma_distribution.pdf}
\includegraphics[width=0.32\linewidth]{img/fish/cv_distribution.pdf}
\includegraphics[width=0.32\linewidth]{img/fish/fr_distribution.pdf}
% cropped_fitcurve_0_2010-08-31-ad-invivo-1_0.pdf: 0x0 px, 300dpi, 0.00x0.00 cm, bb=
\caption{Histogram of spike count distribution (firing rate) and errorfunction fits. 50 bins represent different values of the Gaussian distributed input signal [maybe histogram in background again]. The value of each of those bins is the number of spikes during the times the signal was in that bin. Each of the values was normalized by the signal distribution. For very weak and very strong inputs, the firing rates themselves become noisy, because the signal only assumes those values rarely. To account for delay, we first calculated the cross-correlation of signal and spike train and took its peak as the delay. The lines show fits according to equation \eqref{errorfct}. Left and right plots show two different cells, one with a relatively narrow distribution (left) and one with a distribution that is more broad (right), as indicated by the parameter \(\sigma\). An increase of $\sigma$ is equivalent to an broader distribution. Cells with broader distributions are assumed to be noisier, as their thresholding is less sharp than those with narrow distributions. Different amounts of bins (30 and 100) showed no difference in resulting parameters.}
\label{sigmafits}
\end{figure}
% TODO insert plot with sigma x-axis and delta_cf on y-axis here; also, plot with sigma as function of firing rate, also absoulte cf for different population size as function of sigma.
When we group neurons by their noise and plot coding fraction as a function of population size for averages of the groups, we see results similar to what we see for simulations (figure \ref{ephys_sigma} a)):
Noisier cells (larger $\sigma$, purple) have a lower coding fraction for small populations. However, coding fraction mostly stops increasing with population sizes once a population size of about 16 is reached. The increase is much larger for noisy cells (orange). The averages of the coding fraction for the noisy cells does not increase above the coding fraction of the less noisy cells for the population sizes investigated here (N=128). In contrast to the more regular cells, coding fraction is still improving for the noisy cells, so it is plausible that at a certain population size the noisy cells can outperform the less noisy cells.
Indeed, if results are not averaged and single cells are considered, we find that for large population sizes the noisy cells show a better linear encoding of the signal than the more regular cells (figure \ref{ephys_sigma} b), red).
%figures created with box_script.py
\begin{figure}
%\includegraphics[width=0.45\linewidth]{img/ordnung/sigma_popsize_curves_0to300}
\centering
%\includegraphics[width=0.45\linewidth]{img/sigma/cf_N_ex_lines}
\includegraphics[width=0.45\linewidth]{img/sigma/sigma_cf_quot.pdf}%
\includegraphics[width=0.45\linewidth]{img/sigma/check_fr_quot.pdf}%
\caption{Left: Coding fraction as a function of population size for all recorded neurons. Cells are grouped by \(\sigma\) from the fit of the function in equation \ref{errorfct}. Lines are averages over three cells each, with the shading showing the standard deviation. For stronger noise, coding fraction is far smaller for a single neuron. With increasing population size, coding fraction increases much faster for the noisy cells than for the less noisy cells.
Right: Examples for the two cells with lowest, intermediate and highest $\sigma$. For a population size of N=1, the cell with the largest $\sigma$ (brown) has the lowest coding fraction out of all the cells here. The coding fraction of that cell increases hugely with population size. At a population of N=128, coding fraction is second highest among the pictured cells.}
\label{ephys_sigma}
\end{figure}
%The value of $\sigma$ is not signal independent. The same cell can have different values for $\sigma$ for different input signals.
\input{sigma}
\subsection{Results}