dennis/role_of_noise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add more parts

Browse Source
This commit is contained in:
Dennis Huben 2024-08-02 11:54:03 +02:00
parent 824a06aeff
commit 4da718524e
5 changed files with 193 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -5,4 +5,4 @@
*.dvi
*.toc
.gz
*swp #sic!
*swp

1
bands.tex
View File

@ -1,4 +1,3 @@
\subsection*{Narrowband stimuli}
Using the \(f_{cutoff} = 200 \hertz\usk\) signal, we repeated the analysis for only a part of the spectrum. We did so for two "low frequency" (0--8Hz, 0--50Hz) and two "high frequency" (192--200Hz, 150--200Hz) intervals. We then compared the results to the results we get from narrowband stimuli, with power only in those frequency bands.
To keep the power of the signal inside the two intervals the same as in the broadband stimulus, amplitude of the narrowband signals was less than that of the broadband signal. For the 8Hz intervals, amplitude (i.e. standard deviation) of the signal was 0.2mV, or a fifth of the amplitude of the broadband signal. Because signal power is proportional to the square of the amplitude, this was appropriate for a stimulus with a spectrum 25 times smaller. Similarly, for the 50Hz intervals we used a 0.5mV amplitude, or half of that of the broadband stimulus.

BIN
main.pdf
View File

Binary file not shown.

BIN
main.synctex.gz
View File

Binary file not shown.

196
main.tex
View File

@ -270,20 +270,208 @@ The increase can be explained as the reverse of the effect that leads to decreas
\textbf{C-E}: Coding fraction as a function of signal amplitude for different tuningcurves (noise levels). Three different means, one below the threshold (9.5mV), one at the threshold (10.0mV), and one above the threshold (10.5mV).}
\end{figure}
\section*{Discussion}
\input{simulation_further_considerations}
In this paper we have shown the effect of Suprathreshold Stochastic Resonance (SSR) in ensembles of neurons. We detailed how noise levels affect the impact of population size on the coding fraction. We looked at different frequency ranges and could show that the encoding of high-frequency signals profits particularly well from SSR. Using the tuningcurve we were able to provide a way to extrapolate the effects of SSR for very large populations. Because in general analysis of the impact of changing parameters is complex, we investigated limit cases, in particular the slow stimulus limit and the weak stimulus limit. For low-frequency signals, i.e. the slow stimulus limit, the tuningcurve also allows analyzing the impact of changing signal strength; in addition we were able to show the difference in sub-threshold SR and SSR for different noise levels. For the weak stimulus limit, where noise is relatively strong compared to the signal, we were able to provide an analytical solution for our observations.
\citep{hoch2003optimal} also shows that SSR effects hold for both LIF- and HH- Neurons. However, Hoch et al. have found that optimal noise level depends "close to logarithmatically" on the number of neurons in the population. They used a cutoff frequency of only 20Hz for their simulations. \notedh{Hier fehlt ein plot, der Population size und optimum noise in Verbindung setzt}
We investigated the impact of noise on homogeneous populations of neurons. Neurons being intrinsically noisy is a phenomenom that is well investigated (Grewe et al 2017, Padmanabhan and Urban 2010).
In natural systems however, neuronal populations are rarely homegeneous. Padmanabhan and Urban (2010) showed that heterogeneous populations of neurons carry more information that heterogenous populations.
%\notedh{Aber noisy! Zitieren: Neurone haben intrinsisches Rauschen (Einleitung?)} (Grewe, Lindner, Benda 2017 PNAS Synchronoy code) (Padmanabhan, Urban 2010 Nature Neurosci).
Beiran et al. (2017) investigated SSR in heterogeneous populations of neurons. They made a point that heterogeneous populations are comparable to homogeneous populations where the neurons receive independent noise in addition to a deterministic signal. They make the point that in the case of weak signals, heterogeneous population can encode information better, as strong noise would overwhelm the signal.
\notedh{Unterschiede herausstellen!} Similarly, Hunsberger et al. (2014) showed that both noise and heterogeneity linearize the tuning curve of LIF neurons.
In summary, while noise and heterogeneity are not completely interchangeable. In the limit cases we see similar behaviour.
\citep{Sharafi2013} Sharafi et al. (2013) had already investigated SSR in a similar way. However, they only observed populations of up to three neurons and were focused on the synchronous output of cells. They took spike trains, convolved those with a gaussian and then multiplied the response of the different neurons. In our simulations we instead used the addition of spike trains to calculate the cohenrece between input and output. Instead of changing the noise parameter to find the optimum noise level, they changed the input signal frequency to find a resonating frequency, which was possible for suprathreshold stochastic resonance, but not for subthreshold stochastic resonance. For some combinations of parameters we also found that coding fraction does not decrease monotonically with increasing signal frequency (fig. \ref{cf_for_frequencies}).
It is especially notable for signals that are far from the threshold (fig \ref{cf_for_frequencies} E,F (red markers)).
That we don't see the effect that clearly matches Sharafi et al.'s observation that in the case of subthreshold stochastic resonance, coherence monotonically decreased with increasing frequency. Pakdaman et al. (2001)
\notedh{Besser verkn\"upfen als das Folgende (vergleichen \"uber Gr\"o\ss{}enordnungen; vergleichen mit Abbildung 5\ref{}; mehr als Sharafi zitieren Stichwort ``Coherence Resonance''}
Similar research to Sharafi et al. was done by (de la Rocha et al. 2007). They investigated the output correlation of populations of two neurons and found it increases with firing rate. We found something similar in this paper, where an increase in $\mu$ increases both the firing rate of the neurons and generally also the coding fraction \notedh{Verkn\"upfen mit output correlation}(fig. \ref{codingfraction_means_amplitudes}). Our explanation is that coding fraction and firing rate are linked via the tuningcurve. In addition to simulations of LIF neurons de la Rocha et al. also carried out \textit{in vitro} experiments where they confirmed their simulations.
\notedh{Konkreter machen: was machen die Anderen, das mit uns zu tun hat und was genau hat das mit uns zu tun?}
\notedh{Vielleicht nochmal Stocks, obwohl er schon in der Einleitung vorkommt? Heterogen/homogen}
\notedh{Dynamische stimuli! Bei Stocks z.B. nicht, nur z.B. bei Beiran. Wir haben den \"Ubergang.}
Examples for neuronal systems that feature noise are P-unit receptor cells of weakly electric fish (which paper?) and ...
In the case of low cutoff frequency and strong noise we were able to derive a formula that explains why in those cases coding fraction simply depends on the ratio between noise and population size, whereas generally the two variables have very different effects on the coding fraction.
\subsection{Different frequency ranges}
\subsection{Narrow-/wideband}
\subsection*{Narrow-/wideband}
\subsection*{Narrowband stimuli}
Using the \(f_{cutoff} = 200 \hertz\usk\) signal, we repeated the analysis (fig. \ref{cf_limit}) considering only selected parts of the spectrum. We did so for two "low frequency" (0--8Hz, 0--50Hz) and two "high frequency" (192--200Hz, 150--200Hz) intervals.\notedh{8Hz is not in yet.} We then compared the results to the results we get from narrowband stimuli, with power only in those frequency bands.
To keep the power of the signal inside the two intervals the same as in the broadband stimulus, amplitude of the narrowband signals was less than that of the broadband signal. For the 8Hz intervals, amplitude (i.e. standard deviation) of the signal was 0.2mV, or a fifth of the amplitude of the broadband signal. Because signal power is proportional to the square of the amplitude, this was appropriate for a stimulus with a spectrum 25 times smaller. Similarly, for the 50Hz intervals we used a 0.5mV amplitude, or half of that of the broadband stimulus.
As the square of the amplitude is equal to the integral over the frequency spectrum, for a signal with a quarter of the width we need to half the amplitude to have the same power in the interval defined by the narrowband signals.
\subsection*{Smaller frequency intervals in broadband signals }
\begin{figure}
\includegraphics[width=0.45\linewidth]{img/small_in_broad_spectrum}
\includegraphics[width=0.45\linewidth]{img/power_spectrum_0_50}
\includegraphics[width=0.49\linewidth]{{img/broad_coherence_15.0_1.0}.pdf}
\includegraphics[width=0.49\linewidth]{{img/coherence_15.0_0.5_narrow_both}.pdf}
\includegraphics[width=0.49\linewidth]{{img/broad_coherence_10.5_1.0_200}.pdf}
\includegraphics[width=0.49\linewidth]{{img/coherence_10.5_0.5_narrow_both}.pdf}
\caption{Coherence for broad and narrow frequency range inputs. a) Broad spectrum.
At the frequency of the firing rate (91Hz, marked by the black bar) and its first
harmonic (182Hz) the coding fraction breaks down. For the weak noise level (blue),
population sizes n=4 and n=4096 show indistinguishable coding fraction.
In case of a small population size, coherence is higher for weak noise (blue) than
for strong noise (green) in the frequency range up to about 50\hertz. For higher
frequencies coherence is unchanged. For the case of the larger population size and the
greater noise strength there is a huge increase in the coherence for all frequencies.
b) Coherence for two narrowband inputs with different frequency ranges.
Low frequency range: coherence for
slow parts of the signal is close to 1 for weak noise. SSR works mostly on the
higher frequencies (here >40\hertz). High frequency range: At 182Hz (twice
the firing frequency) there is a very sharp decrease in coding fraction,
especially for the weak noise condition (blue). Increasing the noise makes the drop
less clear. For weak noise (blue) there is another break down at 182-(200-182)Hz.
Stronger noise seems to make this sharp drop disappear. Again, the effect of SSR
is most noticeable for the higher frequencies in the interval.\notedh{Add description for 10.5mV}}
\label{fig:coherence_narrow}
\end{figure}
We want to know how well encoding works for different frequency intervals in the signal.
When we take out a narrower frequency interval from a broadband signal, the other
frequencies in the signal serve as common noise to the neurons encoding the signal.
In many cases we only care about a certain frequency band in a signal of much wider bandwidth.
In figure \ref{fig:coherence_narrow} C we can see that SSR has very different
effects on some frequencies inside the signal than on others. In blue we see the
case of very weak noise (\(10^{-6} \milli\volt\squared\per\hertz\)). Coherence starts somewhat close to 1 but falls off quickly that it reaches about 0.5 by 50Hz and goes down to almost zero around the 91Hz firing rate of the signal. Following that there is a small increase up to about 0.1 at around 130Hz, after which coherence decreases to almost 0.
Increasing the population size from 4 neurons to 2048 neurons has practically no effect.
When we keep population size at 4 neurons, but
add more noise to the neurons (green, \(2\cdot10^{-3} \milli\volt\squared\per\hertz\)),
encoding of the low frequencies (up to about 50\hertz) becomes worse, while
encoding of the higher frequencies stays unchanged. When we increase the population
size to 2048 neurons we have almost perfect encoding for frequencies up to 50\hertz.
Coherence is still reduced around the average firing rate of the neurons, but at
a much higher level than before. For higher frequencies coherence becomes higher again.
For the weaker mean input (figure \ref{coherence_narrow} E results look similar. For weak noise (blue) there is no difference for the increased population. Coherence starts relatively high again (around 0.7). There is a decrease in coherence for increasing frequency which is steep at first, until about the firing rate of the neuron, after which the decrease flattens off. For stronger noise, encoding at low frequencies is worse for small populations; for large populations the coherence is greatly increased for all frequencies. Coherence is very close to 1 at first, decreases slightly in the frequency is increased up to the firing rate, after which coherence stays about constant.
In summary, the high frequency bands inside the broadband stimulus experience a
much greater increase in encoding quality than the low frequency bands,
which were already encoded quite well.
\begin{figure}
\includegraphics[width=0.45\linewidth]{img/broadband_optimum_newcolor.pdf}
\includegraphics[width=0.45\linewidth]{img/smallband_optimum_newcolor.pdf}
\centering
\includegraphics[width=0.9\linewidth]{img/max_cf_smallbroad.pdf}
\caption{
C and D \notedh{B and C right now because the order in the right column was mixed up}: Best amount of noise for different number of neurons. The dashed lines show where coding fraction still is at least 95\% from the maximum. The width of the peaks is much larger for the narrowband signals which encompasses the entire width of the high-frequency interval peak.
Optimum noise values for a fixed number of neurons are always higher for the broadband signal than for narrowband signals.
In the broadband case, the optimum amount of noise is larger for the high-frequency interval than for the low-frequency interval and vice-versa for the narrowband case. %The optimal noise values have been fitted with a function of square root of the population size N, $f(N)=a+b\sqrt{N}$. We observe that the optimal noise value grows with the square root of population size.
E and F: Coding fraction as a function of noise for a fixed population size (N=512). Red dots show the maximum, the red line where coding fraction is at least 95\% of the maximum value.
G: An increase in population size leads to a higher coding fraction especially for broader bands and higher frequency intervals. Coding fraction is
larger for the narrowband signal than in the equivalent broadband interval for all neural population sizes considered here. The coding fraction for the low frequency intervals is always larger than for the high frequency interval.
Signal mean $\mu=15.0\milli\volt$, signal amplitude $\sigma=1.0\milli\volt$ and $\sigma=0.5\milli\volt$ respectively.}
\label{smallbroad}
\end{figure}
\input{bands}
\subsection*{Narrowband Signals vs Broadband Signals}
In nature, often an external stimulus covers a narrow frequency range
that starts at high frequencies, so that only using broadband white noise signals
as input appears to be insufficient to describe realistic scenarios.\notedh{Add examples.}
%, with bird songs\citep{nottebohm1972neural} and ???\footnote{chirps, in a way?}.
%We see that in many animals receptor neurons have adapted to these signals. For example, it was found that electroreceptors in weakly electric fish have band-pass properties\citep{bastian1976frequency}.
Therefore, we investigate the coding of narrowband signals in the ranges
described earlier (0--50Hz, 150--200Hz). Comparing the results from coding of
broadband and coding of narrowband signals, we see several differences.
For both low and high frequency signals, the narrowband signal
can be resolved better than the broadband signal for any amount of noise and at all population sizes (figure \ref{smallbroad}, bottom left).
That coding fractions are higher when we use narrowband signals can be
explained by the fact that the additional
frequencies in the broadband signal are now absent. In the broadband signal
they are a form of "noise" that is common to all the input neurons.
Similar to what we saw for the broadband signal,
the peak of the low frequency input is still much more broad than the peak of the high frequency input.
To encode low frequency signals the exact strength of the noise is not as
important as it is for the high frequency signals which can be seen from the wider peaks.
\subsection{Discussion}
The usefulness of noise on information encoding of subthreshold signals by single neurons has been well investigated. However, the encoding of supra-threshold signals by populations of neurons has received comparatively little attention and different effects play a role for suprathreshold signals than for subthreshold signals \citep{Stocks2001}. This paper delivers an important contribution for the understanding of suprathreshold stochastic resonance (SSR). We simulate populations of leaky integrate-and-fire neurons to answer the question how population size influences the optimal noise strength for linear encoding of suprathreshold signals. We are able to show that this optimal noise is well described as a function of the square root of population size.\notedh{Currently missing, but it is somewhere in my notes ...} This relationship is independent of frequency properties of the input signal and holds true for narrowband and broadband signals.
In this paper, we show that SSR works in LIF-neurons for a variety of signals of different bandwidth and frequency intervals. We show that signal-to-noise ratio is for signals above a certain strength sufficient to describe the optimal noise strength in the population, but that the actual coding fraction depends on the absolute value of signal strength. %We furthermore show that increasing signal strength does not always increase the coding fraction.
We contrast how well the low and high frequency parts of a broadband signal can be encoded. We take an input signal with $f_{cutoff} = \unit{200}\hertz$ and analyse the coding fraction for the frequency ranges 0 to \unit{50}\hertz\usk and 150 to \unit{200}\hertz\usk separately. The maximum value of the coding fraction is lower for the high frequency interval compared to the low frequency interval. This means that inside broadband signals higher frequencies intervals appear more difficult to encode for each level of noise and population size. The low frequency interval has a wider peak (defined as 95\% coding fraction of its coding fraction maximum value), which means around the optimal amount of noise there is a large area where coding fraction is still good. The noise optimum for the low frequency parts of the input is lower than the optimum for the high frequency interval (Fig. \ref{highlowcoherence}). In both cases, the optimal noise value appears to grow with the square root of population size.\notedh{See note above}
In general, narrowband signals can be encoded better than broadband signals.
narrowband vs broadband
Another main finding of this paper is the discovery of frequency dependence of SSR.
We can see from the shape of the coherence between the signal and the output of the simulated
neurons, SSR works mostly for the higher frequencies in the signal. As the lower frequency
components are in many cases already encoded really well, the addition of noise
helps to flatten the shape of the coherence curve. In the case of weak noise, often there
are border effects which disappear with increasing strength of the noise.
In addition, for weak noise there are often visible effects from the firing rate of the neurons, in so far that the encoding
around those frequencies is worse than for the surrounding frequencies. Generally
this effect becomes less pronounced when we add more noise to the simulation, but
we found a very striking exception in the case of narrowband signals.
Whereas for a firing rate of about
91\hertz\usk the coding fraction of the encoding of a signal in the 0-50\hertz\usk band is
better than for the encoding of a signal in the 150-200\hertz\usk band. However, this is
not the case if the neurons have a firing rate about 34\hertz.
We were thus able to show that the firing rate on the neurons in the simulation is of
critical importance to the encoding of the signal.
\section{Theory}
\subsection{Firing rates}
\input{calculation}
\subsection*{For large population sizes and strong noise, coding fraction becomes a function of their quotient}
For the linear response regime of large noise, we can estimate the coding fraction. From Beiran et al. 2018 we know the coherence in linear response is given as
\eq{
C_N(\omega) = \frac{N|\chi(\omega)|^2 S_{ss}}{S_{x_ix_i}(\omega)+(N1)|\chi(\omega)|^2S_{ss}}
\label{eq:linear_response}
}
where \(C_1(\omega)\) is the coherence function for a single LIF neuron. Generally, the single-neuron coherence is given by \citep{??}
\eq{
C_1(\omega)=\frac{r_0}{D} \frac{\omega^2S_{ss}(\omega)}{1+\omega^2}\frac{\left|\mathcal{D}_{i\omega-1}\big(\frac{\mu-v_T}{\sqrt{D}}\big)-e^{\Delta}\mathcal{D}_{i\omega-1}\big(\frac{\mu-v_R}{\sqrt{D}}\big)\right|^2}{\left|{\cal D}_{i\omega}(\frac{\mu-v_T}{\sqrt{D}})\right|^2-e^{2\Delta}\left|{\cal D}_{i\omega}(\frac{\mu-v_R}{\sqrt{D}})\right|^2}
\label{eq:single_coherence}
}
where \(r_0\) is the firing rate of the neuron,
\[r_0 = \left(\tau_{ref} + \sqrt{\pi}\int_\frac{\mu-v_r}{\sqrt{2D}}^\frac{\mu-v_t}{\sqrt{2D}} dz e^{z^2} \erfc(z) \right)^{-1}\].
In the limit of large noise (calculation in the appendix) this equation evaluates to:
\eq{
C_1(\omega) = \sqrt{\pi}D^{-1}
\frac{S_{ss}(\omega)\omega^2/(1+\omega^2)}{2 \sinh\left(\frac{\omega\pi}{2}\right)\Im\left( \Gamma\left(1+\frac{i\omega}{2}\right)\Gamma\left(\frac12-\frac{i\omega}{2}\right)\right)}
\label{eq:simplified_single_coherence}
}
From eqs.\ref{eq:linear_response} and \ref{eq:simplified_single_coherence} it follows that in the case \(D \rightarrow \infty\) the coherence, and therefore the coding fraction, of the population of LIF neurons is a function of \(D^{-1}N\). We plot the approximation as a function of \(\omega\) (fig. \ref{d_n_ratio}). In the limit of small frequencies the approximation matches the exact equation very well, though not for higher frequencies. We can verify this in our simulations by plotting coding fraction as a function of \(\frac{D}{N}\). We see (fig. \ref{d_n_ratio}) that in the limit of large D, the curves actually lie on top of each other. This is however not the case (fig. \ref{d_n_ratio}) for stimuli with a large cutoff frequency \(f_c\), as expected by our evaluation of the approximation as a function of the frequency.
\begin{figure}
\centering
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_10.5_0.5_10_detail}.pdf}
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_15.0_0.5_50_detail}.pdf}
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_15.0_1.0_200_detail}.pdf}
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_10.5_0.5_10_detail}.pdf}
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_15.0_0.5_50_detail}.pdf}
\includegraphics[width=0.32\linewidth]{{img/d_over_n/d_over_n_15.0_1.0_200_detail}.pdf}
\label{d_n_ratio}
\caption{Top row: Coding fraction as a function of noise.
Bottom row: Coding fraction as a function of the ratio between noise strength and population size. For strong noise, coding fraction is a function of this ratio.
Left: signal mean 10.5mV, signal amplitude 0.5mV, $f_{c}$ 10Hz.
Middle: signal mean 15.0mV, signal amplitude 0.5mV, $f_{c}$ 50Hz.
Right: signal mean 15.0mV, signal amplitude 1.0mV, $f_{c}$ 200Hz.}
\end{figure}
\input{firing_rate}