Time to sleep for now.

main.tex

@@ -1805,60 +1805,110 @@ adaptation. But the trade-off between intensity invariance and SNR likely goes
 beyond the particular mechanisms along the pathway. After all, a transformation
 is not expected to compress a range of different input intensities into a
 constant output intensity without sacrificing some of the corresponding input
-SNR. This suggests that the trade-off is a more general principle that applies
-to any transformation that achieves or improves intensity invariance.
+SNR. Accordingly, the trade-off likely is a more general principle that might
+apply to any transformation that achieves or improves intensity invariance.
 
+% Dependence of thresh-LP intensity invariance on threshold value (+unlimited SNR):
 The second mechanism of intensity invariance consists of thresholding and
 temporal averaging of $c_i(t)$ into $f_i(t)$. Here, the trade-off between
-intensity invariance and SNR is mediated by the threshold value $\thr$. A lower
-$\thr$~($\thr\to0$) improves the intensity invariance of $f_i(t)$ by shifting
-the saturation point towards lower $\sca$. However, a lower $\thr$ also raises
-the noise floor of $f_i(t)$ by including more of the pure-noise $c_i(t)$, which
-decreases the SNR of $f_i(t)$. The distribution $\pci$ of the pure-noise
-$c_i(t)$ is very close to a normal distribution with mean $\mu\approx0$ for all
-kernels $k_i(t)$. The value of the pure-noise $f_i(t)$ is hence 0.5 for
-$\thr=0$ and decreases for higher $\thr$. If $\thr$ is set above the maximum of
-$c_i(t)$, the pure-noise feature value is 0, which results in an "unlimited"
-SNR of $f_i(t)$. In this case, any non-zero feature value that is sustained for
-a sufficient duration could serve as indicator for the presence of $\soc(t)$ in
-$\raw(t)$, although at the cost of a higher saturation point. Of course, this
-would require a fine evolutionary tuning of $\thr$ to the properties of the
-natural noise in a certain habitat to avoid false positives.
+intensity invariance and SNR is mediated by the threshold value $\thr$. The
+effects of $\thr$ on the intensity invariance and SNR of $f_i(t)$ are best
+assessed if the mechanism is viewed in isolation. A lower $\thr$~($\thr\to0$)
+improves the intensity invariance of $f_i(t)$ by shifting the saturation point
+towards lower $\sca$. The saturation level of $f_i(t)$ is mostly independent of
+$\thr$, assuming that $\sca$ is sufficiently large. However, the lower $\thr$,
+the more of the pure-noise $c_i(t)$ is included in $f_i(t)$ and hence the
+higher the noise floor of $f_i(t)$, which decreases the SNR of $f_i(t)$. The
+distribution $\pci$ of the pure-noise $c_i(t)$ is very close to a normal
+distribution with mean $\mu\approx0$ for all kernels in the set. The value of
+the pure-noise $f_i(t)$ is hence 0.5 for $\thr=0$ and decreases for higher
+$\thr$. If $\thr$ is set above the maximum of $c_i(t)$, the pure-noise feature
+value is 0, which results in an "unlimited" SNR of $f_i(t)$. In this case, any
+non-zero feature value that is sustained for a sufficient duration could serve
+as indicator for the presence of $\soc(t)$ in $\raw(t)$. Of course, this would
+require a fine evolutionary tuning of $\thr$ to the properties of the natural
+noise in a certain habitat in order to avoid false positives.
 
-The saturation level of $f_i(t)$ is independent of $\thr$ as long as the
-intensity invariance by the previous mechanism is neglected.
+% Interaction between the two mechanisms of intensity invariance (expectations):
+% (Also: Extremely important, but maybe too wordy?)
+The intensity invariance of $f_i(t)$ is not only determined by the second
+mechanism but by the interaction between the two consecutive mechanisms along
+the pathway. This interaction is difficult to assess systematically due to the
+multitude of involved parameters. A basic expectation is that the combined
+effects of the two mechanisms mostly depend on which mechanism achieves a lower
+saturation point, assuming that $f_i(t)$ is always intensity-invariant if
+$\adapt(t)$ is already intensity-invariant. Furthermore, it is necessary to
+distinguish between the intrinsic saturation point of $f_i(t)$ --- the
+saturation point that the second mechanism can achieve in isolation --- and its
+actual saturation point including the effects of the first mechanism. The same
+distinction applies to the saturation level of $f_i(t)$. If the intrinsic
+saturation point of $f_i(t)$ is lower than the saturation point of $\adapt(t)$,
+$f_i(t)$ is expected to reach the intrinsic saturation level at the intrinsic
+saturation point. In contrast, if the intrinsic saturation point of $f_i(t)$ is
+higher than the saturation point of $\adapt(t)$, $f_i(t)$ is expected to
+saturate at the lower saturation point of $\adapt(t)$ instead. This has no
+detrimental effect on the intensity invariance of $f_i(t)$. However, $f_i(t)$
+is then also expected to saturate below its intrinsic saturation level.
+Moreover, the saturation level of $f_i(t)$ is not independent of $\thr$ anymore
+but decreases with increasing $\thr$. A lower saturation level of $f_i(t)$ is
+not necessarily detrimental to the SNR of $f_i(t)$ --- $f_i(t)$ can still
+achieve an arbitrarily high SNR by setting $\thr$ just above the maximum
+pure-noise $c_i(t)$. More importantly, a lower saturation level of $f_i(t)$
+also means that the range of possible feature values that $f_i(t)$ can take on
+is limited compared to the case where $f_i(t)$ can reach its intrinsic
+saturation level. This effectively restricts the part of the feature space that
+is available for species-specific song representation. The interaction between
+the two mechanisms of intensity invariance could therefore have unfavorable
+consequences if the first mechanism results in a lower saturation point than
+the second mechanism.
 
+% Interaction between the two mechanisms of intensity invariance (current results):
+The saturation point and saturation level of a feature in the set varies with
+the specific kernel.
 
-If $f_i(t)$ can achieve an arbitrarily high SNR, it can counteract the
-comparably low SNR of $\adapt(t)$ 
+The combined effects of the two mechanisms on the intensity invariance of a
+specific feature in the set vary between different kernels
 
-The maximum of the pure-noise $c_i(t)$ is assumed to be very
-small due to the various SNR improvements along the pathway, so that the
-required increase in $\thr$ and hence the saturation point of $f_i(t)$ is not
-expected to be substantial.
+Based on the current results, it is difficult to assess which of the two
+mechanisms has a stronger effect on the intensity invariance of a specific
+feature in the set.
 
-
-% \newpage
-% \subsection{Intensity invariance versus intensity invariance}
-
-Two consecutive mechanisms of intensity invariance do not necessarily add up to
-a stronger overall intensity invariance. If the first mechanism results in a
-lower saturation point than the second mechanism by itself, the saturation
-point of feature $f_i(t)$ will be determined solely by the first mechanism. In
-this case, the saturation level of $f_i(t)$ will conform to the intensity that
-$f_i(t)$ can reach for the given saturation point rather than the intrinsic
-saturation level of $f_i(t)$. Conversely, if the second mechanism results in a
-lower saturation point than the first mechanism, both the saturation point and
-the saturation level of $f_i(t)$ will be determined by the second mechanism.
-The saturation points of $f_i(t)$ across the set are distributed over a much
-wider range than those of the preceeding kernel responses $c_i(t)$, which
-suggests that the interaction between the two mechanisms is specific to
-individual kernels. A number of $f_i(t)$ achieve a lower saturation point than
-the respective $c_i(t)$, whereas some $f_i(t)$ exhibit similar or only
-marginally lower saturation points. In these cases, the question arises to what
+The combined effects of the two mechanisms on the intensity invariance of a
+specific feature in the set vary between different kernels. It is difficult to
+assess which of the two mechanisms achieves a lower saturation point for a
+specific feature. On the one hand, the distribution of saturation levels across
+the feature set is not symmetric around a feature value of 0.5, which is the
+case if the logarithmic compression along the pathway is disabled. This result
+indicates that a number of features does not reach the intrinsic saturation
+level, which suggests that the intensity invariance of these features is
+determined by the first mechanism rather than the second mechanism. One the
+other hand, the distribution of saturation points across the feature set
+indicates that a number of features does indeed achieve a lower saturation
+point than the preceeding representations. This result suggests that the
+intensity invariance of these features is determined by the second mechanism
+rather than the first mechanism. In either case, the question arises to what
 extent two consecutive mechanisms of intensity invariance are actually
 beneficial for the overall system.
 
+These cases raise the question whether the first mechanism is actually
+necessary for the overall system if the second mechanism can apparently achieve
+intensity invariance with a lower saturation point. These cases raise the
+question whether intensity invariance by the first mechanism --- while
+achieving a lower saturation point than the second mechanism --- is actually
+beneficial
+
+The saturation point of $f_i(t)$ varies between different kernels in the set. A
+number of $f_i(t)$ achieve a lower saturation point than $c_i(t)$ --- and hence a lower saturation point than $\adapt(t)$ --- which
+indicates that the second mechanism takes precedence over the first mechanism.
+Some $f_i(t)$ exhibit similar or only marginally lower saturation points than
+
+The saturation points of $f_i(t)$ across the set are distributed over a much
+wider range than those of the preceeding $c_i(t)$, 
+
+In these cases, the question arises to what extent two
+consecutive mechanisms of intensity invariance are actually beneficial for the
+overall system.
+
 From a computational perspective, the answer could be that logarithmic
 compression and adaptation is a necessary preprocessing step towards robust
 $f_i(t)$ because it works towards a more consistent distribution $\pci$ of

python/fig_invariance_full.py

@@ -442,6 +442,7 @@ for stage in stages:
         
     # Indicate saturation point(s):
     if stage in ['log', 'inv', 'conv', 'feat']:
+        # Get and plot single curve saturation point:
         ind = get_saturation(curve, **plateau_settings)[1]
         crit_inds[stage] = ind
         scale = scales[ind]
@@ -452,6 +453,13 @@ for stage in stages:
                          transform=raw_axes[0].get_xaxis_transform())
         raw_axes[0].vlines(scale, raw_axes[0].get_ylim()[0], curve[ind],
                            color=color, **plateau_line_kwargs)
+        if stage in ['conv', 'feat']:
+            # Get and log distribution of swarm saturation points:
+            inds = np.array(get_saturation(measure, **plateau_settings)[1])
+            if np.isnan(inds).sum():
+                print('WARNING: Found NaN saturation point(s)!')
+                inds = inds[~np.isnan(inds)].astype(int)
+            crit_scales_swarm[stage] = scales[inds]
 
     ## NORMALIZED MEASURE:
 
@@ -476,11 +484,6 @@ for stage in stages:
         fill_kwargs = dist_fill_kwargs | dict(color=color)
         y_dist(base_insets[i1], measure[-1], nbins=100, log=True,
                line_kwargs=line_kwargs, fill_kwargs=fill_kwargs)
-        # Get and log distribution of saturation points:
-        inds = np.array(get_saturation(measure, **plateau_settings)[1])
-        if np.isnan(inds).sum():
-            inds = inds[~np.isnan(inds)].astype(int)
-        crit_scales_swarm[stage] = scales[inds]
         if stage == 'feat':
             # Plot distribution of saturation points on shared bins:
             bin_lims = [0.01, 1.1 * max([s.max() for s in crit_scales_swarm.values()])]