cleaned up the root dir

2023-04-06 10:15:48 +02:00 · 2023-04-06 10:15:48 +02:00 · cdb4b05605
commit cdb4b05605
parent 027bacb8c0
19 changed files with 72 additions and 331 deletions
--- a/.gitignore
+++ b/.gitignore
@ -5,6 +5,7 @@
 data
 env
 output
 trash
 # Mac Stuff
 *.DS_Store
--- a/.python-version
+++ b/.python-version
@ -1 +1 @@
-chirpdet
+chirpdetection
--- a/script.md
+++ b/script.md
@ -1,141 +0,0 @@
 ---
 date: 2023-01-26
 author: Patrick Weygoldt
 type: talk
 speakers:
  - name: 
    affiliation: 
  - name: 
    affiliation: 
 aliases: 
 tags: 
 ---
 # Chirp detection poster script
 ## 10 minute presentation
 introduction:
 - Project goal: Develop a chirp detection algorithm
 - What are chirps?
 	- short frequency excursions in ms range of EOD (electric organ discharge) of weakly el. fish.
 - Show plot:
 		- spectrogram of the EODf of two fish (two lines)
 			- frequency resolution 150Hz
 			- nfft: number of windows/datapoints over which the Fourier transform is performed
 		- frequency over time is shown
 		- color indicates power
 		- chirp = upper line, frequency increases shortly
 - Problem:
 		- to resolve chirps on the time domain, frequency domain too coarse
 		- if lower fish chirps it becomes even harder
 		=> time-frequency uncertainty problem (general)
 - Goal: Improve existing detection methods to detect and assign chirps for electric recordings with n fish
 Chirp detection algorithm
 Availabe data:
 1. Raw electrical signal (EOD of multiple fish) over n electrodes (n = 11)
 2. Tracked frequency bands on spectrogram (pre-tracked): just as in upper right plot, but with lower sampling rate (3Hz)
 	- for one frequency we want the electrode on which the power of the f is the greatest
 	- power of the strongest of 11 electrodes for each frequencypoint in time was used to track the frequency band
 	- we cannot track freq on spectrogram's time resolution is too low for detecting chirps if you wanna distinguish between the fish
 Feature extraction (in 5s rolling window):
 1. Bandpass filter around the tracked frequency band for one individual (+-5Hz)
 	- first subplot grey, red = envelope of filtered baseline
 2. Dynamic bandpass filter above baseline (+-5Hz) = 2nd subplot, gray filtered search frequency, orange = envelope
 	- dynamic search window above the current fish of interest
 		- Why dynamic: if another fish has a higher frequency, we need to find a window without another fish to be able to detect the chirp
 			- window above fish: look if there's another fish (array stuff), True/false thing, find longest subarray
 		- chirps excursions always increase the frequency and decrease the amplitude
 		- to find chirp, we need to search above the fish and look for break down in amplitude
 		- no peaks in filtered above = no chirp
 		- amplitude break down of baseline can have multiple reasons (e.g. fish swims away, stone)
 3. Instantaneous frequency of baseline = 3rd subplot, gray filtered inst., yellow = envelope
 	- calculated on filtered raw data
 	- get zero crossings of each period and calculate frequency manually
 	- we know that chirps are increases in frequency, here we look at the frequency feature of chirps
 Peak detection:
 1. Detect peaks on bandpass filtered and inverted baseline envelope (lower red line)
 2. Detect peaks on bandpass filtered search frequency (lower orange line)
 3. Detect peaks on absolute inst. freq (lower yellow line)
 	- Peak prominence: Minimal distance from highest peak to next peak
 Peak classification:
 - all three features have to be present at once in a 20ms window (appr. chirp length) in strongest electrodes
 - mean of peak timestamps of features is saved as chirp timestamp
 Chirps in dyadic competitions
 1. Competition experiment by Til Raab:
 	- two fish compete for one shelter
 	- 6h recording, 3h light, 3h dark
 	- electrical and video recordings
 	- with video recordings, behavior was tracked and assigned to an antagonistic category: Chasing (on- and offset) and physical contacts
 - we did behavioral analysis with the detected chirps of our algorithm
 2. Plot: Contact an chasing event timepoints, chirps of both fish, tracked frequency bands
 	- from literature: Chirps assumed as submissive signal by loser fish
 4. Winner Loser boxplot
 	- chirps counts for winner and loser (n=22 recordings with winner and loser)
 	- loser tends to chirp more (Wilcoxon not significant, but trend with 0.054)
 	- white lines are paired fish for competition
 5. Size difference plot
 	- Literature: Larger wish usually wins. (Larger resource holding potential theory, Till with rises)
 	- The smaller the size difference between fish, the more chirps are emitted taken winner and loser together
 	- correlation within winners and losers are not significant
 	- n = 21 because one recording with equal fish size was excluded
 6. Frequency plot
 	- Literature: Males are more aggressive and chirp more, males have a lower EODf
 	- EODf has no effect on the competition outcome
 Chirps emitted by loser fish might stop chasing events
 - Chirps were centered around the timestamps of each event in a +-60s time window (for each category and each recording)
 - kernel density estimation of centered chirps (gaussian kernel with 2s width and 10ms resolution)
 - We show some example plots
 1. First plot: No correlation case for chasing offset
 	- no correlation between chirping and the offset
 	- this was the case for most recordings and all events
 2. Second plot: Correlation case for chirping and offset
 	- For some few dyads/individuals, chirp rate increases drastically before the chasing offset
 	- also slightly visible before chasing onsets
 	- no correlation for physical contacts
 3. Third plot: Time of chasing events in the night VS the chirps during the chasing events and during night
 	- fraction of chirps during chasings is not increased relative to the fraction of chasing events overall
 	- Chirps do not seem to have an increased significance for chasing events
 	- only for some few dyads the chirp rate increased during chasings
 - Gray/black areas:
 	- bootstrapped data (n = 50)
 	- all chirps for one recording during the night (because there more chirps)
 	- all shuffled chirps again centered around event and convolved
 Conclusion:
 - First tests indicate that our algorithm is able to detect chirps in recordings of multiple fish
 - Algorithm results were applied on behavioral data for further analysis
 ## 2 Minute presentation
 Introduction:
 - Project goal: Develop a chirp detection algorithm
 - What are chirps?
 	- Short frequency excursions in ms range of EOD (electric organ discharge) of weakly el. fish.
 	-  to resolve chirps on the time domain, frequency domain too coarse, especially for multiple fish
 - Goal: Improve existing detection methods
 Detection algorithm
 - Improved existing detection methods by extracting 3 features that change during a chirp but are not limited by the time frequency uncertainty
 	1. Amplitude drop of EOD (show trough)
 	2. Peaks of instantaneous frequency of EOD
 	3. Peaks in the dynamically adjusted frequency band above the fish's baseline EODf (special).
 - Detected and classified peaks are chirp times
 Application: Chirps during competition
 - Detected over 10000 chirps in real data from a competition experiment
 - Analysis of the relationship of chirps and competition events
 - Fish competed for a shelter
 - Were able to replicate some findings from literature
 	- e.g. loser fish tend to chirp more
 	- Other findings are not that clear and require the consideration of more factors, e.g. sex
 - We explored how the chirp rate changes during onsets and offsets of chasing events
 	- For some recordings, chirping increased strongly before the offset of a chasing, for some nothing happens
 	- The number of chirps during chasings is only elevated for some dyads
 Conclusion
 - Algorithm can be used to detect chirps
 - We could replicate some literature findings and motivate further examination
--- a/README.md
+++ b/README.md
@ -1,31 +1,5 @@
 <!-- Improved compatibility of back to top link: See: https://github.com/othneildrew/Best-README-Template/pull/73 -->
 <a name="readme-top"></a>
 <!--
 *** Thanks for checking out the Best-README-Template. If you have a suggestion
 *** that would make this better, please fork the repo and create a pull request
 *** or simply open an issue with the tag "enhancement".
 *** Don't forget to give the project a star!
 *** Thanks again! Now go create something AMAZING! :D
 -->
 <!-- PROJECT SHIELDS -->
 <!--
 *** I'm using markdown "reference style" links for readability.
 *** Reference links are enclosed in brackets [ ] instead of parentheses ( ).
 *** See the bottom of this document for the declaration of the reference variables
 *** for contributors-url, forks-url, etc. This is an optional, concise syntax you may use.
 *** https://www.markdownguide.org/basic-syntax/#reference-style-links
 -->
 <!-- [![Contributors][contributors-shield]][contributors-url] -->
 <!-- [![Forks][forks-shield]][forks-url] -->
 <!-- [![Stargazers][stars-shield]][stars-url] -->
 <!-- [![Issues][issues-shield]][issues-url] -->
 <!-- [![MIT License][license-shield]][license-url] -->
 <!-- [![LinkedIn][linkedin-shield]][linkedin-url] -->
 <!-- PROJECT LOGO -->
 <br />
@ -37,7 +11,7 @@
 <h3 align="center">chirpdetector</h3>
  <p align="center">
-    An algorithm to detect the chirps of weakly electric fish.
+    An algorithm to detect the chirps of weakly electric fish on multielectrode recordings.
    <br />
    <a href="https://github.com/github_username/repo_name"><strong>Explore the docs »</strong></a>
    <br />
@ -56,31 +30,14 @@
 <details>
  <summary>Table of Contents</summary>
  <ol>
-    <li>
+    <li><a href="#about-the-project">About The Project</a></li>
-      <a href="#about-the-project">About The Project</a>
+    <li><a href="#getting-started">Getting Started</a></li>
      <ul>
        <li><a href="#built-with">Built With</a></li>
      </ul>
    </li>
    <li>
      <a href="#getting-started">Getting Started</a>
      <ul>
        <li><a href="#prerequisites">Prerequisites</a></li>
        <li><a href="#installation">Installation</a></li>
      </ul>
    </li>
    <li><a href="#usage">Usage</a></li>
-    <li><a href="#roadmap">Roadmap</a></li>
+    <li><a href="#to-do">Roadmap</a></li>
    <li><a href="#contributing">Contributing</a></li>
    <li><a href="#license">License</a></li>
    <li><a href="#contact">Contact</a></li>
    <li><a href="#acknowledgments">Acknowledgments</a></li>
  </ol>
 </details>
 <!-- ABOUT THE PROJECT -->
 ## About The Project
 [![Product Name Screen Shot][product-screenshot]](https://example.com)
@ -90,98 +47,11 @@ Here's a blank template to get started: To avoid retyping too much info. Do a se
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 ```sh
  .
 ├──  2022-06-02-10_00
 │  ├──  2022.06.02_0.mp4
 │  ├──  analysis
 │  │  ├──  baseline_freq_times.npy
 │  │  ├──  baseline_freqs.npy
 │  │  ├──  fish_freq.npy
 │  │  ├──  fish_freq_interp.npy
 │  │  ├──  frame_times.npy
 │  │  ├──  rise_idx.npy
 │  │  └──  rise_size.npy
 │  ├──  backup
 │  │  ├──  fund_v.npy
 │  │  ├──  ident_v.npy
 │  │  ├──  idx_v.npy
 │  │  ├──  meta.npy
 │  │  ├──  sign_v.npy
 │  │  ├──  spec.npy
 │  │  └──  times.npy
 │  ├──  chirp_ids.npy
 │  ├──  chirps.npy
 │  ├──  chirps_ids.npy
 │  ├──  fill_freqs.npy
 │  ├──  fill_spec.npy
 │  ├──  fill_spec_shape.npy
 │  ├──  fill_times.npy
 │  ├──  fishgrid.cfg
 │  ├──  fund_v.npy
 │  ├──  ident_v.npy
 │  ├──  idx_v.npy
 │  ├──  LED_frames.npy
 │  ├──  led_idxs.csv
 │  ├──  led_times.csv
 │  ├──  LED_val.npy
 │  ├──  meta.npy
 │  ├──  sign_v.npy
 │  ├──  spec.npy
 │  ├──  times.npy
 │  └──  traces-grid1.raw
 └──  mount_data
 ```
 <!-- ### Built With -->
 <!-- * [![Next][Next.js]][Next-url] -->
 <!-- * [![React][React.js]][React-url] -->
 <!-- * [![Vue][Vue.js]][Vue-url] -->
 <!-- * [![Angular][Angular.io]][Angular-url] -->
 <!-- * [![Svelte][Svelte.dev]][Svelte-url] -->
 <!-- * [![Laravel][Laravel.com]][Laravel-url] -->
 <!-- * [![Bootstrap][Bootstrap.com]][Bootstrap-url] -->
 <!-- * [![JQuery][JQuery.com]][JQuery-url] -->
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 <!-- GETTING STARTED -->
 ## Getting Started
 This is an example of how you may give instructions on setting up your project locally.
 To get a local copy up and running follow these simple example steps.
 <!-- ### Prerequisites -->
 <!-- This is an example of how to list things you need to use the software and how to install them. -->
 <!-- * npm -->
 <!--   ```sh -->
 <!--   npm install npm@latest -g -->
 <!--   ``` -->
 ### Installation
 1. Get a free API Key at [https://example.com](https://example.com)
 2. Clone the repo
   ```sh
   git clone https://github.com/github_username/repo_name.git
   ```
 3. Install NPM packages
   ```sh
   npm install
   ```
 4. Enter your API in `config.js`
   ```js
   const API_KEY = 'ENTER YOUR API';
   ```
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 <!-- USAGE EXAMPLES -->
 ## Usage
@ -193,68 +63,80 @@ _For more examples, please refer to the [Documentation](https://example.com)_
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
-
+## To do
 <!-- ROADMAP -->
 ## Roadmap
 - [ ] Feature 1
 - [ ] Feature 2
 - [ ] Feature 3
    - [ ] Nested Feature
 See the [open issues](https://github.com/github_username/repo_name/issues) for a full list of proposed features (and known issues).
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 <!-- <!-1- CONTRIBUTING -1-> -->
 <!-- ## Contributing -->
 <!-- Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**. -->
 <!-- If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". -->
 <!-- Don't forget to give the project a star! Thanks again! -->
 <!-- 1. Fork the Project -->
 <!-- 2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`) -->
 <!-- 3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`) -->
 <!-- 4. Push to the Branch (`git push origin feature/AmazingFeature`) -->
 <!-- 5. Open a Pull Request -->
 <!-- <p align="right">(<a href="#readme-top">back to top</a>)</p> -->
 <!-- <!-1- LICENSE -1-> -->
 <!-- ## License -->
 <!-- Distributed under the MIT License. See `LICENSE.txt` for more information. -->
 <!-- <p align="right">(<a href="#readme-top">back to top</a>)</p> -->
 <!-- CONTACT -->
 ## Contact
 Your Name - [@twitter_handle](https://twitter.com/twitter_handle) - email@email_client.com
 Project Link: [https://github.com/github_username/repo_name](https://github.com/github_username/repo_name)
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
-
+<!-- # Chirp detection - GP2023 -->
-<!-- ACKNOWLEDGMENTS -->
+<!-- ## Git-Repository and commands -->
-## Acknowledgments
+
-
+<!-- - Go to the [Bendalab Git-Server](https://whale.am28.uni-tuebingen.de/git/) (https://whale.am28.uni-tuebingen.de/git/) -->
-* []()
+<!-- - Create your own account (and tell me ;D) -->
-* []()
+<!--   * I'll invite you the repository -->
-* []()
+<!-- - Clone the repository -->
-
+<!-- -  -->
-<p align="right">(<a href="#readme-top">back to top</a>)</p>
+<!-- ```sh -->
-
+<!-- git clone https://whale.am28.uni-tuebingen.de/git/raab/GP2023_chirp_detection.git -->
 <!-- ``` -->
 <!-- ## Basic git commands -->
 <!-- - pull changes in git -->
 <!-- ```shell -->
 <!-- git pull origin <branch> -->
 <!-- ``` -->
 <!-- - commit chances -->
 <!-- ```shell -->
 <!-- git commit -m '<explaination>' file  # commit one file -->
 <!-- git commit -a -m '<explaination>'    # commit all files -->
 <!-- ``` -->
 <!-- - push commits -->
 <!-- ```shell -->
 <!-- git push origin <branch> -->
 <!-- ``` -->
 <!-- ## Branches -->
 <!-- Use branches to work on specific topics (e.g. 'algorithm', 'analysis', 'writing', ore even more specific ones) and merge -->
 <!-- them into Master-Branch when it works are up to your expectations. -->
 <!-- The "master" branch should always contain a working/correct version of your project. -->
 <!-- - Create/change into branches -->
 <!-- ```shell -->
 <!-- # list all branches (highlight active branch) -->
 <!-- git banch -a            -->
 <!-- # switch into existing           -->
 <!-- git checkout <existing branch>    -->
 <!-- # switch into new branch -->
 <!-- git checkout master -->
 <!-- git checkout -b <new branch>      -->
 <!-- ``` -->
 <!-- - Re-merging with master branch -->
 <!-- 1) get current version of master and implement it into branch -->
 <!-- ```shell -->
 <!-- git checkout master -->
 <!-- git pull origin master -->
 <!-- git checkout <branch> -->
 <!-- git rebase master -->
 <!-- ``` -->
 <!-- This resets you branch to the fork-point, executes all commits of the current master before adding the commits of you  -->
 <!-- branch. You may have to resolve potential conflicts. Afterwards commit the corrected version and push it to your branch. -->
 <!-- 2) Update master branch master -->
 <!-- - correct way: Create -->
 <!-- ```shell -->
 <!-- git checkout master -->
 <!-- git merge <branch> -->
 <!-- git push origin master -->
 <!-- ``` -->
 <!-- MARKDOWN LINKS & IMAGES -->
@ -289,3 +171,4 @@ Project Link: [https://github.com/github_username/repo_name](https://github.com/
 [JQuery.com]: https://img.shields.io/badge/jQuery-0769AD?style=for-the-badge&logo=jquery&logoColor=white
 [JQuery-url]: https://jquery.com
--- a/poster_printed/figs/algorithm.pdf
+++ b/poster_printed/figs/algorithm.pdf
--- a/poster_printed/figs/algorithm1.pdf
+++ b/poster_printed/figs/algorithm1.pdf
--- a/poster_printed/figs/chirps_in_chasing.pdf
+++ b/poster_printed/figs/chirps_in_chasing.pdf
--- a/poster_printed/figs/chirps_winner_loser.pdf
+++ b/poster_printed/figs/chirps_winner_loser.pdf
--- a/poster_printed/figs/efishlogo.pdf
+++ b/poster_printed/figs/efishlogo.pdf
--- a/poster_printed/figs/introplot.pdf
+++ b/poster_printed/figs/introplot.pdf
--- a/poster_printed/figs/kde.pdf
+++ b/poster_printed/figs/kde.pdf
--- a/poster_printed/figs/logo_all.pdf
+++ b/poster_printed/figs/logo_all.pdf
--- a/poster_printed/figs/timeline.pdf
+++ b/poster_printed/figs/timeline.pdf
--- a/poster_printed/main.pdf
+++ b/poster_printed/main.pdf
--- a/poster_printed/main.tex
+++ b/poster_printed/main.tex
--- a/poster_printed/packages.tex
+++ b/poster_printed/packages.tex
--- a/poster_printed/poster.pdf
+++ b/poster_printed/poster.pdf
--- a/poster_printed/style.tex
+++ b/poster_printed/style.tex
--- a/requirements.txt
+++ b/requirements.txt
@ -1,4 +1,3 @@
 audioio==0.10.0
 cmocean==2.0
 cycler==0.11.0
 ipython==8.12.0
@ -9,5 +8,4 @@ paramiko==2.11.1
 PyYAML==6.0
 scipy==1.10.1
 scp==0.14.5
 thunderfish==1.9.10
 tqdm==4.64.1