Compare commits

..

40 Commits

Author SHA1 Message Date
32487e09a9 [sorting] fixing cleanup 2026-05-07 11:55:37 +02:00
wendtalexander
d9aed0551f [sorting] need more fixing 2026-05-06 16:58:56 +02:00
wendtalexander
11d76b205a [pyproject] adding spikeinterface 2026-05-06 16:30:40 +02:00
wendtalexander
56e6d1c589 [soritng] adding sorign and small fixes 2026-05-06 16:30:24 +02:00
wendtalexander
616dd721f5 [features] fixing duplicate name 2026-04-29 14:00:16 +02:00
wendtalexander
2c39f26a06 [pyproject] adding pyqt for matplotlib 2026-04-29 13:26:31 +02:00
wendtalexander
f8dc845625 [features] adding features 2026-04-29 13:26:20 +02:00
wendtalexander
3b849c0c23 [pyproject] adding ipython 2026-04-22 10:17:41 +02:00
wendtalexander
f068ba3c42 [pyproject] adding shorthandle 2025-10-24 13:32:25 +02:00
wendtalexander
9f4facc7bc [doc] adding samplerates back again 2025-10-24 13:32:13 +02:00
wendtalexander
88250f8f85 [readme] 2025-10-21 16:21:34 +02:00
wendtalexander
d6c5cfdac0 [readme] 2025-10-21 16:20:38 +02:00
wendtalexander
396ebe0c52 [pyproject] updating to new version 2025-10-21 15:07:06 +02:00
wendtalexander
58b88b7e73 [stim] remove embed 2025-10-21 15:06:53 +02:00
wendtalexander
a8778f3596 [doc] adding doc for different comands 2025-10-21 15:06:40 +02:00
wendtalexander
d5499eb3e8 [formatting] 2025-10-21 15:06:19 +02:00
wendtalexander
307709834b [doc] adding algorithm 2025-10-21 15:06:01 +02:00
wendtalexander
a32a1f8cb4 [doc/quarto] adding algorithm 2025-10-21 15:05:50 +02:00
wendtalexander
56a430df26 [doc/assets] adding algorithm png 2025-10-21 15:05:34 +02:00
wendtalexander
ffc4f9995d [docstring] 2025-10-21 11:56:13 +02:00
wendtalexander
ea9dd555d8 [docstring] 2025-10-21 11:56:07 +02:00
wendtalexander
cd7cd43313 [docstring] 2025-10-21 11:56:01 +02:00
wendtalexander
af98fdc2cb [main] renaming main to convert, adding plot and timeline 2025-10-21 11:55:52 +02:00
wendtalexander
6dd148656b [doc] adding calc lag 2025-10-21 11:55:30 +02:00
wendtalexander
ce1fe10050 [doc] adding sam 2025-10-21 11:55:15 +02:00
wendtalexander
1c93c7aa95 [doc/index] fixing typo 2025-10-21 11:55:10 +02:00
wendtalexander
efcae6339e [formatting] 2025-10-21 11:54:46 +02:00
wendtalexander
9ed6d44c4e [docs] adding delays overview 2025-10-21 11:54:23 +02:00
wendtalexander
3da1a323c2 [formatting] 2025-10-21 11:54:11 +02:00
wendtalexander
335a83414b [doc/quarto] adding new contens 2025-10-21 11:53:53 +02:00
wendtalexander
d79fa309bf [stimulus_recreation] lowering repro duration 2025-10-20 16:52:03 +02:00
wendtalexander
f5479f513c [doc] adding helper scripts for plots 2025-10-20 16:51:46 +02:00
wendtalexander
365e309ce7 [doc] calibration moving plotting to util 2025-10-20 16:51:33 +02:00
wendtalexander
de42eba704 [doc] adding different repos for delays 2025-10-20 16:51:13 +02:00
wendtalexander
c09a4768f4 [doc/quarto] adding sample rates, baseline fi_vurve filestimulus 2025-10-20 16:50:54 +02:00
wendtalexander
a628359fe9 [pyproject] adding scipy to docs 2025-10-20 10:20:00 +02:00
wendtalexander
5bc7b31b28 [main] adding timeline command 2025-10-20 10:19:46 +02:00
wendtalexander
f3f5f916fb [doc] adding comparision between stimulus 2025-10-20 10:19:34 +02:00
wendtalexander
77eab8d7db Merge branch 'main' of https://whale.am28.uni-tuebingen.de/git/awendt/oephys2nix 2025-10-20 07:55:41 +02:00
wendtalexander
7e850da74a [main] flipping logic 2025-10-20 07:55:12 +02:00
21 changed files with 2782 additions and 1034 deletions

View File

@@ -0,0 +1,34 @@
## 1. Motivation
`oephys2nix` is a comand-line-interface (cli), which helps converting
[relacs](https://github.com/relacs/relacs) data and
[open-ephys](https://open-ephys.org/gui) data to a single nix file. For making
life easier to work with behavior and neuronal data.
## 2. Installation and Documentation
Here my general workflow for installing the package.
```bash
git clone https://whale.am28.uni-tuebingen.de/git/awendt/oephys2nix
cd oephys2nix
python -m venv .venv
source .venv/bin/activate
pip install -e .
```
To see the documentation you need to install [quarto](https://quarto.org).
Please follow the installation instructions for quarto, and install the extra
dependencies.
```bash
pip install "[.docs]"
cd doc
quartodoc build
quarto preview
```
which should open the documentation at localhost.
## 3.To-Dos
- [x] Analyzed the delay in the two recoding systems
- [x] Analyzed the difference in sample rate
- [ ] Get offset and gain from open-ephys recorded lines.

View File

@@ -5,11 +5,9 @@ project:
format:
html:
# code-fold: true
# code-summary: "Show the code"
theme:
light: flatly
dark: darkly
light: flatly
css:
- api/_styles-quartodoc.css
- styles.css
@@ -34,14 +32,27 @@ website:
contents:
- text: "Introduction"
href: "index.qmd"
- section: "Tutorials"
- text: "Usage"
href: "usage.qmd"
- text: "Sample Rates"
href: "samplerates.qmd"
- text: "Algorithm"
href: "algorithm.qmd"
- section: "Delays"
href: 'delays.qmd'
contents:
- "usage.qmd"
- "baseline.qmd"
- "calibration.qmd"
- "fi_curve.qmd"
- "filestimulus.qmd"
- "sam.qmd"
- section: "API"
href: "api/index.qmd"
contents:
- "api/index.qmd"
- "api/main.qmd"
- "api/metadata.qmd"
- "api/stimulus_recreation.qmd"
- "api/tonix.qmd"
quartodoc:
@@ -56,6 +67,12 @@ quartodoc:
desc: Terminal client
contents:
- main
- title: helper
desc: Helper Function
contents:
- metadata
- stimulus_recreation
- tonix
execute:
freeze: auto

7
doc/algorithm.qmd Normal file
View File

@@ -0,0 +1,7 @@
---
title: Algorithm
---
### 1. Algorithm for automatic detection of repros with TTL pulses
![Algorithm](assets/algorithm.png)

BIN
doc/assets/algorithm.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 170 KiB

110
doc/baseline.qmd Normal file
View File

@@ -0,0 +1,110 @@
---
title: Baseline
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### 1. Loading
Lets look at the calibration and the first trial of the recording.
```{python}
import pathlib
import numpy as np
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from util import plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/AllStimuli/2025-10-20-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/AllStimuli/2025-10-20-aa-invivo-2_relacs/2025-10-20-aa-invivo-2_relacs.nix"
)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("BaselineActivity")[0]
repro_r = relacs.repro_runs("BaselineActivity")[0]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
```
### Plotting the First trial
If you zoom in you can see a little delay between the different recording systems. It seems that open-ephys is before the relacs recording.
```{python}
# | echo: False
# 2. Add traces to the FIRST subplot (row=1, col=1)
# Note: Plotly rows and columns are 1-indexed
fig = trial_plot(repro_d, repro_r, 1.0)
fig.show()
```
### Correlation between the Signals
```{python}
print(f"Duration of the dataset {repro_d.duration}")
print(f"Duration of the relacs {repro_r.duration}")
# Resample the open-ephys data
sinus_resampled = signal.resample(sinus, len(sinus_r))
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r, t_r, sinus_r, sinus_resampled, ["sinus-relacs", "sinus-resampled-open-ephys"]
)
fig.show()
```
We need to scale the two signals
```{python}
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
samples_20kHz = t[-1] * 20_000
lags_lanes = []
for oephys_lane, relacs_lane, names_lane in zip(
oephys_lanes, relacs_lanes, names_lanes, strict=True
):
oephys_lane_resampled = signal.resample(oephys_lane, int(samples_20kHz))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
print(f"{names_lane} has a lag of {lag}")
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r,
t_r,
np.roll(sinus_r, lags_lanes[0]),
sinus_resampled,
["rolled sinus-relacs", "sinus-resampled-open-ephys"],
)
fig.show()
```

View File

@@ -1,36 +1,47 @@
---
title: Calibration
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### Calibration of the Amplitude
Lets look at the calibration and the first trial of the recording.
```{python}
import rlxnix as rlx
import plotly.graph_objects as go
import pathlib
import numpy as np
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from util import plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix"
)
dataset = rlx.Dataset("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs = rlx.Dataset("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix")
repro_d = dataset.repro_runs("CalibEfield_1")[0]
repro_r = relacs.repro_runs("CalibEfield_1")[0]
fig = make_subplots(
rows=4,
cols=1,
shared_xaxes=True,
subplot_titles=(
"Stimulus Comparison",
"Local EOD Comparison",
"Global EOD Comparison",
"Sinus Comparison",
),)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("CalibEfield_1")[0].stimuli[2]
repro_r = relacs.repro_runs("CalibEfield_1")[0].stimuli[2]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
@@ -41,82 +52,55 @@ global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
# 2. Add traces to the FIRST subplot (row=1, col=1)
# Note: Plotly rows and columns are 1-indexed
fig.add_trace(
go.Scatter(x=t_r, y=stimulus_re, name="stimulus (relacs)", line_color="blue"),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(
x=t,
y=stimulus_oe - np.mean(stimulus_oe), # The same data transformation
name="stimulus (open-ephys)",
line_color="red",
),
row=1,
col=1,
)
fig.add_trace(
go.Scatter(x=t, y=ttl, name="ttl-line", line_color="black"),
row=1,
col=1,
)
```
### Plotting the First trial
If you zoom in you can see a little delay between the different recording systems. It seems that open-ephys is before the relacs recording.
# 3. Add traces to the SECOND subplot (row=2, col=1)
fig.add_trace(
go.Scatter(x=t_r, y=local_eod_re, name="local EOD (relacs)", line_color="blue"),
row=2,
col=1,
)
fig.add_trace(
go.Scatter(x=t, y=local_eod_oe, name="local EOD (open-ephys)", line_color="red"),
row=2,
col=1,
)
# 4. Add traces to the THIRD subplot (row=3, col=1)
fig.add_trace(
go.Scatter(x=t_r, y=global_eod_re, name="global EOD (relacs)", line_color="blue"),
row=3,
col=1,
)
fig.add_trace(
go.Scatter(
x=t, y=global_eod_oe, name="global EOD (open-ephys)", line_color="red"
),
row=3,
col=1,
)
# 5. Add traces to the FOURTH subplot (row=4, col=1)
fig.add_trace(
go.Scatter(x=t_r, y=sinus_r, name="sinus (relacs)", line_color="blue"),
row=4,
col=1,
)
fig.add_trace(
go.Scatter(x=t, y=sinus, name="sinus (open-ephys)", line_color="red"),
row=4,
col=1,
)
# 6. Update the layout for a cleaner look
fig.update_layout(
title_text="Relacs vs. Open Ephys Data Alignment",
height=800, # Set the figure height in pixels
# Control the legend
legend=dict(orientation="h", yanchor="bottom", y=1.02, xanchor="right", x=1),
)
# Add a label to the shared x-axis (targeting the last subplot)
fig.update_xaxes(title_text="Time (s)", row=4, col=1)
# 7. Show the figure
```{python}
# | echo: False
fig = trial_plot(repro_d, repro_r, 0.41)
fig.show()
```
### Correlation between the Signals
```{python}
print(f"Duration of the dataset {repro_d.duration}")
print(f"Duration of the relacs {repro_r.duration}")
# Resample the open-ephys data
sinus_resampled = signal.resample(sinus, len(sinus_r))
```
```{python}
# | echo: False
fig = plot_line_comparision(t_r, t, sinus_r, sinus, ["sinus-relacs", "sinus-oephys"])
fig.show()
```
We need to scale the two signals
```{python}
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
lags_lanes = []
for oephys_lane, relacs_lane, names_lane in zip(
oephys_lanes, relacs_lanes, names_lanes, strict=True
):
oephys_lane_resampled = signal.resample(oephys_lane, len(relacs_lane))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
print(f"{names_lane} has a lag of {lag}")
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r,
t_r,
np.roll(sinus_r, lags_lanes[0]),
sinus_resampled,
["sinus-relacs", "sinus-resampled-openepyhs"],
)
fig.show()
```

147
doc/delays.qmd Normal file
View File

@@ -0,0 +1,147 @@
---
title: Delays anaylsis
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### 1. Delays
We noticed a delay in the recodings if you were to plot a comparision between
the base relacs recording and the new generated open-ephys.
```{python}
# | echo: False
import pathlib
import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from rich.progress import track
from rich.table import Table
from util import calc_lag, plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix"
)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("FileStimulus_1")[0].stimuli[2]
repro_r = relacs.repro_runs("FileStimulus_1")[0].stimuli[2]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
fig = plot_line_comparision(
t_r,
t,
stimulus_re,
stimulus_oe - np.mean(stimulus_oe),
["stimulus-relacs", "stimulus-open-ephys"],
)
fig.show()
```
### 2. Look at differnt RePros
Currently implemented repros are:
- [x] [Baseline](baseline.qmd)
- [x] [Calibration](calibration.qmd)
- [x] [FI Curve](fi_curve.qmd)
- [x] [File Stimulus](filestimulus.qmd)
- [ ] Sams
- [ ] Chrips
- [ ] Beats
### 3. General Delay in detail
```{python}
rich_tabel = Table("Repro Run", "Signal", "Lag (samples)")
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
dataframe = []
for repro_idx, (repro_d, repro_r) in enumerate(
zip(dataset.repro_runs(), relacs.repro_runs(), strict=True)
):
if not repro_d.stimuli:
lags = calc_lag(repro_d, repro_r)
for lag, names_lane in zip(lags, names_lanes, strict=True):
rich_tabel.add_row(f"{repro_d.name}", f"{names_lane}", f"{lag}")
dataframe.append(
{"ReproName": repro_d.name, "Line": names_lane, "Lag": lag, "Trial": 0}
)
else:
lags_lanes = {f"{key}": [] for key in names_lanes}
for trial, (stim_oe, stim_re) in enumerate(
zip(repro_d.stimuli, repro_r.stimuli, strict=True)
):
lags = calc_lag(stim_oe, stim_re)
for lag, names_lane in zip(lags, names_lanes):
lags_lanes[names_lane].append(lag)
dataframe.append(
{"ReproName": repro_d.name, "Line": names_lane, "Lag": lag, "Trial": trial}
)
for lane in lags_lanes:
mean_lag = np.mean(lags_lanes[lane])
std_lag = np.std(lags_lanes[lane])
rich_tabel.add_row(f"{repro_d.name}", f"{lane}", f"{mean_lag:.2f}\u00b1{std_lag:.2f}")
rich_tabel
```
```{python}
repros = dataset.repro_runs("Baseline")
print(repros)
exclude = []
for rep in repros:
exclude.append(rep.name)
df = pd.DataFrame(dataframe)
df = df[~df["ReproName"].isin(exclude)]
fig = px.box(
df,
x="Line",
y="Lag",
color="Line",
title="Lag Distribution Across Different Signals",
labels={"Line": "Signal", "Lag": "Lag (samples)"},
)
fig.update_layout(template="plotly_dark")
fig.show()
fig = px.box(
df,
x="Line",
y="Lag",
color="ReproName",
title="Lag Distribution by Signal and Repro Run",
labels={"Line": "Signal", "Lag": "Lag (samples)"},
)
fig.update_layout(template="plotly_dark")
fig.show()
```

107
doc/fi_curve.qmd Normal file
View File

@@ -0,0 +1,107 @@
---
title: FI Curve
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### FI Curve
Lets look at the calibration and the first trial of the recording.
```{python}
import pathlib
import numpy as np
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from util import plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix"
)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("FICurve_1")[0].stimuli[2]
repro_r = relacs.repro_runs("FICurve_1")[0].stimuli[2]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
```
### Plotting the First trial
If you zoom in you can see a little delay between the different recording systems. It seems that open-ephys is before the relacs recording.
```{python}
# | echo: False
fig = trial_plot(repro_d, repro_r, 0.41)
fig.show()
```
### Correlation between the Signals
```{python}
print(f"Duration of the dataset {repro_d.duration}")
print(f"Duration of the relacs {repro_r.duration}")
# Resample the open-ephys data
sinus_resampled = signal.resample(sinus, len(sinus_r))
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r, t_r, sinus_r, sinus_resampled, ["sinus-relacs", "sinus-resampled-open-ephys"]
)
fig.show()
```
We need to scale the two signals
```{python}
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
lags_lanes = []
for oephys_lane, relacs_lane, names_lane in zip(
oephys_lanes, relacs_lanes, names_lanes, strict=True
):
oephys_lane_resampled = signal.resample(oephys_lane, len(relacs_lane))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
print(f"{names_lane} has a lag of {lag}")
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r,
t_r,
np.roll(sinus_r, lags_lanes[0]),
sinus_resampled,
["rolled sinus-relacs", "sinus-resampled-open-ephys"],
)
fig.show()
```

113
doc/filestimulus.qmd Normal file
View File

@@ -0,0 +1,113 @@
---
title: File Stimulus
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### File Stimulus
```{python}
import pathlib
import numpy as np
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from util import plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix"
)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("FileStimulus_1")[0].stimuli[2]
repro_r = relacs.repro_runs("FileStimulus_1")[0].stimuli[2]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
```
### Plotting the First trial
If you zoom in you can see a little delay between the different recording systems. It seems that open-ephys is before the relacs recording.
```{python}
# | echo: False
# 2. Add traces to the FIRST subplot (row=1, col=1)
# Note: Plotly rows and columns are 1-indexed
fig = trial_plot(repro_d, repro_r,1.01)
fig.show()
```
### Correlation between the Signals
```{python}
print(f"Duration of the dataset {repro_d.duration}")
print(f"Duration of the relacs {repro_r.duration}")
# Resample the open-ephys data
sinus_resampled = signal.resample(sinus, len(sinus_r))
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r, t_r, sinus_r, sinus_resampled, ["sinus-relacs", "sinus-resampled-open-ephys"]
)
fig.show()
```
We need to scale the two signals
```{python}
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
lags_lanes = []
for oephys_lane, relacs_lane, names_lane in zip(
oephys_lanes, relacs_lanes, names_lanes, strict=True
):
print(oephys_lane.shape)
print(relacs_lane.shape)
oephys_lane_resampled = signal.resample(oephys_lane, len(relacs_lane))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
print(f"{names_lane} has a lag of {lag}")
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r,
t,
np.roll(stimulus_re, lags_lanes[-1]),
stimulus_oe - np.mean(stimulus_oe),
["rolled sinus-relacs", "sinus-resampled-open-ephys"],
)
fig.show()
print(f"The lag of the whitenoise is {lags_lanes[-1] * (1 / 20_000) * 1000} milli seconds")
```

View File

@@ -38,7 +38,7 @@ Please follow the installation instructions for quarto, and install the extra
dependencies.
```bash
pip install "[.doc]"
pip install "[.docs]"
cd doc
quartodoc build
quarto preview

109
doc/sam.qmd Normal file
View File

@@ -0,0 +1,109 @@
---
title: SAM
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### SAM
```{python}
import pathlib
import numpy as np
import plotly.graph_objects as go
import rlxnix as rlx
import scipy.signal as signal
from plotly.subplots import make_subplots
from util import plot_line_comparision, trial_plot
dataset_path = pathlib.Path("../oephys2nix/test/AllStimuli/2025-10-20-aa-invivo-2-recording.nix")
relacs_path = pathlib.Path(
"../oephys2nix/test/AllStimuli/2025-10-20-aa-invivo-2_relacs/2025-10-20-aa-invivo-2_relacs.nix"
)
dataset = rlx.Dataset(str(dataset_path))
relacs = rlx.Dataset(str(relacs_path))
# INFO: Select the first stimulus of the calibration repro
repro_d = dataset.repro_runs("SAM")[0].stimuli[2]
repro_r = relacs.repro_runs("SAM")[0].stimuli[2]
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
```
### Plotting the First trial
If you zoom in you can see a little delay between the different recording systems. It seems that open-ephys is before the relacs recording.
```{python}
# | echo: False
fig = trial_plot(repro_d, repro_r, 0.2)
fig.update_xaxes(range=[0, 0.2])
fig.show()
```
### Correlation between the Signals
```{python}
print(f"Duration of the dataset {repro_d.duration}")
print(f"Duration of the relacs {repro_r.duration}")
# Resample the open-ephys data
sinus_resampled = signal.resample(sinus, len(sinus_r))
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r, t_r, sinus_r, sinus_resampled, ["sinus-relacs", "sinus-resampled-open-ephys"]
)
fig.show()
```
We need to scale the two signals
```{python}
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
names_lanes = ["sinus", "local-eod", "global-eod", "stimulus"]
lags_lanes = []
for oephys_lane, relacs_lane, names_lane in zip(
oephys_lanes, relacs_lanes, names_lanes, strict=True
):
oephys_lane_resampled = signal.resample(oephys_lane, len(relacs_lane))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
print(f"{names_lane} has a lag of {lag}")
```
```{python}
# | echo: False
fig = plot_line_comparision(
t_r,
t,
np.roll(stimulus_re, lags_lanes[-1]),
stimulus_oe - np.mean(stimulus_oe),
["rolled sinus-relacs", "sinus-resampled-open-ephys"],
)
fig.show()
print(f"The lag of the whitenoise is {lags_lanes[-1] * (1 / 20_000) * 1000} milli seconds")
```

182
doc/samplerates.qmd Normal file
View File

@@ -0,0 +1,182 @@
---
title: Differences between sample rates
format:
html:
toc: true
toc-title: Contents
code-block-bg: true
code-block-border-left: "#31BAE9"
code-line-numbers: true
highlight-style: atom-one
link-external-icon: true
link-external-newwindow: true
eqn-number: true
---
### 1. General Idea
The two aquisition systems have a different default sampling rate and currently
there is a delay and maybe this is due to the different sampling rates.
`open-ephys` has a sample-rate of 30 kHz and `relacs` one of 20 kHz. In this
test we have two different recordings with one where the open-epyhs has 30 kHz
and the other with 20 kHz.
### 2. Loading the data
```{python}
from pathlib import Path
import rlxnix as rlx
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import scipy.signal as signal
import numpy as np
# Path to test recording with different samplerate open-epyhs 30kHz and relacs 20kHz
dataset_path_diff_fs = Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2-recording.nix")
relacs_path_diff_fs = Path("../oephys2nix/test/Test1/2025-10-08-aa-invivo-2_relacs/2025-10-08-aa-invivo-2.nix")
# Path to test recording with same samplerate open-epyhs 20kHz and relacs 20kHz
dataset_path_same_fs = Path("../oephys2nix/test/Test2/2025-10-08-ab-invivo-2-recording.nix")
relacs_path_same_fs = Path("../oephys2nix/test/Test2/2025-10-08-ab-invivo-2_relacs/2025-10-08-ab-invivo-2.nix")
dataset_diff_fs = rlx.Dataset(str(dataset_path_diff_fs))
relacs_diff_fs = rlx.Dataset(str(relacs_path_diff_fs))
dataset_same_fs = rlx.Dataset(str(dataset_path_same_fs))
relacs_same_fs = rlx.Dataset(str(relacs_path_same_fs))
repro_diff_fs_d = dataset_diff_fs.repro_runs("FileStimulus_1")[0].stimuli[2]
repro_diff_fs_r = relacs_diff_fs.repro_runs("FileStimulus_1")[0].stimuli[2]
repro_same_fs_d = dataset_same_fs.repro_runs("FileStimulus_1")[0].stimuli[2]
repro_same_fs_r = relacs_same_fs.repro_runs("FileStimulus_1")[0].stimuli[2]
#sinus, t = repro_diff_fs_d.trace_data("sinus")
#sinus_r, t_r = repro_diff_fs_r.trace_data("V-1")
stimulus_diff_oe, t_diff = repro_diff_fs_d.trace_data("stimulus")
stimulus_diff_re, t_diff_r = repro_diff_fs_r.trace_data("GlobalEFieldStimulus")
stimulus_same_oe, t_same = repro_same_fs_d.trace_data("stimulus")
stimulus_same_re, t_same_r = repro_same_fs_r.trace_data("GlobalEFieldStimulus")
```
### 3. Samples in the different recordings for one stimulus
```{python}
#| echo: False
print(f"Samples open-epyhs [30 kHz] for one trial: {stimulus_diff_oe.shape}")
print(f"Samples relacs for one trial: {stimulus_diff_re.shape}")
print(f"Samples open-epyhs [20 kHz] for one trial: {stimulus_same_oe.shape}")
print(f"Samples relacs for one trial: {stimulus_same_re.shape}")
```
### 4. Plotting first trial
Here we plot the different output stimulus, different sample rates
```{python}
#| echo: False
x_lim = 0.05
fig = make_subplots( rows=2, cols=1, shared_xaxes=True, subplot_titles=("Different fs [30 khz and 20 kHz]", "Same fs [20kHz]"))
fig.add_trace( go.Scattergl(x=t_diff_r[t_diff_r<x_lim],
y=stimulus_diff_re[t_diff_r<x_lim],
showlegend=False, line_color="blue",
mode="markers+lines"), row=1, col=1)
fig.add_trace( go.Scattergl(x=t_diff[t_diff<x_lim],
y=stimulus_diff_oe[t_diff<x_lim],
showlegend=False,
line_color="red", mode="markers+lines"), row=1, col=1)
fig.add_trace( go.Scattergl(x=t_same_r[t_same_r<x_lim],
y=stimulus_same_re[t_same_r<x_lim],
name="GlobalStimulus (relacs)", line_color="blue",
mode="markers+lines") , row=2, col=1)
fig.add_trace( go.Scattergl(x=t_same[t_same<x_lim],
y=stimulus_same_oe[t_same<x_lim],
name="GlobalStimulus (open-ephys)",
line_color="red", mode="markers+lines"),row=2, col=1)
fig.update_layout(
template="plotly_dark",
height=400,
legend=dict(
bgcolor="rgba(0,0,0,0)",
bordercolor="#444",
borderwidth=0,
font=dict(color="#e5ecf6"),
orientation="h",
yanchor="bottom",
y=1.06,
xanchor="right",
x=0.72,
)
)
fig.update_xaxes(range=[0, 0.01])
```
### 5. Lags in recodings
```{python}
# resample to 20 kHz
stimulus_diff_oe_resampled = signal.resample(stimulus_diff_oe, len(stimulus_same_re))
correlation_diff = signal.correlate(stimulus_diff_oe_resampled, stimulus_diff_re, mode="full")
lags_diff = signal.correlation_lags(stimulus_diff_oe_resampled.size, stimulus_diff_re.size, mode="full")
lag_diff = lags_diff[np.argmax(correlation_diff)]
correlation_same = signal.correlate(stimulus_same_oe, stimulus_same_re, mode="full")
lags_same = signal.correlation_lags(stimulus_same_oe.size, stimulus_same_re.size, mode="full")
lag_same = lags_same[np.argmax(correlation_same)]
print(f"The lag in with different sampling rates is {lag_diff}, and with the same sample rate is {lag_same}")
```
```{python}
#| echo: False
fig = make_subplots( rows=2, cols=1, shared_xaxes=True, subplot_titles=("Different fs [30 khz and 20 kHz]", "Same fs [20kHz]"))
fig.add_trace( go.Scattergl(x=t_diff_r[t_diff_r<x_lim],
y=np.roll(stimulus_diff_re[t_diff_r<x_lim], lag_diff),
line_color="blue",
showlegend=False,
mode="markers+lines"), row=1, col=1)
fig.add_trace( go.Scattergl(x=t_diff[t_diff<x_lim],
y=stimulus_diff_oe[t_diff<x_lim], showlegend=False,
line_color="red", mode="markers+lines"), row=1,
col=1)
fig.add_trace( go.Scattergl(x=t_same_r[t_same_r<x_lim],
y=np.roll(stimulus_same_re[t_same_r<x_lim], lag_same),
name="GlobalStimulus (relacs)", line_color="blue",
mode="markers+lines") , row=2, col=1)
fig.add_trace( go.Scattergl(x=t_same[t_same<x_lim],
y=stimulus_same_oe[t_same<x_lim],
name="GlobalStimulus (open-ephys)",
line_color="red", mode="markers+lines"),row=2, col=1)
fig.update_layout(
template="plotly_dark",
height=400,
legend=dict(
bgcolor="rgba(0,0,0,0)",
bordercolor="#444",
borderwidth=0,
font=dict(color="#e5ecf6"),
orientation="h",
yanchor="bottom",
y=1.06,
xanchor="right",
x=0.72,
)
)
fig.update_xaxes(range=[0, 0.01])
```
### 6. Conculsion
Lags of simuliar magnitude exists in both recordings therefor the sample rate is not the problem!

View File

@@ -2,10 +2,38 @@
title: How to use it
---
### 1.Usage
If you have a folder or multiple folders with each containing two recordings one from `relacs` and one from `open-ephys` you can simply run the CLI like this:
```{python}
# leave out the ! at the beginning if you running this in your shell
!oephys2nix ../oephys2nix/test/Test1/
!oephys2nix convert ../oephys2nix/test/Test1/
```
which provides you with information about the transition of the stimuli into the new file.
### 1.2 Timeline plot
```sh
oephys2nix timeline ../oephys2nix/test/Test1/
```
```{python}
# | echo: False
from oephys2nix.main import timeline
path = "../oephys2nix/test/Test1/"
timeline(path)
```
### 1.3 plot
```sh
oephys2nix plot ../oephys2nix/test/Test1/
```
```{python}
# | echo: False
from oephys2nix.main import plot
path = "../oephys2nix/test/Test1/"
plot(path)
```

209
doc/util.py Normal file
View File

@@ -0,0 +1,209 @@
import numpy as np
import plotly.graph_objects as go
import scipy.signal as signal
from plotly.subplots import make_subplots
def trial_plot(repro_d, repro_r, x_lim: int = 1.0):
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
ttl, t = repro_d.trace_data("ttl-line")
mask = t < x_lim
mask_r = t_r < x_lim
t = t[mask]
t_r = t_r[mask_r]
sinus = sinus[mask]
sinus_r = sinus_r[mask_r]
stimulus_oe = stimulus_oe[mask]
stimulus_re = stimulus_re[mask_r]
local_eod_oe = local_eod_oe[mask]
local_eod_re = local_eod_re[mask_r]
global_eod_oe = global_eod_oe[mask]
global_eod_re = global_eod_re[mask_r]
ttl = ttl[mask]
fig = make_subplots(
rows=5,
cols=1,
shared_xaxes=True,
subplot_titles=(
"TTL-Line",
"Stimulus",
"Local EOD",
"Global EOD",
"Sinus",
),
)
fig.add_trace(
go.Scattergl(x=t, y=ttl, name="ttl-line", line_color="magenta"),
row=1,
col=1,
)
fig.add_trace(
go.Scattergl(x=t_r, y=stimulus_re, line_color="blue"),
row=2,
col=1,
)
fig.add_trace(
go.Scattergl(
x=t,
y=stimulus_oe - np.mean(stimulus_oe), # The same data transformation
name="stimulus (open-ephys)",
line_color="red",
),
row=2,
col=1,
)
# 3. Add traces to the SECOND subplot (row=2, col=1)
fig.add_trace(
go.Scattergl(x=t_r, y=local_eod_re, line_color="blue", showlegend=False),
row=3,
col=1,
)
fig.add_trace(
go.Scattergl(x=t, y=local_eod_oe, showlegend=False, line_color="red"),
row=3,
col=1,
)
# 4. Add traces to the THIRD subplot (row=3, col=1)
fig.add_trace(
go.Scattergl(x=t_r, y=global_eod_re, showlegend=False, line_color="blue"),
row=4,
col=1,
)
fig.add_trace(
go.Scattergl(x=t, y=global_eod_oe, showlegend=False, line_color="red"),
row=4,
col=1,
)
fig.add_trace(
go.Scattergl(x=t_r, y=sinus_r, showlegend=False, line_color="blue"),
row=5,
col=1,
)
fig.add_trace(
go.Scattergl(x=t, y=sinus, showlegend=False, line_color="red"),
row=5,
col=1,
)
# 6. Update the layout for a cleaner look
fig.update_layout(
template="plotly_dark",
height=800, # Set the figure height in pixels
# Control the legend
legend=dict(
bgcolor="rgba(0,0,0,0)", # transparent dark (or use "#1f2630" to match bg)
bordercolor="#444",
borderwidth=0,
font=dict(color="#e5ecf6"), # matches plotly_dark foreground
orientation="h",
yanchor="bottom",
y=1.05,
xanchor="right",
x=0.72,
),
)
# Add a label to the shared x-axis (targeting the last subplot)
fig.update_xaxes(title_text="Time (s)", row=4, col=1)
fig.update_xaxes(range=[0, x_lim])
return fig
def plot_line_comparision(
time_relacs,
time_oephys,
data_relacs,
data_oephys,
labels,
):
x_lim = 1.0
mask = time_oephys < x_lim
mask_r = time_relacs < x_lim
time_oephys = time_oephys[mask]
time_relacs = time_relacs[mask_r]
data_oephys = data_oephys[mask]
data_relacs = data_relacs[mask_r]
fig = go.Figure()
fig.add_trace(
go.Scattergl(
x=time_relacs,
y=data_relacs,
name=labels[0],
line_color="blue",
mode="lines+markers",
)
)
fig.add_trace(
go.Scattergl(
x=time_oephys,
y=data_oephys,
name=labels[1],
line_color="red",
mode="lines+markers",
)
)
fig.update_layout(
template="plotly_dark",
height=500, # Set the figure height in pixels
legend=dict(
bgcolor="rgba(0,0,0,0)",
bordercolor="#444",
borderwidth=0,
font_color="#e5ecf6",
orientation="h",
yanchor="bottom",
y=1.05,
xanchor="right",
x=0.72,
),
)
fig.update_xaxes(title_text="Time (s)", range=[0, 0.01])
return fig
def calc_lag(repro_d, repro_r):
sinus, t = repro_d.trace_data("sinus")
sinus_r, t_r = repro_r.trace_data("V-1")
stimulus_oe, t = repro_d.trace_data("stimulus")
stimulus_re, t_r = repro_r.trace_data("GlobalEFieldStimulus")
local_eod_oe, t = repro_d.trace_data("local-eod")
local_eod_re, t_r = repro_r.trace_data("LocalEOD-1")
global_eod_oe, t = repro_d.trace_data("global-eod")
global_eod_re, t_r = repro_r.trace_data("EOD")
oephys_lanes = [sinus, local_eod_oe, global_eod_oe, stimulus_oe]
relacs_lanes = [sinus_r, local_eod_re, global_eod_re, stimulus_re]
lags_lanes = []
for oephys_lane, relacs_lane in zip(oephys_lanes, relacs_lanes, strict=True):
oephys_lane_resampled = signal.resample(oephys_lane, len(relacs_lane))
correlation = signal.correlate(oephys_lane_resampled, relacs_lane, mode="full")
lags = signal.correlation_lags(oephys_lane_resampled.size, relacs_lane.size, mode="full")
lag = lags[np.argmax(correlation)]
lags_lanes.append(lag)
return lags_lanes

View File

@@ -1,13 +1,17 @@
import logging
import sys
from os import path
from pathlib import Path
from typing import Annotated
import nixio
import rlxnix as rlx
import typer
from IPython import embed
from rich.console import Console
from oephys2nix.logging import setup_logging
from oephys2nix.sorting import AppendSorting
from oephys2nix.stimulus_recreation import StimulusToNix
from oephys2nix.tonix import RawToNix
@@ -17,7 +21,43 @@ console = Console()
@app.command()
def main(
def append_sorting(
sorter_name: str = typer.Argument(
"sorting_analyzser",
help="The sorter name that should be appended to the generated nix file",
),
data_path: Path | None = typer.Argument(
None,
help="The source directory containing the generated recording.",
exists=True,
file_okay=False,
dir_okay=True,
readable=True,
resolve_path=True,
),
overwrite: bool = typer.Option(default=True, help="Overwrites the sorter"),
verbose: Annotated[int, typer.Option("--verbose", "-v", count=True)] = 0,
) -> None:
"""Combines open ephys data with relacs data from data_path to a new nix file."""
setup_logging(logging.getLogger("oephys2nix"), verbosity=verbose)
if data_path is None:
data_path = Path.cwd()
log.info(f"Selected path is {data_path}")
rec_data_paths = list(Path(data_path).rglob("*recording.nix"))
log.debug(rec_data_paths)
for recording in rec_data_paths:
parent = recording.parent
sorter_path = parent / sorter_name
if not (sorter_path).is_dir:
log.error(f"Could not find the sorter that was specifided in {parent}")
sys.exit(1)
sorter_cls = AppendSorting(sorter_path, recording)
@app.command()
def convert(
data_path: Path = typer.Argument(
...,
help="The source directory containing the Open Ephys data.",
@@ -27,7 +67,7 @@ def main(
readable=True,
resolve_path=True,
),
ttl: bool = typer.Option(default=True, help="For recordings that did not have a ttl pulse"),
no_ttl: bool = typer.Option(False, help="For recordings that did not have a ttl pulse"),
overwrite: bool = typer.Option(default=True, help="Overwrites nix file"),
debug: bool = typer.Option(default=True, help="Shows more information and plots the results"),
verbose: Annotated[int, typer.Option("--verbose", "-v", count=True)] = 0,
@@ -37,11 +77,11 @@ def main(
setup_logging(logging.getLogger("oephys2nix"), verbosity=verbose)
log.info(f"Selected data_path is {data_path}")
open_ephys_data_paths = list(Path(data_path).rglob("*open-ephys"))
relacs_data_paths = list(Path(data_path).rglob("*relacs/*.nix"))
if not open_ephys_data_paths:
log.error("Did not find any open-ephys data")
raise typer.Exit()
relacs_data_paths = list(Path(data_path).rglob("*relacs/*.nix"))
if not relacs_data_paths:
log.error("Did not find any relacs data")
raise typer.Exit()
@@ -91,10 +131,68 @@ def main(
stim = StimulusToNix(open_ephys, str(relacs), str(nix_path))
stim.create_repros_automatically()
stim.print_table()
# stim.checks()
# if debug:
# stim.plot_stimulus()
@app.command()
def timeline(
data_path: Path = typer.Argument(
...,
help="The source directory containing a dataset.",
exists=True,
file_okay=False,
dir_okay=True,
readable=True,
resolve_path=True,
),
) -> None:
"""Plot the timeline for a given dataset."""
dataset = list(Path(data_path).rglob("*.nix"))
if not dataset:
log.error("Did not find any dataset")
raise typer.Exit()
if len(dataset) > 1:
log.info(f"Found multiple datasets {len(dataset)} taking the first one")
dataset = rlx.Dataset(str(dataset[0]))
dataset.plot_timeline()
dataset.close()
@app.command()
def plot(
data_path: Path = typer.Argument(
...,
help="The source directory containing the open-ephys and relacs data.",
exists=True,
file_okay=False,
dir_okay=True,
readable=True,
resolve_path=True,
),
) -> None:
"""Plot the stimulus, TTL and the relacs stimulus.
You can run this if you converted already the nix-file.
"""
relacs_data_paths = list(Path(data_path).rglob("*relacs/*.nix"))
if not relacs_data_paths:
log.error("Did not find any relacs data")
raise typer.Exit()
open_ephys_data_paths = list(Path(data_path).rglob("*open-ephys"))
if not open_ephys_data_paths:
log.error("Did not find any open-ephys data")
raise typer.Exit()
nix_file_name = relacs_data_paths[0].parent.name.split("_")[0] + "-recording.nix"
nix_path = relacs_data_paths[0].parent.parent / nix_file_name
if not nix_path.is_file:
log.error("Did not find any converted nix file")
raise typer.Exit()
stim = StimulusToNix(open_ephys_data_paths[0], str(relacs_data_paths[0]), str(nix_path))
stim.plot_stimulus()
stim.close()
if __name__ == "__main__":

View File

@@ -2,6 +2,15 @@ import nixio
def create_metadata_from_dict(d: dict, section: nixio.Section) -> None:
"""Creating nix section from dictionary.
Parameters
----------
d : dict
Dictionary that needs to be saved to section
section : nixio.Section
Target section in Nix file
"""
for key, value in d.items():
if isinstance(value, dict):
new_sec = section.create_section(key, f"{type(key)}")
@@ -18,6 +27,18 @@ def create_metadata_from_dict(d: dict, section: nixio.Section) -> None:
def create_dict_from_section(section: nixio.Section) -> dict:
"""Creating dictionary from Nix section
Parameters
----------
section : nixio.Section
Source nix section that will be exported to the dictionary
Returns
-------
dict: dict
Nix section in a dictionary
"""
d = {}
for key, value in section.items():
if isinstance(value, nixio.Section):

96
oephys2nix/sorting.py Normal file
View File

@@ -0,0 +1,96 @@
import logging
import pathlib
import nixio
import numpy as np
import spikeinterface.core as si
from nixio.exceptions import DuplicateName
from rich.console import Console
log = logging.getLogger(__name__)
console = Console()
class AppendSorting:
"""Append the sorting analyzser or a sortign form spikeinterface to the created nix file.
Parameters
----------
sorter_path: pathlib.Path
Path to open-ephys recording
relacs_nix_path : str
Path to relacs nix file
nix_file : str
Path to new nix file
"""
def __init__(self, sorter_path: pathlib.Path, recording_path: pathlib.Path):
self.sorter_path = sorter_path
self.recording_path = recording_path
self.sorting = si.load_sorting_analyzer(self.sorter_path)
self.nixfile = nixio.File.open(str(self.recording_path), nixio.FileMode.ReadWrite)
self.block = self.nixfile.blocks[0]
self.das = self.block.data_arrays
self.channel_ids = si.get_template_extremum_channel(
self.sorting, mode="extremum", peak_sign="neg", outputs="index"
)
self.data = self.block.data_arrays["data"]
self._clean()
self.append_sorting_to_recording()
def append_sorting_to_recording(self):
try:
gr = self.block.create_group("units", "sorting.group")
except DuplicateName:
del self.block.groups["units"]
gr = self.block.create_group("units", "sorting.group")
for unit in self.sorting.unit_ids:
spike_times = self.sorting.sorting.get_unit_spike_train_in_seconds(
unit, segment_index=0
)
unit_channel = self.channel_ids[unit]
channel_tag = np.repeat(unit_channel, spike_times.shape[0])
multi_tag_positions = np.column_stack((spike_times, channel_tag))
try:
positions = self.block.create_data_array(
f"unit-{unit}", "sorting.spike_index", data=spike_times
)
positions.append_range_dimension_using_self()
except DuplicateName:
del self.das[f"unit-{unit}"]
positions = self.block.create_data_array(
f"unit-{unit}", "sorting.spike_index", data=spike_times
)
positions.append_range_dimension_using_self()
gr.data_arrays.append(positions)
try:
multi_tag = self.block.create_multi_tag(
f"unit-{unit}", "sorting.spike_index", multi_tag_positions
)
multi_tag.references.append(self.data)
except DuplicateName:
del self.block.multi_tags[f"unit-{unit}"]
del self.das[f"unit-{unit}-positions"]
multi_tag = self.block.create_multi_tag(
f"unit-{unit}", "sorting.spike_index", multi_tag_positions
)
multi_tag.references.append(self.data)
gr.mulit_tags.append(positions)
def _clean(self):
try:
gr = self.block.groups["units"]
except IndexError:
return
for das in gr.data_arrays:
del self.das[das.name]
for mtag in gr.multi_tags:
del self.block.mulit_tags[mtag.name]
del self.das[mtag.name + "-positions"]
def close(self):
self.nixfile.close()

View File

@@ -1,7 +1,6 @@
import logging
import pathlib
import sys
import tomllib
import matplotlib.pyplot as plt
import nixio
@@ -22,6 +21,47 @@ console = Console()
class StimulusToNix:
"""Processing the stimulus recreation from the relax dataset and the open-ephys data.
Parameters
----------
open_ephys_path: pathlib.Path
Path to open-ephys recording
relacs_nix_path : str
Path to relacs nix file
nix_file : str
Path to new nix file
Attributes
----------
relacs_nix_path : str
Path to relacs nix file
nix_file_path : str
Path to new nix file
dataset : rlx.Dataset
Dataset from the relacs file
relacs_nix_file : nixio.File
Relacs nix file
relacs_block : nixio.Block
Relacs nix block
relacs_sections :nixio.Section
Relacs nix Section
neo_data : neo.OpenEphysBinaryIO
Open-ephys data
fs : float
Sample rate of the open-ephys recording
nix_file : nixio.File
New nix file
block : nixio.Block
New nix block
threshold : float
Threshold for TTL line
new_start_jiggle : float
For finding the new start, ensuring finding next TTL pulse
"""
def __init__(self, open_ephys_path: pathlib.Path, relacs_nix_path: str, nix_file: str):
self.relacs_nix_path = relacs_nix_path
self.nix_file_path = nix_file
@@ -43,7 +83,8 @@ class StimulusToNix:
self.threshold = 2
self.new_start_jiggle = 0.1
def _append_relacs_tag_mtags(self):
def _append_relacs_tag_mtags(self) -> None:
"""Append relacs tags and multi tags to new nix file."""
for t in self.relacs_block.tags:
log.debug(f"Appending relacs tags {t.name}")
tag = self.block.create_tag(f"relacs_{t.name}", t.type, position=t.position)
@@ -80,7 +121,31 @@ class StimulusToNix:
pass
mtag.metadata = self.nix_file.sections[sec.name]
def _find_peak_ttl(self, time_ttl, peaks_ttl, lower, upper):
def _find_next_ttl(
self, time_ttl: np.ndarray, peaks_ttl: np.ndarray, lower: float, upper: float
) -> np.ndarray:
"""Find the next TTL pulse within a specific duration constrained by lower and upper.
Parameters
----------
time_ttl : np.ndarray
Time array of the TTL line
peaks_ttl : np.ndarray
Detected peaks indeces on the TTL line
lower : float
lower bound for searching the TTL pulse
upper : float
upper bound for searching the TTL pulse
Returns
-------
np.ndarray
time point of the new TTL pulse
"""
peak = time_ttl[peaks_ttl[(time_ttl[peaks_ttl] > lower) & (time_ttl[peaks_ttl] < upper)]]
if not peak.size > 0:
@@ -92,7 +157,28 @@ class StimulusToNix:
peak = np.mean(peak)
return peak
def _find_peak_ttl_index(self, time_ttl, peaks_ttl, current_position):
def _find_peak_ttl_index(
self, time_ttl: np.ndarray, peaks_ttl: np.ndarray, current_position: np.ndarray
) -> np.ndarray:
"""Find the next TTL pulse from the indeces of the detected TTL pulses.
Parameters
----------
time_ttl : np.ndarray
Time array of the TTL line
peaks_ttl : np.ndarray
Detected peaks indeces on the TTL line
current_position : np.ndarray
Current time of the TTL pulse
Returns
-------
np.ndarray
Next time of TTL pulse
"""
new_repro_start_index = peaks_ttl[
(time_ttl[peaks_ttl] > current_position - self.new_start_jiggle)
& (time_ttl[peaks_ttl] < current_position + self.new_start_jiggle)
@@ -112,13 +198,34 @@ class StimulusToNix:
@property
def _reference_groups(self) -> list[nixio.Group]:
"""Holds the reference groups.
Returns
-------
list[nixio.Group]
"""
return [
self.block.groups["neuronal-data"],
self.block.groups["efish"],
self.block.groups["relacs"],
]
def _append_mtag(self, repro, positions, extents):
def _append_mtag(self, repro: rlx.Dataset, positions: np.ndarray, extents: np.ndarray) -> None:
"""Apped multi tags of the current repro to the nix file.
Parameters
----------
repro : rlx.Dataset
Current Repro
positions : np.ndarray
postions of the multi tags
extents : np.ndarray
extents of the multi tags
"""
try:
nix_mtag = self.block.create_multi_tag(
f"{repro.name}",
@@ -159,7 +266,22 @@ class StimulusToNix:
nix_group = self.nix_file.blocks[0].groups[repro.name]
nix_group.multi_tags.append(nix_mtag)
def _append_tag(self, repro, position, extent):
self._append_features(repro, nix_mtag)
def _append_tag(self, repro: rlx.Dataset, position: np.ndarray, extent: np.ndarray) -> None:
"""Append tag of the current repro.
Parameters
----------
repro : rlx.Dataset
Current Repro
position : np.ndarray
positions of the multi tags
extent : np.ndarray
extents of the multi tags
"""
try:
nix_tag = self.block.create_tag(
f"{repro.name}",
@@ -201,7 +323,48 @@ class StimulusToNix:
nix_group = self.nix_file.blocks[0].groups[repro.name]
nix_group.tags.append(nix_tag)
def create_repros_automatically(self):
def _append_features(self, repro, mtag):
s = repro.stimuli[0]
if not s.features:
log.debug(f"Repro {repro.name} does not have a feature in multitags")
return
das_names = [f[1] for f in s.features]
for das in das_names:
arr = self.relacs_block.data_arrays[das]
if arr.data.dtype == object:
data = arr.data[:].astype(np.dtypes.StringDType)
else:
data = arr.data[:]
try:
f = self.block.create_data_array(
arr.name,
arr.type,
data=data,
unit=arr.unit,
label=arr.label,
)
except DuplicateName:
f = self.block.data_arrays[arr.name]
for d in arr.dimensions:
if d.dimension_type == nixio.DimensionType.Set:
f.append_set_dimension(labels=d.labels)
elif d.dimension_type == nixio.DimensionType.Range:
f.append_range_dimension(
np.sort(d.ticks),
labels=d.labels,
)
else:
f.append_sampled_dimension(
d.sampling_interval,
labels=d.labels,
)
mtag.create_feature(f, nixio.LinkType.Indexed)
def create_repros_automatically(self) -> None:
"""Create the repros form relacs with the TTL pulses."""
ttl_oeph = self.block.data_arrays["ttl-line"][:]
time_ttl = np.arange(len(ttl_oeph)) / self.fs
time_index = np.arange(len(ttl_oeph))
@@ -214,7 +377,7 @@ class StimulusToNix:
if close_peaks.size > 0:
peaks_ttl = np.delete(peaks_ttl, close_peaks)
first_peak = self._find_peak_ttl(
first_peak = self._find_next_ttl(
time_ttl,
peaks_ttl,
time_ttl[peaks_ttl[0]] - self.new_start_jiggle,
@@ -225,10 +388,9 @@ class StimulusToNix:
for i, repro in enumerate(self.dataset.repro_runs()):
log.debug(repro.name)
log.debug(f"Current Position {current_position.item()}")
if repro.duration < 1.0:
if repro.duration < 0.05:
log.warning(f"Skipping repro {repro.name} because it is two short")
continue
if repro.stimuli:
log.debug("Processing MultiTag")
repetition = len(repro.stimuli)
@@ -244,7 +406,10 @@ class StimulusToNix:
time_ttl, peaks_ttl, current_position
)
# current_position = position_mtags[-1]
if "FICurve_" in repro.name:
delay = repro.stimuli[0].feature_data(1).item()
extents_mtag += delay
self._append_mtag(repro, position_mtags, extents_mtag)
extent = position_mtags[-1] + extents_mtag[-1] - position_mtags[0]
self._append_tag(repro, position_mtags[0], extent)
@@ -282,9 +447,11 @@ class StimulusToNix:
last_repro_position.reshape(-1, 1),
(current_position - last_repro_position).reshape(-1, 1),
)
# self.close()
def create_repros_from_config_file(self):
def create_repros_from_config_file(self) -> None:
"""Creates repros form a config file.
NOT MAINTAINED.
"""
ttl_oeph = self.block.data_arrays["ttl-line"][:]
peaks_ttl = signal.find_peaks(
ttl_oeph.flatten(),
@@ -429,7 +596,8 @@ class StimulusToNix:
]
nix_group.multi_tags.append(nix_mtag)
def print_table(self):
def print_table(self) -> None:
"""Print the converted times in a rich table."""
nix_data_set = rlx.Dataset(self.nix_file_path)
table = Table("Repro Name", "start", "stop", "duration")
for repro_r, repro_n in zip(self.dataset.repro_runs(), nix_data_set.repro_runs()):
@@ -443,7 +611,8 @@ class StimulusToNix:
console.print(table)
nix_data_set.close()
def checks(self):
def checks(self) -> None:
"""Just for debugging currently."""
important_repros = ["FileStimulus", "SAM", "FICurve"]
nix_data_set = rlx.Dataset(self.nix_file_path)
@@ -508,7 +677,8 @@ class StimulusToNix:
v_eod, t_eod = repro_n.trace_data("global-eod")
v_eodr, t_eodr = repro_n.trace_data("EOD")
def plot_stimulus(self):
def plot_stimulus(self) -> None:
"""Plot the relacs stimulus, open-epyhs and TTL line."""
ttl_oeph = self.block.data_arrays["ttl-line"][:]
time_index = np.arange(len(ttl_oeph))

View File

@@ -1,5 +1,6 @@
import logging
import pathlib
import sys
import nixio
import numpy as np
@@ -14,6 +15,36 @@ log = logging.getLogger(__name__)
class RawToNix:
"""Appending all raw data from relacs and open-ephsy to a new nix file.
Parameters
----------
open_ephys_path: pathlib.Path
Path to open-ephys recording
relacs_nix_path : str
Path to relacs nix file
nix_file : str
Path to new nix file
Attributes
----------
relacs_nix_file : nixio.File
Relacs nix file
dataset : rlx.Dataset
Dataset of the relacs file
relacs_block : nixio.Block
Relacs block
relacs_sections : nixio.Section
Relacs section
neo_data : neo.OpenEphysBinaryIO
Open Ephys data
nix_file : nixio.File
New nix file
block : nixio.Block
New nix file block
"""
def __init__(self, open_ephys_path: pathlib.Path, relacs_nix_path: str, nix_file: str):
self.relacs_nix_file = nixio.File.open(relacs_nix_path, nixio.FileMode.ReadOnly)
self.dataset = rlx.Dataset(relacs_nix_path)
@@ -25,7 +56,8 @@ class RawToNix:
self.nix_file.create_block("open-ephys.data", "open-ephys.sampled")
self.block = self.nix_file.blocks[0]
def append_section(self):
def append_section(self) -> None:
"""Append sections from relacs."""
sec = self.nix_file.create_section(
self.relacs_sections[0].name, self.relacs_sections[0].type
)
@@ -33,7 +65,8 @@ class RawToNix:
create_metadata_from_dict(d, sec)
self.block.metadata = sec
def append_fish_lines(self):
def append_fish_lines(self) -> None:
"""Append fish lines from open-ephys."""
efishs = ["ttl-line", "global-eod", "stimulus", "local-eod", "sinus"]
efish_types = [
@@ -46,7 +79,7 @@ class RawToNix:
efish_group = self.block.create_group("efish", "open-ephys.sampled")
efish_neo_data = self.neo_data[0].segments[0].analogsignals[2].load()
efish_neo_data = self._load_neo_object(["Data_ADC", "acquisition_board_ADC"])
for i in np.arange(len(efishs)):
log.debug(f"Appending efish traces {efishs[i]}")
@@ -63,7 +96,16 @@ class RawToNix:
)
efish_group.data_arrays.append(data_array)
def append_relacs_lines(self):
def _load_neo_object(self, names: list[str]):
for sig in self.neo_data[0].segments[0].analogsignals:
if any([sig.name.endswith(n) for n in names]):
return sig.load()
log.error(f"No {names} found in open ephys data")
sys.exit(1)
def append_relacs_lines(self) -> None:
"""Append relacs lines."""
relacs = [
"V-1",
"EOD",
@@ -96,24 +138,31 @@ class RawToNix:
label=efish_relacs_data_array.label,
unit=efish_relacs_data_array.unit,
)
if efish_relacs_data_array.dimensions[0].dimension_type == nixio.DimensionType.Sample:
data_array.append_sampled_dimension(
efish_relacs_data_array.dimensions[0].sampling_interval,
label="time",
unit="s",
)
elif efish_relacs_data_array.dimensions[0].dimension_type == nixio.DimensionType.Range:
data_array.append_range_dimension(
np.sort(efish_relacs_data_array.dimensions[0].ticks),
label="time",
unit="s",
)
for d in efish_relacs_data_array.dimensions:
if d.dimension_type == nixio.DimensionType.Sample:
data_array.append_sampled_dimension(
efish_relacs_data_array.dimensions[0].sampling_interval,
label="time",
unit="s",
)
elif d.dimension_type == nixio.DimensionType.Range:
data_array.append_range_dimension(
np.sort(efish_relacs_data_array.dimensions[0].ticks),
label="time",
unit="s",
)
else:
data_array.append_set_dimension(
label="time",
unit="s",
)
efish_group.data_arrays.append(data_array)
def append_raw_data(self):
def append_raw_data(self) -> None:
"""Append Open-Ephys Raw data."""
gr = self.block.create_group("neuronal-data", "open-epyhs.sampled")
raw_neo_data = self.neo_data[0].segments[0].analogsignals[0].load()
raw_neo_data = self._load_neo_object(["Data", "acquisition_board"])
log.debug("Appending raw data")
nix_data_array = self.block.create_data_array(
@@ -126,8 +175,10 @@ class RawToNix:
nix_data_array.append_sampled_dimension(
1 / raw_neo_data.sampling_rate.magnitude, label="time", unit="s"
)
nix_data_array.append_sampled_dimension(1, label="channel")
gr.data_arrays.append(nix_data_array)
def close(self):
def close(self) -> None:
"""Close all nix files."""
self.nix_file.close()
self.relacs_nix_file.close()

View File

@@ -1,20 +1,25 @@
[project]
name = "oepyhs2nix"
version = "0.1.0"
version = "0.1.1"
description = "Converting ophen-ephys data to nix format"
readme = "README.md"
requires-python = ">=3.13"
dependencies = [
"ipython>=9.6.0",
"matplotlib>=3.10.6",
"neo>=0.14.2",
"nixio>=1.5.4",
"pyqt6>=6.11.0",
"rich>=14.1.0",
"rlxnix",
"scikit-learn>=1.8.0",
"scipy>=1.16.2",
"spikeinterface>=0.104.3",
"typer>=0.19.2",
]
[project.scripts]
oephys2nix = "oephys2nix.main:app"
oe2n = "oephys2nix.main:app"
[project.optional-dependencies]
docs = [
@@ -22,6 +27,7 @@ docs = [
"jupyterlab>=4.4.9",
"plotly>=6.3.1",
"quartodoc>=0.11.1",
"scipy>=1.16.2",
]
[tool.ruff]

2039
uv.lock generated

File diff suppressed because it is too large Load Diff