Skip to content

Commit 5a7dc40

Browse files
tpeulentpeulenThomas-Otavio PeulenAdministrator
authored
Add confocor3 support (#30)
* upgrade pip * Upload linux wheels * Upload non master to testpypi * Add windows wheel * windows wheel * windows wheel * Windows wheel * Skip existing wheels * Use prebuild docker img for windows * Windows wheels * skip win32 and pypy * FFTW dir in windows build * Compiler flags for HDF5 and fftw * Use fftw only with ICS * Add swig * Mask hdf5 dependency * Mask hdf5 * Fix if close * Hdf5 issue * Static boost linking * Windows boost linking * win boost * remove boost system * static boost * boost exception issue * Use vcpkg on win * Static boost builds * avoid boost * Fix typo * auto link * linking * Build photon hdf by default * Remove openmp dependency * remove boost::filesystem and locale depency * remove boost:locale * Enable language * remove vcpkg * only boost header * remove pip swig cmake boost * add cmake * Use cmake findpython * switch findpzthon3 for findpython * Set DPython_ROOT_DIR in setup.py * Add swig * remove cmake swig * remove cmake submodule * move cmake files from submodule to folder * remove cmake submodule * Add pocketfft * Reduce boost dependency * Remove unused import * Move other peoples code to external * Move other peoples code to external * Move to external * Refactoring * refactor * Add itlib * Restore * Add aglib * Add aglib * Add optimization wrapper * Add optimization wrapper * Add msstl * use itlib flat_set * Cleanup * Cleanup * bump version * separate external include files * cleanup * Add documentation * win issue * Use BUILD_PYTHON_DOCS * Win issue * win build issue * Move external thirdparty code to highlight other projects code external third party code is moved to thirdparty folder. * Remove boost dependency * Static linking of boost remove uneeded boost dependency and link (just in case) boost statically * switch fftw for pocketfftw * switch fftw for pocketfftw * fftw -> pocketfft * Fix typos * remove boost fftw * update license info * remote fftw * cleanup namespace * Add windows wheel Dockerfile * Add wheel:win build * Remove unused config * fix indentation * Fix typo * use cmd.exe for wheel:win * use cmd for whee:win * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Simplify test * fix skip on linux * fix wheel:win pypi upload * Add pwsh to docker * Update .gitlab-ci.yml file * fix * xx * ff * hfd * Update file .gitlab-ci.yml * Update file pocketfft_hdronly.h * Add example writing CLSM data * Windows * new c++ on osx * Dynamic hdf5 lib * fix call * wheel build * add header setter * doc fixes * add windows wheel * docs * docs * add osx:wheel * macos wheel * osx fixes * macos wheel * use modern c++ on osx * wheel osx fix * doc fix * Fix badges and license * Removed TODO * doc fixes * DOC FIX * Also upload development docs * doc build fix * Docs buil * docs building * fix doc * fix doc * Redirect RTD page to docs * Support more py versions * docs deposit * pin boost * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Compile wheel for py311 * Set min macos version * Update .gitlab-ci.yml * Update .gitlab-ci.yml * Specify mmacosx version * Update .gitlab-ci.yml * Cleanup * Cleanup * Add read clsm settings method * Add mask flip * Fix typo * Add option for mask * Change default masking. * Bump version * Pypi 2fa requires tokens * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Replace old np.float types * Replace old np.float types * Fix TTTRMask issue need to flip * Improved documentation * Separate doc from doc-tttrlib env * Fix typo * Fix typo * Fix bixtex issue * Fix doxygen issue * Bump version * Improve docstrings * Additional sphinx modules * Fix typo * Fix typo * Fix issue of deprecated std * Remove missing sphinx-extension * Fix windows build * Fix doc deploy * Fix doc deploy * Fix doc deploy * Fix doc deploy * Fix Pda histogram_function * fix wrong path * CI gitlab github sync * Fix doc issues * Add stubs for doc * Add stubs for doc * Fix wrong path * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml file * Update .gitlab-ci.yml * Update pyproject.toml * Add github action * Remove gitlab webide config * Remove gitlab webide config * Remove github action demo * Add Zeiss Confocor3 reader * Fix set_header issue Assigning a raw pointer when using set_header can cause a double free. Use copy constructor instead. * Add x, y attribute to Correlator.py * Also run Correlator when accessing axis * Add Confocor3 support * Add Confocor3 support * Cleanup * Cleanup * Fix wrong link * Remove old interface * Remove work in progress file * Fix plot Remove old interface Adjust plot range * Fix plot Remove old interface Adjust plot range * Fix typo * Fix file path * Fix plot Increase number of photons for MLE and measure time. * Bump version * Build on all platforms * Update plot * Update recipe Use libboost-devel instead of boost-cpp Use matplotlib-base instead of matplotlib -> no qt5 dependency * Ignore .cache dir * No gpg check on yum * Fix header issue * Build issues on Windows * Do not use git in conda builds * Fix MY_INSTALL_PREFIX * Conda windows build issues * Remove tools submodule * Add tools submodule --------- Co-authored-by: tpeulen <[email protected]> Co-authored-by: Thomas-Otavio Peulen <[email protected]> Co-authored-by: Administrator <[email protected]>
1 parent 8b30997 commit 5a7dc40

27 files changed

+362
-151
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ dist/
44
doc/auto_examples
55
tmp/
66
.vscode
7+
.cache
78
.vs
89
.ipynb_checkpoints
910
cmake-build-*/

.gitlab-ci.yml

+14-27
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,10 @@ variables:
22
GIT_SSL_NO_VERIFY: "1"
33

44
stages:
5-
# - sync
65
- build
76
- test
87
- deploy
98

10-
# github:push:
11-
# stage: sync
12-
# image: condaforge/mambaforge
13-
# tags:
14-
# - linux
15-
# before_script:
16-
# - git submodule update --init --recursive --remote
17-
# script:
18-
# - git switch -c ${CI_COMMIT_REF_NAME}
19-
# - git push --force https://${GITHUB_TOKEN}@github.com/fluorescence-tools/imp.bff
20-
21-
229
# general definitions
2310
.build: &build
2411
stage: build
@@ -47,21 +34,21 @@ conda:build:lnx:x86:
4734
- linux
4835
- x86_64
4936
conda:build:lnx:ppc64le:
50-
<<: *conda_build_lnx
51-
image:
52-
entrypoint: [ "/bin/bash", "-i", "-c" ]
53-
name: condaforge/linux-anvil-ppc64le
54-
tags:
55-
- linux
56-
- ppc64le
37+
<<: *conda_build_lnx
38+
image:
39+
entrypoint: [ "/bin/bash", "-i", "-c" ]
40+
name: condaforge/linux-anvil-ppc64le
41+
tags:
42+
- linux
43+
- ppc64le
5744
conda:build:lnx:aarch64:
58-
<<: *conda_build_lnx
59-
image:
60-
entrypoint: [ "/bin/bash", "-i", "-c" ]
61-
name: condaforge/linux-anvil-aarch64
62-
tags:
63-
- linux
64-
- aarch64
45+
<<: *conda_build_lnx
46+
image:
47+
entrypoint: [ "/bin/bash", "-i", "-c" ]
48+
name: condaforge/linux-anvil-aarch64
49+
tags:
50+
- linux
51+
- aarch64
6552

6653
conda:build:osx:
6754
<<: *conda_build_posix

conda-recipe/bld.bat

+9-5
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
cd %SRC_DIR%
2-
git submodule update --recursive --init --remote
32

43
echo "Build app wrapper"
54
:: build app wrapper
@@ -10,20 +9,25 @@ if errorlevel 1 exit 1
109
rmdir b2 /s /q
1110
mkdir b2
1211
cd b2
13-
for /f %%A in ('python -c "import platform; print(platform.python_version())"') do set python_version=%%A
14-
echo Python version: %python_version%
12+
13+
REM Call Python with the --version flag to get the version information
14+
for /f "tokens=2 delims= " %%v in ('%PYTHON% --version 2^>^&1') do set PYTHON_VERSION=%%v
15+
16+
REM Extract only the numeric part of the version
17+
for /f "tokens=1-3 delims=." %%a in ("%PYTHON_VERSION%") do set PYTHON_VERSION_NUMERIC=%%a.%%b.%%c
18+
19+
echo Python version: %PYTHON_VERSION_NUMERIC%
1520
cmake .. -G "NMake Makefiles" ^
1621
-DCMAKE_INSTALL_PREFIX="%LIBRARY_PREFIX%" ^
1722
-DCMAKE_PREFIX_PATH="%PREFIX%" ^
1823
-DBUILD_PYTHON_INTERFACE=ON ^
1924
-DCMAKE_BUILD_TYPE=Release ^
2025
-DCMAKE_LIBRARY_OUTPUT_DIRECTORY="%SP_DIR%" ^
2126
-DCMAKE_SWIG_OUTDIR="%SP_DIR%" ^
22-
-DPYTHON_VERSION="%python_version%" ^
27+
-DPYTHON_VERSION="%PYTHON_VERSION_NUMERIC%" ^
2328
-DBUILD_LIBRARY=ON
2429
nmake install
2530

26-
2731
:: Add wrappers to path for each Python command line tool
2832
:: (all files without an extension)
2933
cd %SRC_DIR%\bin

conda-recipe/build.sh

-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
git submodule update --recursive --init --remote
21
mkdir b2 && cd b2
32

43
if [[ "${target_platform}" == osx-* ]]; then

conda-recipe/meta.yaml

+10-13
Original file line numberDiff line numberDiff line change
@@ -13,41 +13,37 @@ build:
1313
number: 1
1414
detect_binary_files_with_prefix: True # [not win]
1515

16+
requirements:
1617
requirements:
1718
build:
18-
- python
19-
- numpy
20-
- doxygen
2119
- {{ compiler('c') }}
2220
- {{ compiler('cxx') }}
2321
- llvm-openmp # [osx]
2422
- libgomp # [linux]
2523
- pkg-config # [not win]
26-
- swig
27-
- ninja
2824
- cmake
25+
- ninja
2926
- make # [linux]
30-
- typing # [py27]
27+
- doxygen
28+
- swig
29+
- python
30+
- numpy
3131
host:
32-
- boost-cpp 1.78 # [py > 27]
33-
- boost-cpp 1.69 # [py27]
32+
- libboost-devel
3433
- hdf5
3534
- python
3635
- setuptools
3736
- typing # [py27]
3837
- numpy
3938
run:
4039
- python
41-
- tqdm # [py > 27]
42-
- tqdm <=4.14 # [py27]
40+
- tqdm
4341
- click
4442
- click-didyoumean
4543
- {{ pin_compatible('numpy') }}
4644
- scikit-image
47-
- matplotlib
45+
- matplotlib-base
4846
- hdf5
49-
- typing # [py27]
50-
- importlib_metadata # [py27]
5147

5248
test:
5349
imports:
@@ -64,3 +60,4 @@ about:
6460
extra:
6561
recipe-maintainers:
6662
- tpeulen
63+
- khemmen
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
"""
2+
=======================================
3+
Confocor3 two channel cross-correlation
4+
=======================================
5+
6+
The raw FCS data format of the Zeiss Confocor3 is relatively simple.
7+
Zeiss Confocor3 raw files store time-difference between photons.
8+
A relatively small header is followed by a set of 32-bit integers that
9+
contain the time difference to the previous registered photon. Photons
10+
registered by different channels are stored in separate files.
11+
12+
In this example two raw files of a Confocor3 are read and merged into
13+
a single photon stream. Next, the merged photon stream is used to compute
14+
the cross-correlation between the two channels.
15+
16+
"""
17+
import pathlib
18+
import numpy as np
19+
20+
import pylab as plt
21+
import tttrlib
22+
23+
#%%
24+
# Reading data
25+
# ------------
26+
# The photon data registered by different detectors are saved in separate files.
27+
# Read the data of all channels that should be correlated into separate containers.
28+
fns = [str(p) for p in pathlib.Path('../../tttr-data/cz/fcs').glob('5a6ce6a348a08e3da9f7c0ab4ee0ce94_R1_P1_K1_Ch*.raw')]
29+
tttr_data = [tttrlib.TTTR(fn, 'CZ-RAW') for fn in fns]
30+
31+
#%%
32+
# We combine the data in different files into a single TTTR container using the header
33+
# of first file as template.
34+
header = tttr_data[0].header
35+
channels = [t.routing_channels[0] for t in tttr_data]
36+
print("Used channels:", channels)
37+
38+
#%%
39+
# You can check the count rates of the channels using the macro time resolution contained
40+
# in the header
41+
macro_time_resolution = header.macro_time_resolution
42+
count_rates = [len(t) / (t.macro_times[-1] * macro_time_resolution) for t in tttr_data]
43+
print("Count rates:", count_rates)
44+
45+
#%%
46+
# Now we merge the data of the two detectors in a single container. The
47+
# marco times need to be sorted first.
48+
macro_times = np.concatenate([t.macro_times for t in tttr_data])
49+
routing_channels = np.concatenate([t.routing_channels for t in tttr_data])
50+
sorted_indices = np.argsort(macro_times)
51+
52+
#%%
53+
# Using the sorted macro times we sort the routing channel numbers
54+
# and the macro times.
55+
routing_channels = routing_channels[sorted_indices]
56+
macro_times = macro_times[sorted_indices]
57+
58+
#%%
59+
# Note: no micro time and no event type in the raw Confocor3 format.
60+
# Thus, we use ones for the micro time and the event type.
61+
micro_times = np.ones_like(macro_times, dtype=np.uint16)
62+
event_types = np.ones_like(macro_times, dtype=np.int8)
63+
64+
#%%
65+
# Using the merged marcro times and channel numbers, we create a new
66+
# TTTR container.
67+
tttr_merged = tttrlib.TTTR()
68+
tttr_merged.set_header(header)
69+
tttr_merged.append_events(
70+
macro_times=macro_times,
71+
micro_times=micro_times,
72+
routing_channels=routing_channels,
73+
event_types=event_types
74+
)
75+
76+
#%%
77+
# The container can be used for standard analysis, e.g., correlations.
78+
settings = {
79+
"n_bins": 9, # n_bins and n_casc defines the settings of the multi-tau
80+
"n_casc": 19, # correlation algorithm
81+
}
82+
83+
# Create correlator
84+
# Caution: x-axis in units of macro time counter
85+
# tttrlib.Correlator is unaware of the calibration in the TTTR object
86+
correlator = tttrlib.Correlator(
87+
channels=([1], [2]),
88+
tttr=tttr_merged,
89+
**settings
90+
)
91+
plt.semilogx(
92+
correlator.x_axis,
93+
correlator.correlation,
94+
label="Corr(1,2)"
95+
)
96+
97+
correlator = tttrlib.Correlator(
98+
channels=([1], [1]),
99+
tttr=tttr_merged,
100+
**settings
101+
)
102+
plt.semilogx(
103+
correlator.x_axis,
104+
correlator.correlation,
105+
label="Corr(1,1)"
106+
)
107+
108+
correlator = tttrlib.Correlator(
109+
channels=([2], [2]),
110+
tttr=tttr_merged,
111+
**settings
112+
)
113+
plt.semilogx(
114+
correlator.x_axis,
115+
correlator.correlation,
116+
label="Corr(2,2)"
117+
)
118+
119+
plt.ylim(1.032, 1.050)
120+
plt.show()
121+

examples/correlation/plot_correlation_cr_filter.py

100755100644
-1
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@
4242
# ``n_bins`` and ``n_casc`` define the settings of the multi-tau correlation steps.
4343
# If ``make_fine`` is set to false the micro time is not used for correlation.
4444
settings = {
45-
"method": "default",
4645
"n_bins": 3,
4746
"n_casc": 27,
4847
"make_fine": False

examples/correlation/plot_full_correlation.py

100755100644
+1-1
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,6 @@
5858
# times as time information, the default value of parameters ``make_fine``
5959
# when creating a new Correlator needs to by modified:
6060
full_corr_settings = {
61-
"method": 'default',
6261
"n_casc": 37, # n_bins and n_casc defines the settings of the multi-tau
6362
"n_bins": 7, # correlation algorithm
6463
"make_fine": True # Use the microtime information (also called "fine" correlation)
@@ -103,5 +102,6 @@
103102
ax.set_xlabel('corr. time (ms)')
104103
ax.set_ylabel('Correlation Amplitude')
105104
ax.legend()
105+
ax.set_ylim(0, 12) # Set y-axis range to 0-12
106106

107107
plt.show()

examples/correlation/plot_gated_correlation.py

100755100644
File mode changed.

examples/correlation/plot_normal_correlation.py

100755100644
+8-9
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,6 @@
3939

4040
# Correlator settings, if the same settings are used repeatedly it is useful to define them once
4141
settings = {
42-
"method": "default",
4342
"n_bins": 7, # n_bins and n_casc defines the settings of the multi-tau
4443
"n_casc": 27, # correlation algorithm
4544
"make_fine": False # Do not use the microtime information
@@ -64,8 +63,8 @@
6463
correlator.set_events(t1, w1, t2, w2)
6564

6665
# scale the x-axis to have units in milliseconds (common unit in FCS)
67-
x = correlator.curve.x * (data.header.macro_time_resolution)
68-
y = correlator.curve.y
66+
x = correlator.x * data.header.macro_time_resolution
67+
y = correlator.y
6968

7069
plt.semilogx(x, y, label="Gp/Gs")
7170

@@ -82,8 +81,8 @@
8281

8382
# no need to scale axis - correlator aware of macro time units
8483
plt.semilogx(
85-
correlator.curve.x,
86-
correlator.curve.y,
84+
correlator.x,
85+
correlator.y,
8786
label="Gp,Gs/Rp,Rs"
8887
)
8988

@@ -95,8 +94,8 @@
9594
)
9695

9796
plt.semilogx(
98-
correlator.x_axis,
99-
correlator.correlation,
97+
correlator.x,
98+
correlator.y,
10099
label="pR,sR"
101100
)
102101

@@ -108,8 +107,8 @@
108107
)
109108

110109
plt.semilogx(
111-
correlator.x_axis,
112-
correlator.correlation,
110+
correlator.x,
111+
correlator.y,
113112
label="pRsR,pGsG"
114113
)
115114

examples/correlation/plot_sliced_correlation.py

100755100644
+1-1
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@
4444
#%%
4545
# Next, we plan to split the TTTR data into separate data chunks. Here, chunk
4646
# the data into at least 5 seconds long pieces. The method ``get_ranges_by_time_window``
47-
# returns an one-dimensional array with the beginning and the end index of each
47+
# returns a one-dimensional array with the beginning and the end index of each
4848
# chunk. Later, we will use these start/stop indices to define TTTR slices
4949
# that will be correlated. To slice the data into time windows we use the
5050
# macro time calibration we computed previously. Note, the last chunk can

0 commit comments

Comments
 (0)