Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
703b6ce
fixed a bug with warning(s)
luigiferiani Jun 13, 2018
64374b8
compressVideo.py: switched to opencv for median blur
luigiferiani Jun 26, 2018
de2c13f
.gitignore: do not track profile and debugger files
luigiferiani Jun 26, 2018
e1a3817
debug_tests: my folder for testing things and doing debugging. Not us…
luigiferiani Jun 26, 2018
4c35d79
compress_test.py: added test for fluorescence video, it works
luigiferiani Jul 4, 2018
96ffc7c
BackgroundSubtractor.py: new method for subtracting the background, u…
luigiferiani Jul 4, 2018
f1c177d
gitignore: added txt in debug_tests folder
luigiferiani Jul 5, 2018
0197c27
BackgroundSubtractor: fixed new subtraction method to deal with both …
luigiferiani Jul 5, 2018
bf7eea5
compress_test.py: updated compression test. However, it is not very u…
luigiferiani Sep 10, 2018
7c5c08d
BackgroundSubtractor.py: implemented Avelino's changes to the PR
luigiferiani Sep 11, 2018
07d0b73
Synced with 1.5.1-beta
luigiferiani Sep 17, 2018
2f7240b
Merge remote-tracking branch 'upstream/master'
luigiferiani Sep 20, 2018
ecfc613
Merge branch 'master' of https://github.com/ver228/tierpsy-tracker
luigiferiani Dec 14, 2018
a4f3c28
smooth_skeletons_table.py: fixed a bug in frames -> seconds conversion
luigiferiani Dec 14, 2018
b8cd52f
Merge remote-tracking branch 'upstream/master'
luigiferiani Feb 5, 2019
c38962e
Added time windows to the summarizer for the tierpsy features.
em812 Feb 24, 2019
c221b1b
Committing macbook pro changes before pulling master. Only trailing s…
luigiferiani Mar 5, 2019
32b5f33
mistake with git, this is the commit that adds check_default_attrs.py
luigiferiani Mar 5, 2019
3c03633
finish merging
luigiferiani Mar 5, 2019
1e57439
brought checkFinalOrientation.py back in line with Avelino's
luigiferiani Mar 5, 2019
146c588
Fixed summary functions to handle the case of empty featuresN files.
em812 Mar 5, 2019
c7b32e5
corrected typo, added constraint to numpy version because the latest …
em812 Mar 10, 2019
4009862
corrected version constraint
em812 Mar 10, 2019
6930b24
added an exception at makedirs, to avoid errors when processes are ru…
em812 Mar 12, 2019
e683a0f
makedirs exception with one line
em812 Mar 12, 2019
5762506
one more makedirs exception
em812 Mar 13, 2019
a274399
Merge branch 'master' of https://github.com/ver228/tierpsy-tracker
em812 May 7, 2019
cb66a29
Added feature selection options in summarizer.
em812 May 8, 2019
288ab83
Merge remote-tracking branch 'upstream/master'
luigiferiani Jun 28, 2019
ee245b3
FOVMultiwellsSplitter.py: attempts to localise square wells
luigiferiani Jul 4, 2019
f7e72df
Added support to square 96WP
luigiferiani Jul 11, 2019
78f74bd
Features can now be calculates on a well-by-well basis
luigiferiani Aug 7, 2019
60c192e
Compression: Apply the FOV splitting at the masking stage => apply bl…
luigiferiani Aug 13, 2019
bbbc0e2
SKE_CREATE: solved a bug in trajectories2Skeletons. Also performance …
luigiferiani Aug 13, 2019
ddd88e6
FOVMultiWellSplitter.py: refactored the code a bit by only keeping th…
luigiferiani Aug 13, 2019
72650a1
MWTrackerViewer.py: can now deal with the column 'well_name' being pr…
luigiferiani Aug 13, 2019
89b295c
get_tierpsy_features.py: now uses new FOVMultiWellSplitter method for…
luigiferiani Aug 13, 2019
331b2ce
Summary: can now deal with MultiWell plates
luigiferiani Aug 13, 2019
8f6de86
FOVMultiWellsSplitter.py: bug fix. Due to incomplete refactoring was …
luigiferiani Sep 3, 2019
e133792
FOVMultiWellsSplitter.py: a bug in skimage means that min_distance is…
luigiferiani Sep 5, 2019
b99d1dd
compressVideo.py: fixed bug that called the fov splitter class withou…
luigiferiani Sep 5, 2019
0d1c2a0
MWTrackerViewer.py: if /fov_wells exists in the masked video, draw th…
luigiferiani Sep 5, 2019
752a66f
fixing a bug that was affecting the new way of indexing cameras/chann…
luigiferiani Oct 22, 2019
f40910e
tierpsy/analysis/split_fov/wellsbymin.py: attempts to find grid of we…
luigiferiani Oct 22, 2019
57c8766
FOVMultiWellsSplitter.py: bugfix - had left call to debugger
luigiferiani Oct 22, 2019
b588cf9
compressVideo.py: stores the background image, subtracts the bg from …
luigiferiani Oct 23, 2019
991f236
compressVideo.py: changed to take advantage of code Avelino had alrea…
luigiferiani Oct 25, 2019
86e61cb
trackProvenance.py: BUGFIX - was not tracking tierpsy versions correctly
luigiferiani Nov 4, 2019
d9d8dbb
BUGFIX: OpenCV 3.x.x is the only one that asks for 3 outputs from vc2…
luigiferiani Nov 4, 2019
19d2b01
compressVideo.py: added save_interval to attributes of /bgnd in the m…
luigiferiani Nov 5, 2019
5bcea8e
tierpsy/helper/misc/misc.py: bugfix
luigiferiani Nov 5, 2019
4b77345
speedup (hopefully) bgnd initialisation by taking advantage of imgsto…
luigiferiani Nov 5, 2019
5beae5b
compressVideo.py: bugfix (wrong 'save_interval' in /bgnd's _v_attrs)
luigiferiani Nov 5, 2019
6cb9977
FOVMultiWellsSplitter.py
luigiferiani Nov 12, 2019
f924d10
split_fov/helper.py
luigiferiani Nov 12, 2019
3d5efe5
bugfix: edit more parameters now works with multiwell plates
luigiferiani Nov 18, 2019
0d34b6f
FOVMultiWellsSplitter.py: better __init__ from /fov_wells
luigiferiani Nov 30, 2019
e303984
Merge remote-tracking branch 'upstream/master'
luigiferiani Nov 30, 2019
b8f7c15
cleaned up files for PR
luigiferiani Nov 30, 2019
8a7668d
Refined feature selection option in summarizer
em812 Dec 10, 2019
fa3862f
resolved conflicts between luigiferiani-FOVsplitting and em812-summar…
em812 Dec 10, 2019
f02e918
Merge branch 'luigiferiani-FOVsplitting_PR' to make a single pull req…
em812 Dec 10, 2019
7502698
Merge branch 'master' into master
ver228 Dec 11, 2019
04711b5
Update misc.py
ver228 Dec 11, 2019
797727e
Update misc.py
ver228 Dec 11, 2019
325daf1
Merge remote-tracking branch 'upstream/master'
luigiferiani Dec 11, 2019
ace87ff
add headers to features and filenames summaries
em812 Dec 16, 2019
a36a528
merge changes with Avelino
em812 Dec 16, 2019
eddc6fc
added line break between header and feature summaries
em812 Dec 16, 2019
e33e643
resolved conflicts with avelino master
em812 Dec 16, 2019
40755ad
Merge pull request #2 from em812/master
luigiferiani Dec 16, 2019
72f7a9c
Revert "Fixed feature summaries"
luigiferiani Dec 17, 2019
1ab698b
Merge pull request #3 from Tierpsy/revert-2-master
luigiferiani Dec 17, 2019
a4ee8f2
Merge remote-tracking branch 'official_upstream/master' into merging_…
luigiferiani Dec 17, 2019
90b20e6
summary/collect.py, summary/helper.py: fixed bug in features summarie…
luigiferiani Dec 17, 2019
b1e12bc
removed references to IS_OPENCV3
luigiferiani Dec 17, 2019
dc55502
Merge pull request #4 from luigiferiani/merging_tierpsy
em812 Dec 17, 2019
bd0b68c
abbreviate features
ilbarlow Dec 17, 2019
f8327da
Merge pull request #5 from ilbarlow/abbreviate_features
em812 Dec 17, 2019
fe22675
gui abs features only
ilbarlow Dec 17, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 37 additions & 40 deletions tierpsy/analysis/compress/compressVideo.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,16 +122,16 @@ def normalizeImage(img):
# normalise image intensities if the data type is other
# than uint8
image = image.astype(np.double)

imax = img.max()
imin = img.min()
factor = 255/(imax-imin)

imgN = ne.evaluate('(img-imin)*factor')
imgN = imgN.astype(np.uint8)

return imgN, (imin, imax)

def reduceBuffer(Ibuff, is_light_background):
if is_light_background:
return np.min(Ibuff, axis=0)
Expand Down Expand Up @@ -175,36 +175,36 @@ def createImgGroup(fid, name, tot_frames, im_height, im_width, is_expandable=Tru

return img_dataset

def initMasksGroups(fid, expected_frames, im_height, im_width,
def initMasksGroups(fid, expected_frames, im_height, im_width,
attr_params, save_full_interval, is_expandable=True):

# open node to store the compressed (masked) data
mask_dataset = createImgGroup(fid, "/mask", expected_frames, im_height, im_width, is_expandable)


tot_save_full = (expected_frames // save_full_interval) + 1
full_dataset = createImgGroup(fid, "/full_data", tot_save_full, im_height, im_width, is_expandable)
full_dataset._v_attrs['save_interval'] = save_full_interval


assert all(x in ['expected_fps', 'is_light_background', 'microns_per_pixel'] for x in attr_params)
set_unit_conversions(mask_dataset, **attr_params)
set_unit_conversions(full_dataset, **attr_params)

if is_expandable:
mean_intensity = fid.create_earray('/',
mean_intensity = fid.create_earray('/',
'mean_intensity',
atom=tables.Float32Atom(),
shape=(0,),
expectedrows=expected_frames,
filters=TABLE_FILTERS)
else:
mean_intensity = fid.create_carray('/',
mean_intensity = fid.create_carray('/',
'mean_intensity',
atom=tables.Float32Atom(),
shape=(expected_frames,),
filters=TABLE_FILTERS)

return mask_dataset, full_dataset, mean_intensity


Expand All @@ -230,13 +230,13 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,
'''

#get the default values if there is any bad parameter
output = compress_defaults(masked_image_file,
expected_fps,
buffer_size = buffer_size,
output = compress_defaults(masked_image_file,
expected_fps,
buffer_size = buffer_size,
save_full_interval = save_full_interval)

buffer_size = output['buffer_size']
save_full_interval = output['save_full_interval']
buffer_size = output['buffer_size']
save_full_interval = output['save_full_interval']

if len(bgnd_param) > 0:
is_bgnd_subtraction = True
Expand All @@ -253,14 +253,14 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,

# processes identifier.
base_name = masked_image_file.rpartition('.')[0].rpartition(os.sep)[-1]

# select the video reader class according to the file type.
vid = selectVideoReader(video_file)

# delete any previous if it existed
with tables.File(masked_image_file, "w") as mask_fid:
pass

#Extract metadata
if is_extract_timestamp:
# extract and store video metadata using ffprobe
Expand All @@ -270,7 +270,7 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,

else:
expected_frames = 1

# Initialize background subtraction if required

if is_bgnd_subtraction:
Expand All @@ -282,28 +282,28 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,
frame_number = 0
full_frame_number = 0
image_prev = np.zeros([])

# Initialise FOV splitting if needed
if is_fov_tosplit:
# masked video does not exist yet so have to initialise from data
# masked video does not exist yet so have to initialise from data
# use either background or first frame
if is_bgnd_subtraction:
img_fov = bgnd_subtractor.bgnd.astype(np.uint8)
else:
ret, img_fov = vid.read()
# close and reopen the video, to restart from the beginning
vid.release()
vid = selectVideoReader(video_file);
# TODO: change class creator so it only needs the video name? by using
vid = selectVideoReader(video_file);
# TODO: change class creator so it only needs the video name? by using
# Tierpsy's functions such as selectVideoReader it can then read the first image by itself

camera_serial = parse_camera_serial(masked_image_file)

fovsplitter = FOVMultiWellsSplitter(img_fov,
camera_serial=camera_serial,
px2um=microns_per_pixel,
**fovsplitter_param)



# initialize timers
Expand All @@ -325,26 +325,26 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,
microns_per_pixel = microns_per_pixel,
is_light_background = int(mask_param['is_light_background'])
)
mask_dataset, full_dataset, mean_intensity = initMasksGroups(mask_fid,
mask_dataset, full_dataset, mean_intensity = initMasksGroups(mask_fid,
expected_frames, vid.height, vid.width,
attr_params, save_full_interval)

if is_bgnd_subtraction:
bg_dataset = createImgGroup(mask_fid, "/bgnd", 1, vid.height, vid.width, is_expandable=False)
# because we only save the one background:
bg_dataset._v_attrs['save_interval'] = vid.frame_max-vid.first_frame + 1
bg_dataset._v_attrs['save_interval'] = vid.frame_max-vid.first_frame + 1
bg_dataset[0,:,:] = img_fov

if vid.dtype != np.uint8:
# this will worm as flags to be sure that the normalization took place.
normalization_range = mask_fid.create_earray('/',
normalization_range = mask_fid.create_earray('/',
'normalization_range',
atom=tables.Float32Atom(),
shape=(0, 2),
expectedrows=expected_frames,
filters=TABLE_FILTERS
)

while frame_number < max_frame:

ret, image = vid.read()
Expand Down Expand Up @@ -396,23 +396,23 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,
Ibuff = Ibuff[:ind_buff + 1]

# mask buffer and save data into the hdf5 file
if (ind_buff == buffer_size - 1 or ret == 0) and Ibuff.size > 0:
if (ind_buff == buffer_size - 1 or ret == 0) and Ibuff.size > 0:
if is_bgnd_subtraction:
Ibuff_b = bgnd_subtractor.apply(Ibuff, frame_number)
else:
Ibuff_b = Ibuff

#calculate the max/min in the of the buffer
img_reduce = reduceBuffer(Ibuff_b, mask_param['is_light_background'])

mask = getROIMask(img_reduce, **mask_param)

Ibuff *= mask

# now apply the well_mask if is MWP
if is_fov_tosplit:
fovsplitter.apply_wells_mask(Ibuff) # Ibuff will be modified after this

# add buffer to the hdf5 file
frame_first_buff = frame_number - Ibuff.shape[0]
mask_dataset.append(Ibuff)
Expand All @@ -421,20 +421,17 @@ def compressVideo(video_file, masked_image_file, mask_param, expected_fps=25,
# calculate the progress and put it in a string
progress_str = progressTime.get_str(frame_number)
print_flush(base_name + ' ' + progress_str)

# finish process
if ret == 0:
break

# close the video
vid.release()

# save fovsplitting data
if is_fov_tosplit:
fovsplitter.write_fov_wells_to_file(masked_image_file)

read_and_save_timestamp(masked_image_file)
print_flush(base_name + ' Compressed video done.')



Loading