Skip to content

Commit a94f40e

Browse files
mariana-gferreiramariana-gferreira
committed
notebook updates
1 parent 57bdd04 commit a94f40e

3 files changed

Lines changed: 133 additions & 19 deletions

File tree

‎notebooks/0.1_summary_fov_per_sample_id.ipynb‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,20 +5,20 @@
55
"id": "1415df36",
66
"metadata": {},
77
"source": [
8-
"# 0. Split the dataset into train and test subsets\n",
8+
"# 0.1. Generate a summary of the available data\n",
99
"Data from the same origin should be kept in the same subset to avoid data leakage.\n",
1010
"\n",
1111
"This notebook crosses the image names with the identifiers in the csv file to get the number of FOVs (Fields of View) for each sample. Also counting the number of empty FOVs.\n",
1212
"\n",
13-
"These counts can then be used to split the dataset into train and test subsets. Usually 80% for training and 20% for testing.\n"
13+
"These counts can then be used to do an informed split of the dataset into train and test subsets. Usually 80% for training and 20% for testing.\n"
1414
]
1515
},
1616
{
1717
"cell_type": "markdown",
1818
"id": "d1ebf2ae",
1919
"metadata": {},
2020
"source": [
21-
"## 0.1. Load libraries and custom functions\n",
21+
"## 0.1.1. Load libraries and custom functions\n",
2222
"\n",
2323
"Load the `pandas`, `os`, and `skimage` libraries.\n"
2424
]
@@ -95,7 +95,7 @@
9595
"id": "0baddb91",
9696
"metadata": {},
9797
"source": [
98-
"## 0.2. Code\n"
98+
"## 0.1.2. Code\n"
9999
]
100100
},
101101
{

‎notebooks/0.2_label_watershed.ipynb‎

Lines changed: 49 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
"cell_type": "markdown",
55
"metadata": {},
66
"source": [
7-
"# 0.1. 3D Seeded Watershed Segmentation\n",
7+
"# 0.2. 3D Seeded Watershed Segmentation\n",
88
"\n",
99
"This script uses the `watershed` algorithm from `skimage.segmentation` to perform seeded segmentation on a label image where a cluster of objects is classified with a single label. \n",
1010
"\n",
@@ -15,7 +15,7 @@
1515
"cell_type": "markdown",
1616
"metadata": {},
1717
"source": [
18-
"## 0.1.1. Load Python Libraries\n",
18+
"## 0.2.1. Load Python Libraries\n",
1919
"\n",
2020
"Load the necessary Python libraries for image processing and visualization. \n",
2121
"\n",
@@ -50,7 +50,7 @@
5050
"cell_type": "markdown",
5151
"metadata": {},
5252
"source": [
53-
"## 0.1.2. Load Functions\n",
53+
"## 0.2.2. Load Functions\n",
5454
"\n",
5555
"Load custom functions to handle image processing tasks."
5656
]
@@ -267,7 +267,7 @@
267267
"cell_type": "markdown",
268268
"metadata": {},
269269
"source": [
270-
"## 0.1.3. Pipeline"
270+
"## 0.2.3. Pipeline"
271271
]
272272
},
273273
{
@@ -306,7 +306,51 @@
306306
"cell_type": "code",
307307
"execution_count": null,
308308
"metadata": {},
309-
"outputs": [],
309+
"outputs": [
310+
{
311+
"name": "stdout",
312+
"output_type": "stream",
313+
"text": [
314+
"Processing C4-16122021_Label45_367L_w3_1076_100x_0p21_01_scaled_oriScale.tif...\n",
315+
"Processing label 1...\n",
316+
"Only one conected component found for label 1, no markers for watershed applied.\n",
317+
"Transfering to relabeled array\n",
318+
"Processing label 2...\n",
319+
"Only one conected component found for label 2, no markers for watershed applied.\n",
320+
"Transfering to relabeled array\n",
321+
"Processing label 3...\n",
322+
"Processing label 4...\n",
323+
"Only one conected component found for label 4, no markers for watershed applied.\n",
324+
"Transfering to relabeled array\n",
325+
"Processing label 5...\n",
326+
"Only one conected component found for label 5, no markers for watershed applied.\n",
327+
"Transfering to relabeled array\n",
328+
"Processing label 6...\n",
329+
"Processing label 7...\n",
330+
"Processing label 8...\n",
331+
"Only one conected component found for label 8, no markers for watershed applied.\n",
332+
"Transfering to relabeled array\n",
333+
"Processing label 9...\n",
334+
"Only one conected component found for label 9, no markers for watershed applied.\n",
335+
"Transfering to relabeled array\n",
336+
"Processing C4-12012022_Label46_367L_Cd16_100x_0p21_01_scaled_oriScale.tif...\n",
337+
"Processing label 1...\n",
338+
"Processing C3-03022022_Label49_t1_100x_0p21_02_POS_current_scaled_oriScale.tif...\n",
339+
"Processing label 1...\n",
340+
"Processing label 2...\n",
341+
"Only one conected component found for label 2, no markers for watershed applied.\n",
342+
"Transfering to relabeled array\n",
343+
"Processing label 3...\n",
344+
"Processing label 4...\n",
345+
"Processing C4-26012022_Label48_t1strep_100x_0p21_03_POS_current_scaled_oriScale.tif...\n",
346+
"Processing label 1...\n",
347+
"Only one conected component found for label 1, no markers for watershed applied.\n",
348+
"Transfering to relabeled array\n",
349+
"Processing C4-16122021_Label45_367L_w3_1076_100x_0p21_01_scaled_oriScale 2.tif...\n",
350+
"C4-16122021_Label45_367L_w3_1076_100x_0p21_01_scaled_oriScale 2.tif is likely not a label image. Skipping...\n"
351+
]
352+
}
353+
],
310354
"source": [
311355
"# Create The save directory if it does not exist\n",
312356
"# If it exists, it will not raise an error\n",

‎notebooks/0.3_normalize_and_crop.ipynb‎

Lines changed: 80 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"id": "7cfb11f5",
66
"metadata": {},
77
"source": [
8-
"# 0.2. Normalize and Crop Training Data\n",
8+
"# 0.3. Normalize and Crop Training Data\n",
99
"\n",
1010
"Due to the large size of the training data, we will normalize and crop the images to a smaller size. This will help in reducing the computational load and make it easier to work with the data.\n",
1111
"\n",
@@ -17,7 +17,7 @@
1717
"id": "b21b2780",
1818
"metadata": {},
1919
"source": [
20-
"## 0.2.1. Load Python Libraries"
20+
"## 0.3.1. Load Python Libraries"
2121
]
2222
},
2323
{
@@ -39,7 +39,7 @@
3939
"id": "e44e1ad1",
4040
"metadata": {},
4141
"source": [
42-
"## 0.2.2. Load Custom Functions"
42+
"## 0.3.2. Load Custom Functions"
4343
]
4444
},
4545
{
@@ -119,7 +119,7 @@
119119
"id": "105eb73d",
120120
"metadata": {},
121121
"source": [
122-
"## 0.2.3. Code to Normalize and Crop Training Data"
122+
"## 0.3.3. Code to Normalize and Crop Training Data"
123123
]
124124
},
125125
{
@@ -143,19 +143,24 @@
143143
"metadata": {},
144144
"outputs": [],
145145
"source": [
146-
"# The path to the images and masks should be specified in the variables `img_directory`, `og_mask_directory`, and `watershed_label_directory`.\n",
146+
"# The path to the images and masks should be specified in the variables `img_directory`, and `watershed_label_directory`.\n",
147147
"img_directory = \"directory/to/images\"\n",
148148
"\n",
149-
"og_mask_directory = \"directory/to/original/masks\"\n",
150-
"\n",
151149
"watershed_label_directory = \"directory/to/watershed/labels\"\n",
152150
"\n",
151+
"# OPTIONAL: If you have an original mask directory, specify it here.\n",
152+
"# This is used to reduce the amount of similiar crops generated by cropping each label in a cluster individually.\n",
153+
"og_mask_directory = \"directory/to/original/masks\" # Set to None if not used\n",
154+
"\n",
153155
"# Provide the directories to store the cropped images and labels\n",
154156
"# They will be created if they do not exist\n",
155157
"cropped_img_directory = \"directory/to/cropped/images\"\n",
156158
"cropped_lbl_directory = \"directory/to/cropped/labels\"\n",
157159
"\n",
158160
"# Size of the crop in the x and y dimensions\n",
161+
"# Should be at least twice the patch size used for training the model\n",
162+
"# This is to ensure that an edge crop still has a sufficient size for training\n",
163+
"# For example, if the patch size is 128, a crop size of 256 is recommended.\n",
159164
"crop_size_xy = 256\n",
160165
"\n",
161166
"# Minimum size of the crop in the z-dimension\n",
@@ -181,9 +186,64 @@
181186
{
182187
"cell_type": "code",
183188
"execution_count": null,
184-
"id": "48a21f98",
189+
"id": "b60a1b77",
185190
"metadata": {},
186-
"outputs": [],
191+
"outputs": [
192+
{
193+
"name": "stdout",
194+
"output_type": "stream",
195+
"text": [
196+
"DONE: 08082024_rLabel_014.2_TAMRA_sense_P00002_C4scaled_oriScale.tif\n",
197+
"DONE: 08082024_rLabel_014.2_TAMRA_sense_P00006_C4scaled_oriScale.tif\n",
198+
"DONE: 08082024_rLabel_014.2_TAMRA_sense_P00009_C4scaled_oriScale.tif\n",
199+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00013_C4scaled_oriScale.tif\n",
200+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00017_C4scaled_oriScale.tif\n",
201+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00020_C4scaled_oriScale.tif\n",
202+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00023_C4scaled_oriScale.tif\n",
203+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00027_C4scaled_oriScale.tif\n",
204+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00031_C4scaled_oriScale.tif\n",
205+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00033_C4scaled_oriScale.tif\n",
206+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00045_C4scaled_oriScale.tif\n",
207+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00049_C4scaled_oriScale.tif\n",
208+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00051_C4scaled_oriScale.tif\n",
209+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00052_C4scaled_oriScale.tif\n",
210+
"DONE: 21082024_rLabel_024_TAMRA_sense_P00055_C4scaled_oriScale.tif\n",
211+
"DONE: C3-03022022_Label49_t1_100x_0p21_02_POS_current_scaled_oriScale.tif\n",
212+
"DONE: C3-03022022_Label49_t1_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
213+
"DONE: C3-03022022_Label49_t3_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
214+
"DONE: C3-03022022_Label49_t3_100x_0p21_04_POS_current_scaled_oriScale.tif\n",
215+
"DONE: C3-26012022_Label48_t2_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
216+
"DONE: C4-02122021_Label43_label1_343_0p25_100x_0p21_01_scaled_oriScale.tif\n",
217+
"DONE: C4-02122021_Label43_label1_343_0p25_100x_0p21_02_scaled_oriScale.tif\n",
218+
"DONE: C4-02122021_Label43_label1_343_0p25_100x_0p21_04_scaled_oriScale.tif\n",
219+
"DONE: C4-03022022_Label49_t2_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
220+
"DONE: C4-03022022_Label49_t2_100x_0p21_04_POS_current_scaled_oriScale.tif\n",
221+
"DONE: C4-12012022_Label46_367L_BCd16low_100x_0p21_02_scaled_oriScale.tif\n",
222+
"DONE: C4-12012022_Label46_367L_Cd16_100x_0p21_01_scaled_oriScale.tif\n",
223+
"DONE: C4-12012022_Label46_367L_Cd16_100x_0p21_03_scaled_oriScale.tif\n",
224+
"DONE: C4-16122021_Label44_CD16_367L_w1_closetolabel_100x_0p21_01_scaled_oriScale.tif\n",
225+
"DONE: C4-16122021_Label44_CD16_367L_w1_closetolabel_100x_0p21_02_scaled_oriScale.tif\n",
226+
"DONE: C4-16122021_Label44_CD16_367L_w1_closetolabel_100x_0p21_03_scaled_oriScale.tif\n",
227+
"DONE: C4-16122021_Label44_CD16_367L_w1_closetolabel_100x_0p21_04_scaled_oriScale.tif\n",
228+
"DONE: C4-16122021_Label45_367L_w3_1076_100x_0p21_01_scaled_oriScale.tif\n",
229+
"DONE: C4-16122021_Label45_367L_w3_1076_100x_0p21_03_scaled_oriScale.tif\n",
230+
"DONE: C4-16122021_Label45_367L_w3_1077_100x_0p21_01_scaled_oriScale.tif\n",
231+
"DONE: C4-16122021_Label45_367L_w3_1079_100x_0p21_03_scaled_oriScale.tif\n",
232+
"DONE: C4-26012022_Label48_t1_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
233+
"DONE: C4-26012022_Label48_t1strep_100x_0p21_02_POS_current_scaled_oriScale.tif\n",
234+
"DONE: C4-26012022_Label48_t1strep_100x_0p21_03_POS_current_scaled_oriScale.tif\n",
235+
"DONE: L72_w6_P00124_scaled_oriScale.tif\n",
236+
"DONE: L74_w9_P00101_scaled_oriScale.tif\n",
237+
"DONE: L74_w9_P00107_scaled_oriScale.tif\n",
238+
"DONE: rLabel_012.2_TAMRA_sense_P00019_C4scaled_oriScale.tif\n",
239+
"DONE: rLabel_012.2_TAMRA_sense_P00028_C4scaled_oriScale.tif\n",
240+
"DONE: rLabel_012.2_TAMRA_sense_P00033_C4scaled_oriScale.tif\n",
241+
"DONE: rLabel_012.2_TAMRA_sense_P00034_C4scaled_oriScale.tif\n",
242+
"DONE: rLabel_012.2_TAMRA_sense_P00041_C4scaled_oriScale.tif\n",
243+
"DONE: rLabel_012.2_TAMRA_sense_P00042_C4scaled_oriScale.tif\n"
244+
]
245+
}
246+
],
187247
"source": [
188248
"# Get the list of files in the specified image directory\n",
189249
"img_dir_list = sorted(os.listdir(img_directory))\n",
@@ -193,9 +253,19 @@
193253
" # only process files with .tif or .tiff extensions\n",
194254
" if file.endswith((\".tif\", \".tiff\")):\n",
195255
" img = imread(os.path.join(img_directory, file))\n",
196-
" mask = imread(os.path.join(og_mask_directory, file))\n",
197256
" lbl = imread(os.path.join(watershed_label_directory, file))\n",
198257
"\n",
258+
" # Handle different cases for og_mask_directory\n",
259+
" try:\n",
260+
" # Check if variable exists and is a valid directory path\n",
261+
" if og_mask_directory and os.path.isdir(og_mask_directory):\n",
262+
" mask = imread(os.path.join(og_mask_directory, file))\n",
263+
" else:\n",
264+
" mask = np.copy(lbl)\n",
265+
" except (NameError, TypeError):\n",
266+
" # Variable doesn't exist or is None\n",
267+
" mask = np.copy(lbl)\n",
268+
"\n",
199269
" # Normalize the image from 1 to 99.8 percentile\n",
200270
" img = normalize(img, 1, 99.8, axis=(0, 1, 2))\n",
201271
"\n",

0 commit comments

Comments
 (0)