Hello! I've found a performance issue in utils/data_loader.py: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.
Detailed description is listed below:
- data_loader.py:
train_dataset.batch(batchsize)(line 103) should be called before train_dataset.map(parse_data, num_parallel_calls=2)(line 101).
- data_loader.py:
val_dataset.batch(batchsize)(line 140) should be called before val_dataset.map(parse_data_without_augmentation)(line 138).
- data_loader.py:
train_dataset.batch(batchsize)(line 194) should be called before train_dataset.map(parse_single_record, num_parallel_calls=4)(line 192).
Besides, you need to check the function called in map()(e.g., parse_single_record called in train_dataset.map(parse_single_record, num_parallel_calls=4)) whether to be affected or not to make the changed code work properly. For example, if parse_single_record needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).
Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
Hello! I've found a performance issue in utils/data_loader.py:
batch()should be called beforemap(), which could make your program more efficient. Here is the tensorflow document to support it.Detailed description is listed below:
train_dataset.batch(batchsize)(line 103) should be called beforetrain_dataset.map(parse_data, num_parallel_calls=2)(line 101).val_dataset.batch(batchsize)(line 140) should be called beforeval_dataset.map(parse_data_without_augmentation)(line 138).train_dataset.batch(batchsize)(line 194) should be called beforetrain_dataset.map(parse_single_record, num_parallel_calls=4)(line 192).Besides, you need to check the function called in
map()(e.g.,parse_single_recordcalled intrain_dataset.map(parse_single_record, num_parallel_calls=4)) whether to be affected or not to make the changed code work properly. For example, ifparse_single_recordneeds data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.