Skip to content

Allow creating images with a different grid for the gain map. #2397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Allow creating images with a different grid for the gain map. #2397

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion include/avif/internal.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -774,7 +774,7 @@ typedef struct avifSequenceHeader
AVIF_NODISCARD avifBool avifSequenceHeaderParse(avifSequenceHeader * header, const avifROData * sample, avifCodecType codecType);

// ---------------------------------------------------------------------------
// gain maps
// Gain maps

#if defined(AVIF_ENABLE_EXPERIMENTAL_GAIN_MAP)

Expand All @@ -786,6 +786,37 @@ avifResult avifFindMinMaxWithoutOutliers(const float * gainMapF, int numPixels,

#endif // AVIF_ENABLE_EXPERIMENTAL_GAIN_MAP

// ---------------------------------------------------------------------------
// Internal encode
Copy link
Collaborator

@wantehchang wantehchang Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is a comment on the pull request.) I probably would add the generated test images to the source tree rather than generate them on the fly during testing.

This PR consists of two parts. The avifEncoderInternalOptions part is straightforward. On the other hand, it is also easy to rewrite this part from scratch when necessary -- one just needs to make simple changes to the avifWriteToneMappedImagePayload() function. So it is less valuable to check in this part.

The avifEncoderAddImageInternal part is more complicated. It could be difficult to maintain. The new function parameters also need to be documented.

//
// These functions/options give extra flexibility to create non standard images for use in testing.
Copy link
Collaborator

@wantehchang wantehchang Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: non standard => nonstandard


typedef struct avifEncoderInternalOptions
{
#if defined(AVIF_ENABLE_EXPERIMENTAL_GAIN_MAP)
// Options related to the 'tmap' (tone mapped image) box.
uint8_t tmapVersion; // Value that should be written for the 'version' field (use default if 0)
uint16_t tmapMinimumVersion; // Value that should be written for the 'minimum_version' field (use default if 0)
uint16_t tmapWriterVersion; // Value that should be written for the 'writerVersion' field (use default if 0)
avifBool tmapAddExtraBytes; // Add arbitrary bytes at the end of the box
Copy link
Collaborator

@wantehchang wantehchang Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: replace "the box" with the name of the class (ToneMapImage or GainMapMetadata?)

#endif
char dummy; // Avoid emptry struct error when AVIF_ENABLE_EXPERIMENTAL_GAIN_MAP is off
Copy link
Collaborator

@wantehchang wantehchang Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: dummy => unused

} avifEncoderInternalOptions;

// Sets extra encoding options.
void avifEncoderSetInternalOptions(avifEncoder * encoder, const avifEncoderInternalOptions * internalOptions);

// Variant of avifEncoderAddImageGrid() that allows creating images where 'gridCols' and 'gridRows' differ for
// the base image and the gain map image.
avifResult avifEncoderAddImageGridInternal(avifEncoder * encoder,
uint32_t gridCols,
uint32_t gridRows,
const avifImage * const * cellImages,
uint32_t gainMapGridCols,
uint32_t gainMapGridRows,
const avifImage * const * gainMapCellImages,
avifAddImageFlags addImageFlags);

#define AVIF_INDEFINITE_DURATION64 UINT64_MAX
#define AVIF_INDEFINITE_DURATION32 UINT32_MAX

Expand Down
147 changes: 114 additions & 33 deletions src/write.c
View file
Open in desktop

Large diffs are not rendered by default.

44 changes: 27 additions & 17 deletions tests/data/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -500,8 +500,9 @@ exiftool "-icc_profile<=p3.icc" paris_exif_xmp_icc_gainmap_bigendian.jpg

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a 4x3 color grid, a 4x3 alpha grid, and a 2x2 gain map grid.

Expand All @@ -511,8 +512,9 @@ Contains a 4x3 color grid, a 4x3 alpha grid, and a 2x2 gain map grid.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a single color image, single alpha image, and a 2x2 gain map grid.

Expand All @@ -522,8 +524,9 @@ Contains a single color image, single alpha image, and a 2x2 gain map grid.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a 4x3 color grid, a 4x3 alpha grid, and a single gain map image.

Expand All @@ -533,8 +536,9 @@ Contains a 4x3 color grid, a 4x3 alpha grid, and a single gain map image.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a gain map with the `version` field set to 99 in the tmap box.
`minimum_version` and `writer_version` are 0.
Expand All @@ -545,8 +549,9 @@ Contains a gain map with the `version` field set to 99 in the tmap box.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a gain map with the `minimum_version` field set to 99 in the tmap box.
`version` and `writer_version` are 0.
Expand All @@ -557,8 +562,9 @@ Contains a gain map with the `minimum_version` field set to 99 in the tmap box.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a gain map with the `writer_version` field set to 99 in the tmap box,
and some extra unexpected bytes at the end of the gain map metadata.
Expand All @@ -570,8 +576,9 @@ and some extra unexpected bytes at the end of the gain map metadata.

License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source: generated with a modified libavif at https://github.com/maryla-uc/libavif/tree/weirdgainmaps
by running `./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateGainMapImages ../tests/data/` 
Source: generated by `avifgainmaptest.cc`. To update, set `kUpdateTestImages` to true
in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

Contains a gain map with some extra unexpected bytes at the end of the gain map metadata.
`version`, `minimum_version` and `writer_version` are 0.
Expand Down Expand Up @@ -631,7 +638,8 @@ SDR image with a gain map to allow tone mapping to HDR.
![](seine_sdr_gainmap_big_srgb.avif)

Source : modified version of `seine_sdr_gainmap_srgb.avif` with an upscaled gain map, generated using libavif's API.
See `CreateTestImages` in `avifgainmaptest.cc` (set kUpdateTestImages to update images).
To update, set `kUpdateTestImages` to true in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

SDR image with a gain map to allow tone mapping to HDR. The gain map's width and height are doubled compared to the base image.
This is an atypical image just for testing. Typically, the gain map would be either the same size or smaller as the base image.
Expand All @@ -643,7 +651,8 @@ This is an atypical image just for testing. Typically, the gain map would be eit
License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source : created from `seine_hdr_srgb.avif` (for the base image) and `seine_sdr_gainmap_srgb.avif` (for the gain map) with libavif's API.
See `CreateTestImages` in `avifgainmaptest.cc` (set kUpdateTestImages to update images).
To update, set `kUpdateTestImages` to true in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

HDR image with a gain map to allow tone mapping to SDR.

Expand All @@ -654,7 +663,8 @@ HDR image with a gain map to allow tone mapping to SDR.
License: [same as libavif](https://github.com/AOMediaCodec/libavif/blob/main/LICENSE)

Source : modified version of `seine_hdr_gainmap_srgb.avif` with a downscaled gain map, generated using libavif's API.
See `CreateTestImages` in `avifgainmaptest.cc` (set kUpdateTestImages to update images).
To update, set `kUpdateTestImages` to true in `avifgainmaptest.cc` and run the test:
`./tests/avifgainmaptest --gtest_filter=GainMapTest.CreateTestImages ../tests/data/`

SDR image with a gain map to allow tone mapping to HDR. The gain map's width and height are halved compared to the base image.

Expand Down
Binary file modified tests/data/color_grid_alpha_grid_gainmap_nogrid.avif
View file
Open in desktop
Binary file not shown.
Binary file modified tests/data/color_grid_gainmap_different_grid.avif
View file
Open in desktop
Binary file not shown.
Binary file modified tests/data/seine_hdr_gainmap_small_srgb.avif
View file
Open in desktop
Binary file not shown.
128 changes: 128 additions & 0 deletions tests/gtest/avifgainmaptest.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,74 @@ TEST(GainMapTest, EncodeDecodeGrid) {
// .write(reinterpret_cast<char*>(encoded.data), encoded.size);
}

void MakeTestImage(const std::string& path, uint32_t grid_cols,
uint32_t grid_rows, uint32_t cell_width,
uint32_t cell_height, uint32_t gain_map_grid_cols,
uint32_t gain_map_grid_rows, uint32_t gain_map_cell_width,
uint32_t gain_map_cell_height, uint8_t tmap_version = 0,
uint16_t minimum_version = 0, uint16_t writer_version = 0,
bool add_extra_bytes = false) {
std::vector<ImagePtr> cells;
std::vector<const avifImage*> cell_ptrs;
std::vector<ImagePtr> gain_map_cells;
std::vector<const avifImage*> gain_map_ptrs;

avifGainMapMetadata gain_map_metadata =
GetTestGainMapMetadata(/*base_rendition_is_hdr=*/true);
for (uint32_t i = 0; i < grid_cols * grid_rows; ++i) {
ImagePtr image =
testutil::CreateImage(cell_width, cell_height, /*depth=*/10,
AVIF_PIXEL_FORMAT_YUV444, AVIF_PLANES_ALL);
ASSERT_NE(image, nullptr);
image->gainMap = avifGainMapCreate();
image->gainMap->metadata = gain_map_metadata;
image->transferCharacteristics = AVIF_TRANSFER_CHARACTERISTICS_PQ;
image->gainMap->altDepth = 8;
image->gainMap->altPlaneCount = 3;
image->gainMap->altColorPrimaries = AVIF_COLOR_PRIMARIES_SRGB;
image->gainMap->altTransferCharacteristics =
AVIF_TRANSFER_CHARACTERISTICS_SRGB;
image->gainMap->altMatrixCoefficients = AVIF_MATRIX_COEFFICIENTS_BT601;
testutil::FillImageGradient(image.get());

cell_ptrs.push_back(image.get());
cells.push_back(std::move(image));
}
for (uint32_t i = 0; i < gain_map_grid_cols * gain_map_grid_rows; ++i) {
ImagePtr gain_map = testutil::CreateImage(
gain_map_cell_width, gain_map_cell_height, /*depth=*/8,
AVIF_PIXEL_FORMAT_YUV420, AVIF_PLANES_YUV);
ASSERT_NE(gain_map, nullptr);
gain_map->gainMap = avifGainMapCreate();
gain_map->gainMap->metadata = gain_map_metadata;
testutil::FillImageGradient(gain_map.get());
gain_map_ptrs.push_back(gain_map.get());
gain_map_cells.push_back(std::move(gain_map));
}

EncoderPtr encoder(avifEncoderCreate());
ASSERT_NE(encoder, nullptr);
encoder->speed = 10;
avifEncoderInternalOptions internalOptions;
internalOptions.tmapVersion = tmap_version;
internalOptions.tmapMinimumVersion = minimum_version;
internalOptions.tmapWriterVersion = writer_version;
internalOptions.tmapAddExtraBytes = add_extra_bytes;
avifEncoderSetInternalOptions(encoder.get(), &internalOptions);
testutil::AvifRwData encoded;
avifResult result = avifEncoderAddImageGridInternal(
encoder.get(), grid_cols, grid_rows, cell_ptrs.data(), gain_map_grid_cols,
gain_map_grid_rows, gain_map_ptrs.data(), AVIF_ADD_IMAGE_FLAG_SINGLE);
ASSERT_EQ(result, AVIF_RESULT_OK)
<< avifResultToString(result) << " " << encoder->diag.error;
result = avifEncoderFinish(encoder.get(), &encoded);
ASSERT_EQ(result, AVIF_RESULT_OK)
<< avifResultToString(result) << " " << encoder->diag.error;

std::ofstream(path, std::ios::binary)
.write(reinterpret_cast<char*>(encoded.data), encoded.size);
}

TEST(GainMapTest, InvalidGrid) {
std::vector<ImagePtr> cells;
std::vector<const avifImage*> cell_ptrs;
Expand Down Expand Up @@ -1162,6 +1230,66 @@ TEST(GainMapTest, CreateTestImages) {
encoded_small_gainmap.size);
}
}

if (kUpdateTestImages) {
MakeTestImage(
std::string(data_path) + "color_grid_gainmap_different_grid.avif",
/*grid_cols=*/4,
/*grid_rows=*/3,
/*cell_width=*/128, /*cell_heigh=*/200,
/*gain_map_grid_cols=*/2, /*gain_map_grid_rows=*/2,
/*gain_map_cell_width=*/64, /*gain_map_cell_height=*/80);
MakeTestImage(
std::string(data_path) + "color_grid_alpha_grid_gainmap_nogrid.avif",
/*grid_cols=*/4,
/*grid_rows=*/3,
/*cell_width=*/128, /*cell_heigh=*/200,
/*gain_map_grid_cols=*/1, /*gain_map_grid_rows=*/1,
/*gain_map_cell_width=*/64, /*gain_map_cell_height=*/80);
MakeTestImage(
std::string(data_path) + "color_nogrid_alpha_nogrid_gainmap_grid.avif",
/*grid_cols=*/1,
/*grid_rows=*/1,
/*cell_width=*/128, /*cell_heigh=*/200,
/*gain_map_grid_cols=*/2, /*gain_map_grid_rows=*/2,
/*gain_map_cell_width=*/64, /*gain_map_cell_height=*/80);
MakeTestImage(std::string(data_path) + "unsupported_gainmap_version.avif",
/*grid_cols=*/1,
/*grid_rows=*/1,
/*cell_width=*/100, /*cell_heigh=*/100,
/*gain_map_grid_cols=*/1, /*gain_map_grid_rows=*/1,
/*gain_map_cell_width=*/50, /*gain_map_cell_height=*/50,
/*tmap_version=*/99, /*minimum_version=*/0,
/*writer_version=*/0);
MakeTestImage(
std::string(data_path) + "unsupported_gainmap_minimum_version.avif",
/*grid_cols=*/1,
/*grid_rows=*/1,
/*cell_width=*/100, /*cell_heigh=*/100,
/*gain_map_grid_cols=*/1, /*gain_map_grid_rows=*/1,
/*gain_map_cell_width=*/50, /*gain_map_cell_height=*/50,
/*tmap_version=*/0, /*minimum_version=*/99, /*writer_version=*/99);
MakeTestImage(
std::string(data_path) +
"unsupported_gainmap_writer_version_with_extra_bytes.avif",
/*grid_cols=*/1,
/*grid_rows=*/1,
/*cell_width=*/100, /*cell_heigh=*/100,
/*gain_map_grid_cols=*/1, /*gain_map_grid_rows=*/1,
/*gain_map_cell_width=*/50, /*gain_map_cell_height=*/50,
/*tmap_version=*/0, /*minimum_version=*/0, /*writer_version=*/99,
/*add_extra_bytes=*/1);
MakeTestImage(std::string(data_path) +
"supported_gainmap_writer_version_with_extra_bytes.avif",
/*grid_cols=*/1,
/*grid_rows=*/1,
/*cell_width=*/100, /*cell_heigh=*/100,
/*gain_map_grid_cols=*/1, /*gain_map_grid_rows=*/1,
/*gain_map_cell_width=*/50, /*gain_map_cell_height=*/50,
/*tmap_version=*/0, /*minimum_version=*/0,
/*writer_version=*/0,
/*add_extra_bytes=*/1);
}
}

class ToneMapTest
Expand Down