Skip to content

Commit 6fce97f

Browse files
authored
Htj2k fixes (#103)
* Adding jpeg2000 codec. Signed-off-by: [email protected] <[email protected]> * WIP, Had problems running this in a docker container, since the timing returned didnt make sense, so we have a clock version and a non-clock version. Also am experimenting with the thread-type. Signed-off-by: [email protected] <[email protected]> * A bit more docker ffmpeg-7.1 cleanup dont need aces now, since it comes with OCIO. Signed-off-by: [email protected] <[email protected]> * Adding the terrific parallel-frames flag, for faster conversion. Signed-off-by: [email protected] <[email protected]> * Added tests for htj2k, currently relying on ppm files. Signed-off-by: [email protected] <[email protected]> * Wrapper script for generating h2k files and then wrapping in mxf (or mov) file. Signed-off-by: [email protected] <[email protected]> * Frame range fix. Signed-off-by: [email protected] <[email protected]> (cherry picked from commit 13fbea652773b0c65027914efca741c0dd7052ad) Signed-off-by: [email protected] <[email protected]> * Lets just use DPX files since they are supported. Signed-off-by: [email protected] <[email protected]> (cherry picked from commit c999d88c4225eb0d9285a47b4d6ca320fb6181a9) Signed-off-by: [email protected] <[email protected]> * Update new model location. Signed-off-by: [email protected] <[email protected]> (cherry picked from commit 02646963e6fd7c2fc2dd2f176cecfaed50defba6) Signed-off-by: [email protected] <[email protected]> * Update env location so it works on both osx and docker/rockylinux Signed-off-by: [email protected] <[email protected]> (cherry picked from commit 2df535f36b50e7f8c6ec0c4aa907052b31fbdd8a) Signed-off-by: [email protected] <[email protected]> * This is measuring fps rates, purely using ffmpeg. (cherry picked from commit 685b9cc) Signed-off-by: [email protected] <[email protected]> * Adding a bunch of htj2k tests. Signed-off-by: [email protected] <[email protected]> * Adding a dwab test. Signed-off-by: [email protected] <[email protected]> * Wrappers to help map htj2k and dwab into the encoding framework. Signed-off-by: [email protected] <[email protected]> * Adding some of the 4k media as reference. Signed-off-by: [email protected] <[email protected]> * Adding support for cambi. Signed-off-by: [email protected] <[email protected]> * Put each bit of media in a separate folder. Signed-off-by: [email protected] <[email protected]> * UPgrade to use the vmaf3 code, and do some better comparisons. Signed-off-by: [email protected] <[email protected]> * New templates for vmaf3. Signed-off-by: [email protected] <[email protected]> * Handle the cambi and other parameters into the template better. Signed-off-by: [email protected] <[email protected]> * Special case to handle QFactor= parameters, although this would work with any "=" parameter. Signed-off-by: [email protected] <[email protected]> * comments, and labeling. Signed-off-by: [email protected] <[email protected]> * Dont need this varient. Signed-off-by: [email protected] <[email protected]> * Adding the vmaf3 test. Signed-off-by: [email protected] <[email protected]> * Too extreme, makes graph hard to read. Signed-off-by: [email protected] <[email protected]> * Note for getting exr versions of some of the files. Signed-off-by: [email protected] <[email protected]> * Enable ffplay, so we can do some of the playback testing on the VM. Signed-off-by: [email protected] <[email protected]> * Improve units here and there. Signed-off-by: [email protected] <[email protected]> --------- Signed-off-by: [email protected] <[email protected]>
1 parent e9af967 commit 6fce97f

34 files changed

+1971
-82
lines changed

docker/ffmpeg-7.1/Dockerfile

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -653,18 +653,6 @@ RUN \
653653
make qt-faststart && cp qt-faststart ${PREFIX}/bin/
654654

655655

656-
RUN \
657-
echo "Downloading ACES." && \
658-
DIR=/opt/aces && \
659-
mkdir -p ${DIR} && \
660-
cd ${DIR} && \
661-
curl -sLO https://github.com/colour-science/OpenColorIO-Configs/archive/refs/tags/v1.2.tar.gz && \
662-
tar zxvf v1.2.tar.gz OpenColorIO-Configs-1.2/aces_1.2 && \
663-
rm v1.2.tar.gz && \
664-
mkdir -p /usr/share/ocio/aces_1.2 && \
665-
cp -vr ${DIR}/OpenColorIO-Configs-1.2/aces_1.2 /usr/share/ocio && \
666-
rm -rf ${DIR}
667-
668656
RUN \
669657
ldd ${PREFIX}/bin/ffmpeg | grep opt/ffmpeg | cut -d ' ' -f 3 | xargs -i cp {} /usr/local/lib64/ && \
670658
for lib in /usr/local/lib64/*.so.*; do ln -s "${lib##*/}" "${lib%%.so.*}".so; done && \
@@ -690,7 +678,7 @@ RUN \
690678
# Need to add /opt/rh/httpd24/root/usr/lib64/ to get git clone to work.
691679

692680
ENV LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/opt/rh/httpd24/root/usr/lib64/ \
693-
OCIO=/usr/share/ocio/aces_1.2/config.ocio \
681+
OCIO=ocio://studio-config-v1.0.0_aces-v1.3_ocio-v2.1 \
694682
VMAF_MODEL=${VMAF_LIB_DIR}
695683

696684
RUN pip install OpenTimelineIO

docker/rocky-ffmpeg-7.0/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -645,7 +645,7 @@ RUN \
645645
--disable-doc \
646646
# --enable-cuda-nvcc \
647647
--enable-libnpp \
648-
--disable-ffplay \
648+
# --disable-ffplay \
649649
--enable-fontconfig \
650650
--enable-cuda-nvcc \
651651
--enable-gpl \

enctests/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,7 @@ rm v1.2.tar.gz
224224
export OCIO=$PWD/OpenColorIO-Configs-1.2/aces_1.2/config.ocio
225225
226226
# Set VMAF_Models NOTE, should probably change this.
227-
export VMAF_MODEL=/opt/homebrew/Cellar/libvmaf/2.3.1/share/libvmaf/model/
227+
export VMAF_MODEL_DIR=/opt/homebrew/Cellar/libvmaf/3.0.0/share/libvmaf/model/
228228

229229
# Run tests (for now)
230230
.venv/bin/python main.py
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
#!/usr/bin/env python3
2+
3+
4+
import os, sys
5+
import argparse
6+
from concurrent.futures import ThreadPoolExecutor
7+
from pathlib import Path
8+
from functools import partial
9+
import subprocess
10+
11+
# This allows you to compress the file with the compressor of your choice, and then wrap the result in a quicktime file
12+
# So we can do the comparison with vmaf.
13+
# We assume that what ever the intermediate compressor is doing that the resulting intermediate file is a png file.
14+
15+
def process_file(args, tmpoutputdir, ext, file_path):
16+
input_path = Path(file_path)
17+
output_path = Path(os.path.join(tmpoutputdir, os.path.basename(str(input_path)))).with_suffix(ext)
18+
if os.path.exists(str(output_path)):
19+
print("Removing file:", output_path)
20+
os.remove(str(output_path))
21+
try:
22+
# Your processing here
23+
# Example: copy file with new extension
24+
cmd = args[:]
25+
cmd.extend(['-i', file_path])
26+
27+
cmd.extend(["-o", str(output_path)])
28+
print("Running:", " ".join(cmd))
29+
subprocess.run(cmd)
30+
return True, input_path, output_path
31+
except Exception as e:
32+
print("ERROR:", e)
33+
return False, input_path, str(e)
34+
35+
def main():
36+
print("Root:", os.path.dirname(os.path.abspath(__file__)))
37+
os.environ['PATH'] = os.path.dirname(os.path.abspath(__file__)) + ":" + os.environ['PATH']
38+
39+
parser = argparse.ArgumentParser()
40+
41+
parser.add_argument(
42+
'-i', '--input',
43+
dest="input",
44+
action='store',
45+
default=[""],
46+
help='Provide an input file.'
47+
)
48+
parser.add_argument(
49+
'-start_number',
50+
dest="start_frame",
51+
default="0",
52+
help='Start frame for input file.'
53+
)
54+
parser.add_argument(
55+
'-r',
56+
dest="framerate",
57+
default="25",
58+
help='Frame rate'
59+
)
60+
61+
parser.add_argument(
62+
'--extension',
63+
dest="extension",
64+
default="png",
65+
help='Intermediate extension'
66+
)
67+
68+
args, otherargs = parser.parse_known_args()
69+
ext = args.extension
70+
print("EXT:", ext)
71+
outputfile = otherargs[-1]
72+
otherargs = otherargs[:-1] #Strip of the last argument.
73+
74+
# Configure number of workers
75+
max_workers = 4
76+
77+
outputdir = os.path.dirname(outputfile)
78+
tmpoutputdir = os.path.join(outputdir, "tmppngfiles", os.path.basename(outputfile))
79+
if not os.path.exists(tmpoutputdir):
80+
os.makedirs(tmpoutputdir)
81+
82+
# Get list of files
83+
inputfile = args.input
84+
print("Processing input file spec:", inputfile)
85+
files = []
86+
if "%" in inputfile:
87+
startframe = int(args.start_frame)
88+
infile = inputfile % startframe
89+
while os.path.exists(infile):
90+
files.append(infile)
91+
startframe = startframe + 1
92+
infile = inputfile % startframe
93+
print("Checking:", infile)
94+
else:
95+
files.append(inputfile)
96+
# files = list(Path('directory').glob('*.txt'))
97+
98+
# Process files with progress bar
99+
with ThreadPoolExecutor(max_workers=max_workers) as executor:
100+
results = executor.map(partial(process_file, otherargs, str(tmpoutputdir), "." + ext), files)
101+
102+
# Report results
103+
successes = [r for r in results if r[0]]
104+
failures = [r for r in results if not r[0]]
105+
106+
print(f"\nProcessed {len(successes)} files successfully")
107+
if failures:
108+
print(f"Failed to process {len(failures)} files:")
109+
for _, input_path, error in failures:
110+
print(f" {input_path}: {error}")
111+
else:
112+
intputfilej2k = str(Path(os.path.join(tmpoutputdir, os.path.basename(inputfile))).with_suffix("." + ext))
113+
cmd = ['ffmpeg', '-f', 'image2', '-r', args.framerate,
114+
'-start_number', args.start_frame, '-i', intputfilej2k, '-vcodec', 'copy',
115+
'-color_range', '1', '-colorspace', '1', '-color_primaries', '1',
116+
'-color_trc', '2', '-y', outputfile]
117+
print(" ".join(cmd))
118+
subprocess.run(cmd)
119+
# Now we remove the intermediate files.
120+
121+
122+
123+
124+
if __name__ == '__main__':
125+
main()
126+

enctests/bin/htj2k_encodewrap.csh

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
#!/usr/bin/env python3
2+
3+
4+
import os
5+
import argparse
6+
from concurrent.futures import ThreadPoolExecutor
7+
from pathlib import Path
8+
from functools import partial
9+
import subprocess
10+
11+
# This allows you to do the encoding of a j2h file in a different app, and then wrap the result into a movie file with ffmpeg.
12+
13+
def process_file(args, tmpoutputdir, file_path):
14+
input_path = Path(file_path)
15+
output_path = Path(os.path.join(tmpoutputdir, os.path.basename(str(input_path)))).with_suffix('.j2c')
16+
if os.path.exists(str(output_path)):
17+
print("Removing file:", output_path)
18+
os.remove(str(output_path))
19+
try:
20+
# Your processing here
21+
# Example: copy file with new extension
22+
cmd = [f.replace("--reversible", "-reversible") for f in args[:]]
23+
cmd.extend(['-i', file_path, "-o", str(output_path)])
24+
print("Running:", " ".join(cmd))
25+
subprocess.run(cmd)
26+
return True, input_path, output_path
27+
except Exception as e:
28+
print("ERROR:", e)
29+
return False, input_path, str(e)
30+
31+
def main():
32+
parser = argparse.ArgumentParser()
33+
34+
parser.add_argument(
35+
'-i', '--input',
36+
dest="input",
37+
action='store',
38+
default=[""],
39+
help='Provide an input file.'
40+
)
41+
parser.add_argument(
42+
'-start_number',
43+
dest="start_frame",
44+
default="0",
45+
help='Start frame for input file.'
46+
)
47+
parser.add_argument(
48+
'-r',
49+
dest="framerate",
50+
default="25",
51+
help='Frame rate'
52+
)
53+
54+
args, otherargs = parser.parse_known_args()
55+
outputfile = otherargs[-1]
56+
otherargs = otherargs[:-1] #Strip of the last argument.
57+
58+
# Need to look for arguments ending in "=" so we know to merge with next arg.
59+
# so ["Qfactor=", "40", "Clevels=", "5"] would become ['Qfactor=40', 'Clevels=5']
60+
newotherargs = []
61+
while otherargs:
62+
print("OTHERARGS:", otherargs)
63+
arg = otherargs.pop(0)
64+
if arg.endswith("="):
65+
nextarg = otherargs.pop(0)
66+
arg = arg[:-1] + "=" + nextarg
67+
newotherargs.append(arg)
68+
otherargs = newotherargs
69+
70+
# Configure number of workers
71+
max_workers = 4
72+
73+
outputdir = os.path.dirname(outputfile)
74+
tmpoutputdir = os.path.join(outputdir, "tmpj2kfiles", os.path.basename(outputfile))
75+
if not os.path.exists(tmpoutputdir):
76+
os.makedirs(tmpoutputdir)
77+
78+
# Get list of files
79+
inputfile = args.input
80+
print("Processing input file spec:", inputfile)
81+
files = []
82+
if "%" in inputfile:
83+
startframe = int(args.start_frame)
84+
infile = inputfile % startframe
85+
while os.path.exists(infile):
86+
files.append(infile)
87+
startframe = startframe + 1
88+
infile = inputfile % startframe
89+
print("Checking:", infile)
90+
else:
91+
files.append(inputfile)
92+
# files = list(Path('directory').glob('*.txt'))
93+
94+
# Process files with progress bar
95+
with ThreadPoolExecutor(max_workers=max_workers) as executor:
96+
results = executor.map(partial(process_file, otherargs, str(tmpoutputdir)), files)
97+
98+
# Report results
99+
successes = [r for r in results if r[0]]
100+
failures = [r for r in results if not r[0]]
101+
102+
print(f"\nProcessed {len(successes)} files successfully")
103+
if failures:
104+
print(f"Failed to process {len(failures)} files:")
105+
for _, input_path, error in failures:
106+
print(f" {input_path}: {error}")
107+
else:
108+
intputfilej2k = str(Path(os.path.join(tmpoutputdir, os.path.basename(inputfile))).with_suffix('.j2c'))
109+
cmd = ['ffmpeg', '-f', 'image2', '-r', args.framerate,
110+
'-start_number', args.start_frame, '-i', intputfilej2k, '-vcodec', 'copy',
111+
'-color_range', '1', '-colorspace', '1', '-color_primaries', '1',
112+
'-color_trc', '2', '-y', outputfile]
113+
print("Running:", " ".join(cmd))
114+
subprocess.run(cmd)
115+
# Now we remove the intermediate files.
116+
117+
118+
119+
120+
if __name__ == '__main__':
121+
main()
122+
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
#!/usr/bin/env python3
2+
3+
4+
import os
5+
import argparse
6+
from concurrent.futures import ThreadPoolExecutor
7+
from pathlib import Path
8+
from functools import partial
9+
import subprocess
10+
11+
12+
13+
def main():
14+
parser = argparse.ArgumentParser()
15+
16+
parser.add_argument(
17+
'-i', '--input',
18+
dest="input",
19+
action='store',
20+
default=[""],
21+
help='Provide an input file.'
22+
)
23+
parser.add_argument(
24+
'-start_number',
25+
dest="start_frame",
26+
default="0",
27+
help='Start frame for input file.'
28+
)
29+
parser.add_argument(
30+
'-r',
31+
dest="framerate",
32+
default="25",
33+
help='Frame rate'
34+
)
35+
36+
args, otherargs = parser.parse_known_args()
37+
file_path = args.input
38+
outputfile = otherargs[-1]
39+
otherargs = otherargs[:-1] #Strip of the last argument.
40+
41+
input_path = Path(file_path)
42+
43+
outputdir = os.path.dirname(outputfile)
44+
tmpoutputdir = os.path.join(outputdir, "tmpj2kfiles", os.path.basename(outputfile))
45+
)
46+
output_path = Path(os.path.join(tmpoutputdir, os.path.basename(str(input_path)))).with_suffix('.j2c')
47+
48+
if not os.path.exists(tmpoutputdir):
49+
os.makedirs(tmpoutputdir)
50+
51+
# Get list of files
52+
inputfile = args.input
53+
54+
oiio_args = []
55+
launch_arg = otherargs.pop(0) # Get rid of the app to run.
56+
arg = otherargs.pop(0)
57+
while otherargs:
58+
if arg.startswith("-jph:"):
59+
oiio_args.append("--attrib")
60+
oiio_args.append(arg.replace("-jph:", "jph:"))
61+
arg = otherargs.pop(0)
62+
oiio_args.append(arg)
63+
else:
64+
print("Misc arg:", arg)
65+
oiio_args.append(arg)
66+
if not otherargs:
67+
break
68+
arg = otherargs.pop(0)
69+
70+
endframe = int(args.start_frame)
71+
file = str(input_path) % endframe
72+
print("FILE:", file, " from :", input_path)
73+
while(os.path.exists(str(input_path) % endframe)):
74+
endframe += 1
75+
endframe -= 1
76+
77+
oiio_args = [launch_arg, '-v', '-i', str(input_path), "--parallel-frames", "--frames", "%s-%d" % (args.start_frame, endframe)]+oiio_args+['-o', str(output_path)]
78+
print(" ".join(oiio_args))
79+
subprocess.run(oiio_args)
80+
81+
82+
intputfilej2k = str(Path(os.path.join(tmpoutputdir, os.path.basename(inputfile))).with_suffix('.j2c'))
83+
cmd = ['ffmpeg', '-f', 'image2', '-r', args.framerate,
84+
'-start_number', args.start_frame, '-i', intputfilej2k, '-vcodec', 'copy',
85+
'-color_range', '1', '-colorspace', '1', '-color_primaries', '1',
86+
'-color_trc', '2', '-y', outputfile]
87+
subprocess.run(cmd)
88+
# Now we remove the intermediate files.
89+
90+
91+
92+
93+
if __name__ == '__main__':
94+
main()
95+

0 commit comments

Comments
 (0)