TIKA-4779: Add audio:track-count and audio:disc-count, normalize n/total values (#2947)

* [TIKA-4779] Add audio:track-count and audio:disc-count, normalize n/total values

The total number of tracks and discs is present in all common audio
containers but was dropped: the MP4 parser read the second value of the
trkn/disk atoms and discarded it, ID3 TRCK/TPOS "n/total" values were
passed through unsplit, and the Vorbis TRACKTOTAL/DISCTOTAL comments were
not mapped. XMPDM has no property for the totals and cannot be extended,
so this adds a Tika-owned org.apache.tika.metadata.Audio interface
(audio:track-count, audio:disc-count), following the pattern of
Geographic and the Google properties from TIKA-4775.

xmpDM:trackNumber and xmpDM:discNumber are declared as Integer
properties but previously received the raw tag value, including the
combined "n/total" form and non-numeric forms like vinyl "A1". They now
only receive clean positive integers, parsed by a small NumberAndTotal
value class (same shape as the recently added ISO6709 parser). Nothing
is lost: the value exactly as tagged stays available under the new
audio:raw-track-number and audio:raw-disc-number properties.

The Ogg path also gains the previously unmapped DISCNUMBER comment, and
explicit TRACKTOTAL/TOTALTRACKS/DISCTOTAL/TOTALDISCS comments win over
the combined form.

The existing testMP4.m4a fixture already carries the totals (track 1 of
42, disc 6 of 12); the ID3v2.4 fixture gains a TRCK "3/12" frame.

* [TIKA-4779] Consume the declared length of the disk atom

The disk branch read a fixed 6 bytes with no length guard, unlike the
adjacent trkn branch. Some encoders pad the disk atom to 8 bytes like
trkn, in which case totalRead diverged from the bytes actually consumed
and every following ilst record was read misaligned. Read the 6 known
bytes when at least that much is declared, then skip the remainder, so
the walk always consumes exactly the declared length.

The udta fixture gains an iTunes-style meta/mdir/ilst with a padded
8-byte disk atom followed by a title entry; without the guard the
misalignment derails the remaining parse.

Author: dschmidt

tika-core/src/main/java/org/apache/tika/metadata/Audio.java

@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+/**
+ * Audio metadata properties that have no XMPDM equivalent. XMPDM defines
+ * {@link XMPDM#TRACK_NUMBER} and {@link XMPDM#DISC_NUMBER} but no properties
+ * for the totals, although the common audio containers all carry them.
+ * See TIKA-4779.
+ *
+ * @since Apache Tika 4.0.0
+ */
+public interface Audio {
+
+    /**
+     * Total number of tracks on the album / in the set
+     * (MP4 'trkn' second value, ID3 TRCK "n/total", Vorbis TRACKTOTAL).
+     */
+    Property TRACK_COUNT = Property.internalInteger("audio:track-count");
+
+    /**
+     * Total number of discs in the set
+     * (MP4 'disk' second value, ID3 TPOS "n/total", Vorbis DISCTOTAL).
+     */
+    Property DISC_COUNT = Property.internalInteger("audio:disc-count");
+
+    /**
+     * The track value exactly as tagged (e.g. "3/12" or a non-numeric form
+     * like vinyl "A1"). {@link XMPDM#TRACK_NUMBER} only receives clean
+     * integers, so nothing is lost.
+     */
+    Property RAW_TRACK_NUMBER = Property.internalText("audio:raw-track-number");
+
+    /**
+     * The disc value exactly as tagged, see {@link #RAW_TRACK_NUMBER}.
+     */
+    Property RAW_DISC_NUMBER = Property.internalText("audio:raw-disc-number");
+}

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/audio/NumberAndTotal.java

@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.audio;
+
+/**
+ * The combined "n/total" form used by ID3 TRCK/TPOS frames and Vorbis
+ * track/disc comments, e.g. {@code "3/12"}. See TIKA-4779.
+ */
+public final class NumberAndTotal {
+
+    //null when the part is absent or not a positive integer; non-numeric
+    //forms (vinyl "A1") are preserved by the raw properties instead
+    public final Integer number;
+    public final Integer total;
+
+    NumberAndTotal(Integer number, Integer total) {
+        this.number = number;
+        this.total = total;
+    }
+
+    /**
+     * @param s a track or disc value, plain ("3") or combined ("3/12"), or null
+     * @return the parsed value, or null if neither part is a positive integer
+     */
+    public static NumberAndTotal parse(String s) {
+        if (s == null) {
+            return null;
+        }
+        int slash = s.indexOf('/');
+        Integer number;
+        Integer total;
+        if (slash < 0) {
+            number = positiveInteger(s);
+            total = null;
+        } else {
+            number = positiveInteger(s.substring(0, slash));
+            total = positiveInteger(s.substring(slash + 1));
+        }
+        if (number == null && total == null) {
+            return null;
+        }
+        return new NumberAndTotal(number, total);
+    }
+
+    private static Integer positiveInteger(String s) {
+        try {
+            int parsed = Integer.parseInt(s.trim());
+            return parsed > 0 ? parsed : null;
+        } catch (NumberFormatException e) {
+            return null;
+        }
+    }
+}

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/mp3/Mp3Parser.java

@@ -30,12 +30,14 @@
 import org.apache.tika.exception.TikaException;
 import org.apache.tika.io.TailStream;
 import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Audio;
 import org.apache.tika.metadata.Metadata;
 import org.apache.tika.metadata.TikaCoreProperties;
 import org.apache.tika.metadata.XMPDM;
 import org.apache.tika.mime.MediaType;
 import org.apache.tika.parser.ParseContext;
 import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.audio.NumberAndTotal;
 import org.apache.tika.parser.mp3.ID3Tags.ID3Comment;
 import org.apache.tika.sax.XHTMLContentHandler;
 
@@ -216,11 +218,29 @@ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata
             sb.append(tag.getAlbum());
             if (tag.getTrackNumber() != null) {
                 sb.append(", track ").append(tag.getTrackNumber());
-                metadata.set(XMPDM.TRACK_NUMBER, tag.getTrackNumber());
+                metadata.set(Audio.RAW_TRACK_NUMBER, tag.getTrackNumber());
+                NumberAndTotal trackNumberAndTotal = NumberAndTotal.parse(tag.getTrackNumber());
+                if (trackNumberAndTotal != null) {
+                    if (trackNumberAndTotal.number != null) {
+                        metadata.set(XMPDM.TRACK_NUMBER, trackNumberAndTotal.number);
+                    }
+                    if (trackNumberAndTotal.total != null) {
+                        metadata.set(Audio.TRACK_COUNT, trackNumberAndTotal.total);
+                    }
+                }
             }
             if (tag.getDisc() != null) {
                 sb.append(", disc ").append(tag.getDisc());
-                metadata.set(XMPDM.DISC_NUMBER, tag.getDisc());
+                metadata.set(Audio.RAW_DISC_NUMBER, tag.getDisc());
+                NumberAndTotal discNumberAndTotal = NumberAndTotal.parse(tag.getDisc());
+                if (discNumberAndTotal != null) {
+                    if (discNumberAndTotal.number != null) {
+                        metadata.set(XMPDM.DISC_NUMBER, discNumberAndTotal.number);
+                    }
+                    if (discNumberAndTotal.total != null) {
+                        metadata.set(Audio.DISC_COUNT, discNumberAndTotal.total);
+                    }
+                }
             }
 
             xhtml.element("h1", tag.getTitle());
@@ -268,4 +288,5 @@ private float durationSeconds() {
             return duration / 1000;
         }
     }
+
 }

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/mp4/boxes/TikaUserDataBox.java

@@ -27,6 +27,7 @@
 import org.xml.sax.SAXException;
 
 import org.apache.tika.exception.RuntimeSAXException;
+import org.apache.tika.metadata.Audio;
 import org.apache.tika.metadata.Metadata;
 import org.apache.tika.metadata.TikaCoreProperties;
 import org.apache.tika.metadata.XMP;
@@ -157,14 +158,29 @@ private void processIList(SequentialReader reader, long totalLen)
                         long numA = reader.getUInt32();
                         long numB = reader.getUInt32();
                         metadata.set(XMPDM.TRACK_NUMBER, (int)numA);
+                        //2 bytes track total, 2 bytes reserved
+                        int trackCount = (int) (numB >>> 16);
+                        if (trackCount > 0) {
+                            metadata.set(Audio.TRACK_COUNT, trackCount);
+                        }
                     } else {
                         //log
                         reader.skip(toRead);
                     }
                 } else if ("disk".equals(fieldName)) {
-                    int a = reader.getInt32();
-                    short b = reader.getInt16();
-                    metadata.set(XMPDM.DISC_NUMBER, a);
+                    //2 bytes reserved, 2 bytes disc, 2 bytes total; some encoders
+                    //pad to 8 bytes like trkn, so consume exactly toRead either way
+                    if (toRead >= 6) {
+                        int a = reader.getInt32();
+                        short b = reader.getInt16();
+                        metadata.set(XMPDM.DISC_NUMBER, a);
+                        if (b > 0) {
+                            metadata.set(Audio.DISC_COUNT, b);
+                        }
+                        reader.skip(toRead - 6);
+                    } else {
+                        reader.skip(toRead);
+                    }
                 } else {
                     String val = reader.getString(toRead, StandardCharsets.UTF_8);
                     try {

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java

@@ -32,11 +32,13 @@
 import org.xml.sax.SAXException;
 
 import org.apache.tika.exception.TikaException;
+import org.apache.tika.metadata.Audio;
 import org.apache.tika.metadata.Metadata;
 import org.apache.tika.metadata.TikaCoreProperties;
 import org.apache.tika.metadata.XMP;
 import org.apache.tika.metadata.XMPDM;
 import org.apache.tika.parser.AbstractParser;
+import org.apache.tika.parser.audio.NumberAndTotal;
 import org.apache.tika.sax.XHTMLContentHandler;
 
 /**
@@ -46,6 +48,27 @@
 public abstract class OggAudioParser extends AbstractParser {
     private static final long serialVersionUID = 5168743829615945633L;
 
+
+    /**
+     * Returns the first positive integer found under the given comment keys,
+     * or null if there is none.
+     */
+    private static Integer firstPositiveInteger(VorbisStyleComments comments, String... keys) {
+        for (String key : keys) {
+            for (String value : comments.getComments(key)) {
+                try {
+                    int parsed = Integer.parseInt(value.trim());
+                    if (parsed > 0) {
+                        return parsed;
+                    }
+                } catch (NumberFormatException e) {
+                    //skip unparseable values
+                }
+            }
+        }
+        return null;
+    }
+
     protected static void extractChannelInfo(Metadata metadata, OggAudioInfoHeader info) {
         extractChannelInfo(metadata, info.getNumChannels());
     }
@@ -107,10 +130,40 @@ protected static void extractComments(Metadata metadata, XHTMLContentHandler xht
         // Album and Track number
         if (comments.getTrackNumber() != null) {
             xhtml.element("p", comments.getAlbum() + ", track " + comments.getTrackNumber());
-            metadata.set(XMPDM.TRACK_NUMBER, comments.getTrackNumber());
+            metadata.set(Audio.RAW_TRACK_NUMBER, comments.getTrackNumber());
+            NumberAndTotal trackNumberAndTotal = NumberAndTotal.parse(comments.getTrackNumber());
+            if (trackNumberAndTotal != null) {
+                if (trackNumberAndTotal.number != null) {
+                    metadata.set(XMPDM.TRACK_NUMBER, trackNumberAndTotal.number);
+                }
+                if (trackNumberAndTotal.total != null) {
+                    metadata.set(Audio.TRACK_COUNT, trackNumberAndTotal.total);
+                }
+            }
         } else {
             xhtml.element("p", comments.getAlbum());
         }
+        for (String discValue : comments.getComments("discnumber")) {
+            metadata.set(Audio.RAW_DISC_NUMBER, discValue);
+            NumberAndTotal discNumberAndTotal = NumberAndTotal.parse(discValue);
+            if (discNumberAndTotal != null) {
+                if (discNumberAndTotal.number != null) {
+                    metadata.set(XMPDM.DISC_NUMBER, discNumberAndTotal.number);
+                }
+                if (discNumberAndTotal.total != null) {
+                    metadata.set(Audio.DISC_COUNT, discNumberAndTotal.total);
+                }
+            }
+        }
+        //explicit totals win over the combined "n/total" form
+        Integer trackTotal = firstPositiveInteger(comments, "tracktotal", "totaltracks");
+        if (trackTotal != null) {
+            metadata.set(Audio.TRACK_COUNT, trackTotal);
+        }
+        Integer discTotal = firstPositiveInteger(comments, "disctotal", "totaldiscs");
+        if (discTotal != null) {
+            metadata.set(Audio.DISC_COUNT, discTotal);
+        }
 
         // A few other bits
         xhtml.element("p", comments.getDate());
@@ -161,4 +214,5 @@ private static String formatDuration(double durationSeconds) {
             return String.format(Locale.ROOT, "%d:%02d", minutes, seconds);
         }
     }
+
 }

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/audio/NumberAndTotalTest.java

@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.audio;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+
+import org.junit.jupiter.api.Test;
+
+public class NumberAndTotalTest {
+
+    @Test
+    public void testPlainNumber() {
+        NumberAndTotal value = NumberAndTotal.parse("3");
+        assertEquals(3, value.number);
+        assertNull(value.total);
+    }
+
+    @Test
+    public void testCombinedForm() {
+        NumberAndTotal value = NumberAndTotal.parse("3/12");
+        assertEquals(3, value.number);
+        assertEquals(12, value.total);
+    }
+
+    @Test
+    public void testWhitespace() {
+        NumberAndTotal value = NumberAndTotal.parse(" 1 / 2 ");
+        assertEquals(1, value.number);
+        assertEquals(2, value.total);
+    }
+
+    @Test
+    public void testDegenerateForms() {
+        assertNull(NumberAndTotal.parse(null));
+        assertNull(NumberAndTotal.parse("   "));
+        //non-numeric forms parse to nothing; the raw properties keep them
+        assertNull(NumberAndTotal.parse("A1"));
+        assertNull(NumberAndTotal.parse("3a/of twelve"));
+
+        NumberAndTotal totalOnly = NumberAndTotal.parse("/12");
+        assertNull(totalOnly.number);
+        assertEquals(12, totalOnly.total);
+
+        NumberAndTotal nonNumericTotal = NumberAndTotal.parse("3/of twelve");
+        assertEquals(3, nonNumericTotal.number);
+        assertNull(nonNumericTotal.total);
+
+        NumberAndTotal zeroTotal = NumberAndTotal.parse("3/0");
+        assertEquals(3, zeroTotal.number);
+        assertNull(zeroTotal.total);
+    }
+}