support mets:div/mets:fptr reference besides mets:structLink, or full ENMAP

[stefanCCS]

It looks like, that the METS parser does not allow structures like this in METS:

                  <div ID="DIVL5" TYPE="TITLE_OF_WORK">
                     <fptr>
                        <area BETYPE="IDREF" FILEID="ALTO00001" BEGIN="P1_TB00002"/>
                     </fptr>

If I call mm2tei with this kind of METS I get an exception:

Traceback (most recent call last):
  File "/home/calamariadmin/tei_venv_3.7/bin/mm2tei", line 8, in <module>
    sys.exit(cli())
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/mets_mods2tei/scripts/mets_mods2tei.py", line 56, in cli
    tei.fill_from_mets(mets, ocr)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/mets_mods2tei/api/tei.py", line 175, in fill_from_mets
    self.add_div_structure(div)
  File "/home/calamariadmin/tei_venv_3.7/lib/python3.7/site-packages/mets_mods2tei/api/tei.py", line 831, in add_div_structure
    div = div.get_div()[0]
IndexError: list index out of range

As a starting point an "ignore" of <fptr><area> in <div> area would be good.
In general it would be even better, if the OCR text from ALTO is taken from the link referenced there.

[bertsky]

div = div.get_div()[0]
IndexError: list index out of range

That's actually #64 (which would entail the ignore strategy), but the additional issue here is indeed that mets:fptr/mets:area references instead of mere mets:structLink matches for the mets:div/@ID should be supported.

Maximally, support for the ENMAP profile is desired.

[bertsky]

Further reference: ENMAP examples

[bertsky]

@t-mayer I have been digging a little deeper, and found that ENMAP support would require a lot more besides flexible mets:div parsing (to identify the top content element encompassing all sections/paragraphs/...) in the mets:structMap:

• ENMAP allows different sub-element types than DFG Strukturdatenset, viz. (any recursive number of) content_section and (any recursive number of) content_unit above the other types (article, table etc);
  and there seem to be additional sub-element types below content_unit in use, too (e.g. caption, advertisement, body, body_content, heading, title...)
• the DFG profile requires fileGrps to be flat and named by @USE (in a predefined set of names), while ENMAP requires recursive fileGrps named by @ID in a certain way
• we currently only use the mets:structLink mechanism to map (ranges of) whole pages to divs, but ENMAP uses mets:area (with @COORDS, which would be hard to match with fulltext files, or @BEGIN, which references the ALTO elements directly)
• ENMAP even seems to allow mets:area/@FILEID references to the mets:file entries instead of direct mets:fptr/@FILEID in the physical structMap

There are probably more challenges, but this is already a lot of work. So I'm afraid there is no simple workaround. Sorry, I cannot promise any progress on this matter ATM. But PRs are always welcome of course!

Originally posted in issue 64