Updating pdf2htmlEX/pdf2htmlEX

[stephengaito]

@ViliusSutkus89

I am about to issue a pull request from my stephengaito/pdf2htmlEX development repository back into the main pdf2htmlEX/pdf2htmlEX repository.

My changes are likely to break your build (if and when you move to our more recent version).

Highlight of changes:

1. pdf2htmlEX now uses poppler-0.89.0 (released last month)
2. pdf2htmlEX now uses fontforge 20200314
3. pdf2htmlEX used to copy "interface" code from poppler and fontforge into pdf2htmlEX's own srouce tree. These sources are no longer copied, instead, the pdf2htmlEX source now expects the poppler and fontforge sources inside the main pdf2htmlEX repository tree (see below).
4. both poppler and fontforge are now automatically downloaded from their releases pages, and statically compiled, and then statically linked directly into the pdf2htmlEX binary. (This ensures that pdf2htmlEX's use of poppler and fontforge will not conflict with any poppler and fontforge libraries installed in the user's OS).
5. to support these changes, the pdf2htmlEX source has been moved one directory deeper in the repository tree, to allow poppler and fontforge to be downloaded and built in "sister" directories (keeping things as "simple" and "explicit" as possible).
6. there is now a "buildScripts" directory in the root of the repository, which contains all of the bash scripts required to download, build, and package poppler, fontforge and ultimately pdf2htmlEX. (I have kept to using bash scripts so as to make as few assumptions as possible about the underlying OS's tools -- however, at the moment, I have assumed the use of "apt" to install dependencies -- though the use of other package installation tools could be added).
7. pdf2htmlEX is now distributed using AppImages, Docker Images, and Debian Packages.

I have just had a look at how you have structured your use of pdf2htmlEX. Your use of Gradle to automate the build is impressive... very nice, explicit and clean.

I have considered creating an official shared library for pdf2htmlEX... but alas I have not yet found the time to do that. (I can see your wrapper, when I get time to create an official shared library, I will attempt to model it on your wrapper's needs).

I notice a number of patch files for both pdf2htmlEX and (a now rather old version of) fontforge. Once I have completed my current pull request, I will look at what parts of these patches are still relevant and then apply them.

This issue is meant to open a conversation on the impacts of my changes to pdf2htmlEX

At the moment, and until you move from pdf2htmlEX/pdf2htmlEX/archive/v0.18.7-poppler-0.81.0, I do not see any major problems.

So I am likely to proceed over the next couple of days with my pull request from my development repo into the main pdf2htmlEX repo.

I hope this is OK.

Regards,
Stephen Gaito

[stephengaito]

Question do you make independent use of either the poppler or fontforge shared libraries that you build?

Alternatively, if poppler and fontforge are statically linked inside the pdf2htmlEX shared library, do you have a need to build poppler and fontforge separately?

[ViliusSutkus89]

This is extra good to hear :)

The upgraded dependencies are more than fine with me. Music to my ears. As long as we upgrade forwards, not backwards. That list of my Fontforge patches will get cut at least in half, I hope. Don't bother looking too much into those patches. Half of them are there to get the software compiling on Android, the other half is there to get them not crashing on every run ... because Android.

When we use Poppler and Fontforge, do we only use the actual library and some headers? We don't need any fontforge.cpp files to compile pdf2htmlEX, do we? After your merge, I'll make some patches, to consume Poppler and Fontforge through pkg-config. They can still be built locally. Consuming through pkg-config allows a compile time switch between system provided libraries and our bundled ones. This helps me with crosscompile a whole lot. All the CFlags, dependency libs and et cetera are included as Poppler and Fontforge intended.

I don't currently use any of those libraries for any other purpose, but somewhy I get the feeling that you want to patch them somehow and we would be incompatible with upstream Poppler and Fontforge. This might be somewhat of a disadvantage for me in the long run. Currently I build all the dependencies myself, but I have a dream that one day we will have a decent native library manager for Android I will be able to just android-apt-get all the libs that I need and just build the pdf2htmlEX-Android without none of that dependency-builder stuff which takes an hour to run. In that scenario, upstream Poppler and Fontforge compatibility would be extra useful. But until then ... I have dependency-builder and some other ugly stuff, which is implementations like tmpfile-Android.

I think it would be useful to extract the CMake dependency builder as a separate tool, but I will have to do some work before it's actually useful. In the current state it's somewhat portable, but there may be places where environment and compiler is implied as Android. Main purpose for it was to take whatever compiler environment you have and pass it to subprojects so that they would be cross compiled too, if the main project is cross compiled.

My version for library is a draft. It works, but still a draft. Progress reporting would be quite useful, but I haven't implemented that yet. Low end Android phones aren't that fast, a progress bar instead of a static "Converting ..." would be nicer.

Thanks for the Gradle compliment :) Although I've used it because I had to. Gradle is the default build system for Android, which means I have less stuff to worry about during tooling upgrades.

If your patches work for desktop pdf2htmlEX, feel free to merge them, I'll update Android's usage.

Regards,
Vilius

[stephengaito]

Vilius,

I have now merged the Update to most recent Poppler/FontForge and improved build scripts #61 pull request from the most recent changes in my development fork.

Other than the updates to poppler and fontforge in the source code, you will now find fairly explicit build scripts for each step of the rather complex build sequence.

The ultimate goal of this new build sequence is to ensure poppler and fontforge are statically linked into the pdf2htmlEX executable.

Alas, I have taken just over a week to provide better testing using a local CI tool. (Travis is not quite flexible enough for pdf2htmlEX's needs).

---

I just noticed your pull request on pdf2htmlEX. I suspect it is now redundant but I will compare it with my new cmake scripts and make any changes suggested.

Note that fontforge 20200314 now uses cmake as well.

---

If there are any problems with these changes on pdf2htmlEX, please let me know... I will try to respond in a timely fashion (alas I do need to return to other work).

Regards,
Stephen Gaito