Skip to content
This repository was archived by the owner on Aug 14, 2025. It is now read-only.

Commit aa873b9

Browse files
authored
Merge pull request #1 from cpeel/ebookmaker-0.10.1
Update ebookmaker to 0.10.1
2 parents a477a46 + a9c712d commit aa873b9

9 files changed

Lines changed: 799 additions & 185 deletions

File tree

Pipfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ verify_ssl = true
66
[dev-packages]
77

88
[packages]
9-
ebookmaker = "==0.9.1"
109
pywin32-ctypes = "*"
11-
pyinstaller = {git = "https://github.com/pyinstaller/pyinstaller.git", ref = "8714423aa"}
10+
pyinstaller = "==4.1"
11+
ebookmaker = "==0.10.1"
1212

1313
[requires]
1414
python_version = "3.8"

Pipfile.lock

Lines changed: 175 additions & 143 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 83 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,13 @@
22

33
This is a hacky little repo used to build Windows binaries for
44
[ebookmaker](https://github.com/gutenbergtools/ebookmaker) using
5-
[pyinstaller](https://github.com/pyinstaller/pyinstaller). It uses an
6-
in-development version of pyinstaller to work around issues in v3.6 (problems
7-
with setuptools==45 and the lack of an installer hook for `jaraco.text`).
5+
[pyinstaller](https://github.com/pyinstaller/pyinstaller).
86

97
These instructions use bash-isms and thus assume you are running the
108
following commands in a git bash shell on Windows (presumably after having
119
installed [Git for Windows](https://git-scm.com/download/win)).
1210

13-
To use:
11+
## Set up build environment
1412

1513
1. Clone this repository
1614

@@ -25,49 +23,89 @@ To use:
2523
will need all of the things this wants to install. All 4GB of it.
2624
5. Start the install.
2725

28-
3. Install `pipenv`
26+
3. If not present already, you may need to add the following to your
27+
PATH to pick up python, pip, and pipenv:
28+
* `%USERPROFILE%\AppData\Local\Programs\Python\Python38`
29+
* `%USERPROFILE%\AppData\Local\Programs\Python\Python38\Scripts`
30+
* `%USERPROFILE%\AppData\Roaming\Python\Python38\Scripts`
31+
32+
4. Install `pipenv`
2933
```
3034
pip install --user pipenv
3135
```
3236

33-
4. Within the repository clone, install the pipenv virtual environment
37+
5. Within the repository clone, install the pipenv virtual environment
3438
```
3539
pipenv install
3640
```
3741

38-
5. Enter into a pipenv shell
39-
```
40-
pipenv shell
41-
```
42+
## Build ebookmaker
4243

43-
6. Build ebookmaker
44-
```
45-
pyinstaller -F \
46-
--hidden-import ebookmaker.parsers.AuxParser \
47-
--hidden-import ebookmaker.parsers.CSSParser \
48-
--hidden-import ebookmaker.parsers.GutenbergTextParser \
49-
--hidden-import ebookmaker.parsers.HTMLParser \
50-
--hidden-import ebookmaker.parsers.ImageParser \
51-
--hidden-import ebookmaker.parsers.RSTParser \
52-
--hidden-import ebookmaker.parsers.WrapperParser \
53-
--hidden-import ebookmaker.writers.EpubWriter \
54-
--hidden-import ebookmaker.writers.HTMLWriter \
55-
--hidden-import ebookmaker.writers.KindleWriter \
56-
--hidden-import ebookmaker.writers.PDFWriter \
57-
--hidden-import ebookmaker.writers.PicsDirWriter \
58-
--hidden-import ebookmaker.writers.RSTWriter \
59-
--hidden-import ebookmaker.writers.TxtWriter \
60-
--hidden-import ebookmaker.packagers.GzipPackager \
61-
--hidden-import ebookmaker.packagers.HTMLPackager \
62-
--hidden-import ebookmaker.packagers.PDFPackager \
63-
--hidden-import ebookmaker.packagers.PushPackager \
64-
--hidden-import ebookmaker.packagers.RSTPackager \
65-
--hidden-import ebookmaker.packagers.TxtPackager \
66-
"$(which ebookmaker)"
67-
```
44+
ebookmaker dynamically loads the set of available parsers, packagers,
45+
and writers from the filesystem using `pkg_resources`. Unfortunately
46+
this does not work with pyinstaller so we need to adjust the installed
47+
ebookmaker code to provide it with an explicit list using the
48+
`ebookmaker-pyinstaller.patch` file.
49+
50+
```bash
51+
# Start a pipenv shell
52+
pipenv shell
53+
54+
# Apply the patch
55+
./apply_patch.sh
56+
```
57+
58+
Create the ebookmaker binary with pyinstaller:
59+
60+
```bash
61+
# Start a pipenv shell if you are not already
62+
pipenv shell
63+
64+
# Run the installer
65+
pyinstaller -F \
66+
--hidden-import ebookmaker.parsers.AuxParser \
67+
--hidden-import ebookmaker.parsers.CSSParser \
68+
--hidden-import ebookmaker.parsers.GutenbergTextParser \
69+
--hidden-import ebookmaker.parsers.HTMLParser \
70+
--hidden-import ebookmaker.parsers.ImageParser \
71+
--hidden-import ebookmaker.parsers.RSTParser \
72+
--hidden-import ebookmaker.parsers.WrapperParser \
73+
--hidden-import ebookmaker.writers.EpubWriter \
74+
--hidden-import ebookmaker.writers.HTMLWriter \
75+
--hidden-import ebookmaker.writers.KindleWriter \
76+
--hidden-import ebookmaker.writers.PDFWriter \
77+
--hidden-import ebookmaker.writers.PicsDirWriter \
78+
--hidden-import ebookmaker.writers.RSTWriter \
79+
--hidden-import ebookmaker.writers.TxtWriter \
80+
--hidden-import ebookmaker.packagers.GzipPackager \
81+
--hidden-import ebookmaker.packagers.HTMLPackager \
82+
--hidden-import ebookmaker.packagers.PDFPackager \
83+
--hidden-import ebookmaker.packagers.PushPackager \
84+
--hidden-import ebookmaker.packagers.RSTPackager \
85+
--hidden-import ebookmaker.packagers.TxtPackager \
86+
"$(which ebookmaker)"
87+
```
6888

6989
This will create the Windows binary `dist/ebookmaker.exe`.
7090

91+
## Validate binary
92+
93+
To validate the `ebookmaker.exe` binary is working correctly, you must
94+
build an epub, not just `ebookmaker.exe --version`:
95+
96+
```bash
97+
./dist/ebookmaker.exe \
98+
--verbose \
99+
--make=epub.images \
100+
--output-dir build \
101+
--title sample_ebook \
102+
sample_ebook/ebmtest.htm
103+
```
104+
105+
This should create `build/sample_ebook-images-epub.epub`. You may get an
106+
error if `tidy` isn't installed on your system -- this is fine and can
107+
be ignored.
108+
71109
## Releasing binaries
72110

73111
To release a new binary:
@@ -97,12 +135,17 @@ To release a new binary:
97135
98136
## Updating the version of ebookmaker
99137
100-
To update the version of ebookmaker this tool builds you will need to
101-
`pipenv install` the newer ebookmaker package version. You may (or may not)
102-
need to also adjust the version of pyinstaller or any of the other magical
103-
package dependencies specified inside the Pipfile.
138+
To update the version of ebookmaker this tool builds, install a new
139+
version of the ebookmaker package with `pipenv install`:
104140
105-
For example:
106141
```
107142
pipenv install "ebookmaker==0.9.2"
108143
```
144+
145+
Then re-apply the patch to enable building with pyinstaller and build
146+
the new binary as instructed above. Depending on the changes within
147+
ebookmaker this may require updating the patch and/or the list of hidden
148+
imports passed into pyinstaller.
149+
150+
Be sure and check in the updated `Pipfile` and `Pipfile.lock` along with
151+
any other changes before tagging and releasing a new binary.

apply_patch.sh

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/bin/bash
2+
3+
set -x
4+
5+
EBM_DIR=$(python -c 'import ebookmaker; import os.path; print(os.path.dirname(ebookmaker.__file__))')
6+
7+
PATCH_FILE=$(pwd)/ebookmaker-pyinstaller.patch
8+
9+
cd "$EBM_DIR"
10+
patch -p1 < "$PATCH_FILE"

ebookmaker-pyinstaller.patch

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
--- ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/packagers/__init__.py 2020-12-27 15:09:18.381644200 -0800
2+
+++ ./packagers/__init__.py 2020-06-17 17:06:40.168812700 -0700
3+
@@ -18,11 +18,20 @@
4+
import gzip
5+
import zipfile
6+
7+
-from pkg_resources import resource_listdir # pylint: disable=E0611
8+
+from importlib import import_module
9+
10+
from libgutenberg.Logger import debug, info, warning, error
11+
import libgutenberg.GutenbergGlobals as gg
12+
13+
+packagerlist = [
14+
+ "GzipPackager",
15+
+ "HTMLPackager",
16+
+ "PDFPackager",
17+
+ "PushPackager",
18+
+ "RSTPackager",
19+
+ "TxtPackager",
20+
+]
21+
+
22+
GZIP_EXTENSION = '.gzip'
23+
24+
class BasePackager (object):
25+
@@ -157,16 +166,12 @@
26+
def load_packagers (cls):
27+
""" Load the packagers in the packagers directory. """
28+
29+
- for fn in resource_listdir ('ebookmaker.packagers', ''):
30+
- modulename, ext = os.path.splitext (fn)
31+
- if ext == '.py':
32+
- if modulename.endswith ('Packager'):
33+
- module = __import__ ('ebookmaker.packagers.' + modulename,
34+
- fromlist = [modulename])
35+
- debug ("Loading packager type: %s from module: %s for formats: %s" % (
36+
- module.TYPE, modulename, ', '.join (module.FORMATS)))
37+
- for format_ in module.FORMATS:
38+
- cls.packagers[cls.mk_key (module.TYPE, format_)] = module
39+
+ for modulename in packagerlist:
40+
+ module = import_module("ebookmaker.packagers." + modulename)
41+
+ debug ("Loading packager type: %s from module: %s for formats: %s" % (
42+
+ module.TYPE, modulename, ', '.join (module.FORMATS)))
43+
+ for format_ in module.FORMATS:
44+
+ cls.packagers[cls.mk_key (module.TYPE, format_)] = module
45+
46+
return cls.packagers.keys ()
47+
48+
diff -r -u ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/ParserFactory.py ./ParserFactory.py
49+
--- ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/ParserFactory.py 2020-12-27 15:09:18.357809500 -0800
50+
+++ ./ParserFactory.py 2020-06-17 16:28:56.358764100 -0700
51+
@@ -18,7 +18,7 @@
52+
from six.moves import urllib
53+
import six
54+
55+
-from pkg_resources import resource_listdir, resource_stream # pylint: disable=E0611
56+
+from pkg_resources import resource_stream # pylint: disable=E0611
57+
import requests
58+
59+
from libgutenberg.Logger import debug, error
60+
@@ -27,21 +27,22 @@
61+
from ebookmaker.Version import VERSION
62+
from ebookmaker import parsers
63+
64+
+from importlib import import_module
65+
+from ebookmaker.parsers import parserlist
66+
+
67+
+
68+
options = Options()
69+
parser_modules = {}
70+
71+
def load_parsers():
72+
""" See what types we can parse. """
73+
74+
- for fn in resource_listdir('ebookmaker.parsers', ''):
75+
- modulename, ext = os.path.splitext(fn)
76+
- if ext == '.py':
77+
- if modulename.endswith('Parser'):
78+
- module = __import__('ebookmaker.parsers.' + modulename, fromlist=[modulename])
79+
- debug("Loading parser from module: %s for mediatypes: %s" % (
80+
- modulename, ', '.join(module.mediatypes)))
81+
- for mediatype in module.mediatypes:
82+
- parser_modules[mediatype] = module
83+
+ for modulename in parserlist:
84+
+ module = import_module("ebookmaker.parsers." + modulename)
85+
+ debug("Loading parser from module: %s for mediatypes: %s" % (
86+
+ modulename, ', '.join(module.mediatypes)))
87+
+ for mediatype in module.mediatypes:
88+
+ parser_modules[mediatype] = module
89+
90+
return parser_modules.keys()
91+
92+
diff -r -u ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/parsers/__init__.py ./parsers/__init__.py
93+
--- ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/parsers/__init__.py 2020-12-27 15:09:18.397267800 -0800
94+
+++ ./parsers/__init__.py 2020-06-17 16:29:19.533619300 -0700
95+
@@ -31,6 +31,17 @@
96+
97+
from ebookmaker.CommonCode import Options
98+
99+
+# explicitly define the set of available parsers in this directory
100+
+parserlist = [
101+
+ "AuxParser",
102+
+ "CSSParser",
103+
+ "GutenbergTextParser",
104+
+ "HTMLParser",
105+
+ "ImageParser",
106+
+ "RSTParser",
107+
+ "WrapperParser",
108+
+]
109+
+
110+
options = Options()
111+
112+
BROKEN = 'resource://ebookmaker.parsers/broken.png'
113+
diff -r -u ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/WriterFactory.py ./WriterFactory.py
114+
--- ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/WriterFactory.py 2020-12-27 15:09:18.362809400 -0800
115+
+++ ./WriterFactory.py 2020-06-17 16:57:30.094866100 -0700
116+
@@ -16,7 +16,8 @@
117+
118+
import os.path
119+
120+
-from pkg_resources import resource_listdir # pylint: disable=E0611
121+
+from importlib import import_module
122+
+from ebookmaker.writers import writerlist
123+
124+
from libgutenberg.Logger import error, debug
125+
from ebookmaker.CommonCode import Options
126+
@@ -28,19 +29,17 @@
127+
def __load_writers_from (package_name):
128+
""" See what types we can write. """
129+
130+
- for fn in resource_listdir (package_name, ''):
131+
- modulename, ext = os.path.splitext (fn)
132+
- if ext == '.py' and modulename.endswith ('Writer'):
133+
- type_ = modulename.lower ().replace ('writer', '')
134+
- try:
135+
- debug ("Loading writer type %s from module %s" % (type_, modulename))
136+
- module = __import__ (package_name + '.' + modulename, fromlist = [modulename])
137+
- writers[type_] = module
138+
- except ImportError as what:
139+
- error (
140+
- "Could not load writer type %s from module %s. %s" %
141+
- (type_, modulename, what)
142+
- )
143+
+ for modulename in writerlist:
144+
+ type_ = modulename.lower ().replace ('writer', '')
145+
+ try:
146+
+ module = import_module("ebookmaker.writers." + modulename)
147+
+ debug ("Loading writer type %s from module %s" % (type_, modulename))
148+
+ writers[type_] = module
149+
+ except ImportError as what:
150+
+ error (
151+
+ "Could not load writer type %s from module %s. %s" %
152+
+ (type_, modulename, what)
153+
+ )
154+
155+
156+
def load_writers ():
157+
diff -r -u ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/writers/__init__.py ./writers/__init__.py
158+
--- ../../../../ebm_builder-GUvbroJI/Lib/site-packages/ebookmaker/writers/__init__.py 2020-12-27 15:09:18.412937600 -0800
159+
+++ ./writers/__init__.py 2020-06-17 16:53:26.250041000 -0700
160+
@@ -29,6 +29,16 @@
161+
from ebookmaker.Version import VERSION, GENERATOR
162+
163+
164+
+writerlist = [
165+
+ "EpubWriter",
166+
+ "HTMLWriter",
167+
+ "KindleWriter",
168+
+ "PDFWriter",
169+
+ "PicsDirWriter",
170+
+ "RSTWriter",
171+
+ "TxtWriter",
172+
+]
173+
+
174+
class BaseWriter(object):
175+
"""
176+
Base class for EpubWriter, PluckerWriter, ...

0 commit comments

Comments
 (0)