The Great GitBook Rewrite

[Trashtalk217]

GitBook takes the content and converts it to the HTML code webbrowsers can read. However, it became apparent that GitBook is no longer a great fit for the AAA.

Reasons

The version of GitBook we are using is no longer supported. This means that security flaws and bugs are no longer fixed by the dev team.

These problems are the main reasons for a migration away from GitBook, but there are also other potential benefits we could enjoy by switching.

Other Benefits

• A Faster Build
  The build process is currently unbearably slow. I timed it very roughly with my phone and on my machine it takes about 26.7 seconds before you can open the website locally on my machine. This is way too long if you consider that you have to rebuild the website every time you make an edit to an .md file. Debugging becomes especially annoying and time consuming. Switching from GitBook or at least optimizing the build process could mitigate this downtime.

• An Easier Time Adding Features
  Adding features like the language picker can be a bit of a problem with GitBook. You have to work around GitBook instead of working with it and that can sometimes be a real headache.

• Faster Load Times
  This one is not a problem yet. Using a bit of Chrome analytics gives the first meaningful paint at around 250 milliseconds after page startup. This is however with a good wifi connection. It would be worth finding out how fast the page loads with airplane internet. It's also worth seeing how extra chapters affect the load times. When we get more chapters, will it slow down the loading as it loads all the content in one go. Again, this isn't a problem yet, but worth looking into when going for a rewrite.

There are probably other advantages to switching I forgot to mention, but these are the main ones. I think I've made my case pretty clear and most of the archivists already agree with me that switching is a good move. This is not my idea, I'm just stating what has already been said before.

Now the question becomes: 'What do we switch to?"

Options

• The New GitBook
  We are using an old gitbook which is no longer supported, but there is a newer version out there. Sadly, this version isn't open source and monitized. While this version would solve some problems it would create some new ones and is therefore an unattractive candidate.

• Nev's Python Version
  There already have been made some private attempts at improving the build system of the archive. One of these attempts comes from community member Nev (or Neverik). While I'm not sure if it is totally operational it can be found here.

• MdBook
  This is a program @berquist has been looking into. It's written in rust and presumably quite fast, but adding the features we need is going to be difficult.

• Just Fork It
  Another way to fix the problems we have is to just fork the version of GitBook we currently use, rename it AAABook and develop it further to our wishes. The license might get in the way however.

• Alternatives To GitBook
  There are some other out-of-the-box solutions available, which is probably worth checking out first.

• A Tabula Rasa (blank slate)
  We could always start from scratch. This way you can make it exactly like you want and use whatever language you want. I would personally advise against this since it takes a long time and you end up reinventing the wheel a lot.

Final Advice

My final advice is to just start. The current system works, but when something works it's all to easy to become complacent. There current setup can improve and if we don't do anything, it won't improve. So I recommend Leios to open up a separate chat in the Discord server under projects called something along the lines of 'AAABook'. It's also important that we have an official place to work on this projects so I recommend Leios to open up a separate project on GitHub. And while I know that it will still take a while to properly get started, I hope that this can be a start.

[leios]

Thanks for the write-up, it is thorough and clear!

I think @berquist was working on an mdbook solution, but couldn't replicate the language switching behaviour. I am also not sure how @Neverik's python build is.

@c252 has also started working on a script to translate the AAA into LaTeX, which might be worth looking at too.

[berquist]

Here is the Rust "conversion" with https://github.com/rust-lang-nursery/mdBook:

• https://github.com/berquist/algorithm-archive/tree/mdbook
• berquist@08cf6fe (the diff showing what I changed)

It looks like I renamed contents/ to src/, nuked the JS config, and added a minimal mdbook config. Most notably,

• there is no language picker (which was a JS addon anyway),
• I think MathJax is broken,
• and it was not clear how to extend mdbook, and if you need to do it in Rust or if you can do it in JS.

To try and build it,

• Install Rust (https://www.rust-lang.org/tools/install) and do cargo install mdbook
• Inside the repo, mdbook build and mdbook serve work just like gitbook.

[berquist]

Maybe relevant for syntax highlighting, it indicates you can add HTML/JS and have it picked up: rust-lang/mdBook#657

[june128]

I created a project board which holds requirements and optional requirements for the AAAbook solution: https://github.com/orgs/algorithm-archivists/projects/2
Please feel free to discuss and extend the requirements, so that we end up with a good understanding of our needs.

[Trashtalk217]

Thanks to the work of @Neverik, we have a working alternative to gitbook. This has been posted in the discord chat but I feel that documenting it here is important.

Minimum viable alternative: https://aaapy.herokuapp.com/

[neverix]

Just as an example of what aaabook can look like, here's an AAA-Py screenshot

[leios]

Yeah, I have no doubt the aaa-py can work for this. I'm excited to see where it goes. I'm open to other ideas from the community as well.

[Trashtalk217]

We have to make a decision between mdbook and aaa-py someday and I feel like providing some stats might help with that. Below is listed a hassel comparison.

Hassel

Hassel is defined as the difficulty in installing and using the software. To stay open towards newer programmers trying to contribute to open source software, the hassel required getting aaa to build should be as low as possible.

Mdbook

I'll describe the setup required to set up the program in these sections.

Linux setup

The Linux setup is relatively simple although it's not all out of the box.

• Download the special version of the aaa
  • Go to berquist's repo
  • Go to the download button and dowload the zip or clone the repo
  • extract the zip and put it where you want
• Download Rust
  • $ curl https://sh.rustup.rs -sSf | sh
  • $ sudo apt install gcc-multilib (if you don't do this, the next step will not work!)
• Dowload mdbook
  • $ cargo install mdbook

Windows setup

• Copy the modified aaa.
  • I took the zip for this perticular setup, since the cloning gave me a weird error
  • Take it from here
• Install Rust
  • go here
  • press the button with "RUSTUP-INIT.EXE"
  • in command prompt type “y” and “1” and then enter to do a normal installation
  • install additional software
    • go to this site
    • press "Build Tools for Visual Studio 2019"
    • only select "Visual C++ build tools" in the wizard and then install
• Install mdbook
  • Open the command prompt
  • > cargo install mdbook

Build process (both Linux and Windows)

• Go into the algorithm-archive-mdbook folder (with a shell or a terminal)
• mdbook serve

AAA-py

Linux setup

• Clone or download the aaa-py repo
• Python is already pre-installed with most Linux distributions
• Open up a shell
  • $ sudo apt install python3-pip (this may not be necessary)
  • $ pip3 install jinja2
  • $ pip3 install markdown
  • $ pip3 install requests
  • $ pip3 install pybtex
  • $ pip3 install pygments

Windows setup

• Clone or download the zip form nev's repo
• Download Python (not pre-installed on Windows, turns out)
  • Do this the normal way
  • Or via the Windows app store (IT HAS USE!)
  • Make sure you download the latest version (3.x)
• Open up a command prompt
  • > pip install jinja2
  • > pip install markdown
  • > pip install requests
  • > pip install pybtex
  • > pip install pygments

Build process Linux

• Open up a shell
  • cd into the aaa-py folder
  • $ python3 build.py
  • $ cd _book
  • $ python3 -m http.server 8080
• Go to http://localhost:8080/ in your browser

Build process Windows

• Open up a command prompt
  • cd into the aaa-py folder
  • > python build.py
  • > cd _book
  • > python -m http.server 8080
• Go to http://localhost:8080/ in your browser

Conclusion

The Rust for Windows installation was definitely the most painfull, since I had to figure a lot out on the fly. The "Build Tools" download was definitely the most annoying. I had to install about 5 GB of stuff to make an error go away. Mdbook had a definite advantage over aaa-py in the build department. Just one command and it just works. While you have to type like three to get the aaa-py online. AAA-py has a major advantage over mdbook in that we can in fact do something about the current setup and build steps. The pip downloads are probably easy to change.

P.S. These instructions give you a working algorithm archive, but this doesn't mean this is also the shortest way to do things. The instructions are unoptimized and some corners could potentially be cut.

[Trashtalk217]

Now a short speed comparison.

Speed

All of the statistics come from a test done on Linux using the time command.

Mdbook

$ time mdbook load

Gives the following output:

real	0m0,007s
user	0m0,007s
sys	0m0,000s

I tried this several times, but the main takeaway here is that it's ridiculously small. Instant for all practical purposes.

AAA-py

$ time python3 build.py

Gives the following output in normal operations:

real	0m1,133s
user	0m2,750s
sys	0m0,144s

The same command gives the following time when the contents folder is not present (it needs to be cloned). But this barely happens since you only really need to get the full folder once. This is the worst case scenario.

real	0m4,645s
user	0m3,009s
sys	0m0,278s

Conclusion

Mdbook is instant while aaa-py is just very fast. The main thing is that they are both infinitely faster than GitBook.

[Trashtalk217]

Feature comparison

Compare for yourself!
Mdbook vs aaa-py

[neverix]

The commands you ran do different things: build.py builds the entire project from scratch and mdbook load... Shouldn't even exist? This comparison isn't really fair because mdbook load just fails. I believe you used the wrong command.
But even then, aaa-py still doesn't have live reload - which is why it may sometimes be slower. I would suggest someone to try to comprehend its source and maybe add live reload. aaa-py doesn't have an integrated web server either - that would be an interesting thing to add too.