Documentation overhaul

[tybug]

What's in this PR

• Restructure docs following Diataxis
• Move to the furo theme
  • I'm partial to this theme (it's the default I reach for with sphinx), but it's not perfect. In particular contrast for parameter types in method/class signatures is poor IMO. But overall it's the best theme I know of
• New tutorial section
  • All pages are new except for the Adding notes and Stateful tests pages, which have copied content from old docs
• New how-to section (all pages are new)
• New explanation section (all pages are new)
• Split-up reference section. I think very little content here is new, just restructured
• Removed projects below ~500 stars from the "projects using hypothesis" page
  • We can be more than a little selective about this now that hypothesis is so popular!

For reviewers

https://hypothesis--4300.org.readthedocs.build/en/4300/ or make documentation && open hypothesis-python/docs/_build/html/index.html

All critiques welcome, up to and including "we should change the entire structure". I think there's a perception that people are more sensitive to writing criticisms than code criticisms, but my sole goal is to have great docs, and my writing style coming through is habit / ease, not intention. So comment away! Similarly I don't view comments for single word changes etc as "too small", everything adds up and even small stuff can be important in docs (especially on the index page). I'm expecting a lot of back and forth in this pull before we merge it.

[jobh]

Nice! Love the new explanations 👍

[tybug]

we can probably vendor the redirect extension honestly, it's pretty small (and MIT licensed): https://github.com/documatt/sphinx-reredirects/blob/main/sphinx_reredirects/__init__.py

[Zac-HD]

quick thoughts, apologies for limited time for engagement lately

• ⭐ I'm really excited that we're doing this! A lot of review inevitably leans negative about it, so I wanted to say that up front. This is a big project and will be really, really valuable to our users.
• ⭐ I mentioned it's a big project? I'm actually concerned about our ability to effectively review it, and the volume of work that will be in-flight across both micro-scale editing and macro-scale design decisions.
  • Therefore: I'd like to split this back up into smaller PRs, and track with a checklist-style issue as we did for the IR project.
  • Working incrementally does seem to add overhead, but I think that's largely an illusion, and it lets us start delivering valuable changes sooner + I'd bet finish with a better result. We should have an end goal in mind though, so we can e.g. mostly avoid adding content somewhere that we'll shortly need to move or refactor.
• I think we need to be pretty careful about over-documenting internals which we don't actually want users to think about.
  • e.g. shrinking is valuable for maintainers (including to some extent of downstream libraries) to understand, and OK for power users, but I'd keep the latter version pretty abstract and with clear warnings not to over-optimize on that.
• Ideas for subprojects: fixing formatting etc. issues, adding tutorials, adding how-to guides, adding explanations, rearranging existing content. After that, top-level changes and the last round of redirects.
  • I know I'm going to have a lot of opinions on each of these, sorry in advance, and also ideas for more
    • e.g.: configuring databases for company scale development + ci + fuzzing/smoketests
  • For the tutorials in particular, I'd really like to set up client-side code execution - I think pyodide could give us an amazing learning experience. Inspiration: https://github.com/rsokl/testing-tutorial, ghostwriter demo, Examplar, teaching PBT and followup work - in short, what if we had learners write only the strategy or the test body, and checked that their proposal correctly classified each of several correct and incorrect implementations? (this may be a bigger or at least separable project, though 😅. Maybe we should invite Shriram to get involved?)

[tybug]

That makes sense, I'll split this up. (have I said before how glad I am that you think thoughtfully about these things? 😄)

over-documenting internals

we can debate this in a future pull xD, but I'm reminded of Guido's commentary about this:

Ah, here we get to the crux of the matter. What's a "strategy"? Maybe I would be more inclined to support Hypothesis if it was clearer what it does. I like libraries that do stuff for me that I know how to do myself. With Hypothesis, the deeper I get into the docs, the more I get the impression that there is deep magic going on that a poor sap like myself isn't supposed to understand. When I write @given(st.lists(st.integers)) what does that do? I haven't the foggiest idea, and from the docs I get the impression I'm not supposed to worry about it, and that worries me. Clearly it can't be generating all lists of 0 integers followed by all lists of 1 integer followed by all lists of 2 integers ... So in what order does it generate test data? The docs are pretty vague about that.
https://bugs.python.org/issue42109

which I'm highly sympathetic to. I want to make Hypothesis feel as little like magic as possible, and that means not hiding implementation details (to a certain extent). I don't want to document the shrinker or promise anything about its behavior, but we should explain it, and similarly for generation.

executable tutorial examples

there's space for some great stuff here but there's just no way I have time to work on this right now lol, or probably in their near future either

[Zac-HD]

• Yes! "explaining vs documenting" seems like a good way to think about the distinction here, and mark it clearly "if you're interested / not required to use"
• 💯% valid on the executable tutorial side. Maybe I'll have free time and still be nerdsniped over the summer or something.

[tybug]

I'm planning to keep this branch up to date as we merge each piece, so I don't lose any work. I'll continue to split off this PR into smaller ones, so I don't expect this PR to be merged until the very end, if at all.

[ngoldbaum]

It seems the docs on thread safety have disappeared. Any chance the old text could come back so that this isn't a broken link, or so I can fix this link?

https://hypothesis.readthedocs.io/en/latest/details.html#thread-safety-policy

[tybug]

Looks like it got temporarily lost during the transition, thanks. I'll fix this later today!

[ngoldbaum]

sorry for sharing the wrong link earlier 🙃

[tybug]

@ngoldbaum this section is now at https://hypothesis.readthedocs.io/en/latest/supported.html#thread-safety-policy, and we've left a redirect from https://hypothesis.readthedocs.io/en/latest/details.html#thread-safety-policy as well!

[Zac-HD]

consider rebasing? It seems like much of the diff for this PR has already landed.

[tybug]

I don't think so - most of the diff is articles we decided to defer.

I think we should close this PR though, to avoid notification overhead. I'm still keeping the branch up to date as I go, but only to track my own work.

I'm very excited about the new docs that have landed as a result of (splitting off) this pull!