Whenever I setup a custom base url I can't get Grav to work (starting with 1.7)

I'm getting the opposite behavior described in #3238

Setup:
https://mydomain.com/foo/ -> [reverse proxy to http://127.0.0.4:8000/] -> Grav

• With 1.6, setting custom_base_url: /foo was fine.
• With 1.7, I'm really blocked:
  • No custom_base_url : Assets are missing (looked for at https://mydomain.com/ without foo/)
  • With custom_base_url: /foo : Pages are not found (even /foo/admin is not found)

custom_base_url clearly affects pages retrieval (and more generally, the whole routing process), but I can't find how to make it work.
Help very welcomed!