Rendering engine without view files

[davidmerfield]

Hello! I want to write an express rendering engine which uses views retrieved from a database, not the file system. I've checked this repo's issue history and done a lot of googling but I've found nothing...

It seems express requires a view file to exist on disk before passing the req/res to a rendering engine. Is there a way to prevent this behaviour?

Ideally, when I invoke res.render(name)my rendering engine is called with (name, options, callback) without /[name].html necessarily existing on disk.

I could overwrite express' res.render with my own renderer. However in order to handle errors properly it seems I'd have to pass in next each time I called res.render. That seems repetitive.

[hacksparrow]

@davglass Express' view rendering is based on the file system. There is no inherent feature to support what you are looking for.

Mind explaining why you want to do what you are trying to do?

[davglass]

@hacksparrow I think you meant @davidmerfield 😄

[hacksparrow]

Err yes, indeed.

[davidmerfield]

Thank you for your response. I'm not quite sure what other information I can provide.

My question is about the decision to prevent the rendering engine from being invoked if a local template file doesn't exist.

As I understand it, the rendering engine is passed a file's path (and not its contents). The rendering engine is responsible for reading that file from disk and handling any errors it encounters.

Why not just pass the view's name directly to the rendering engine? Let the rendering engine resolve the path to the template and then retrieve its contents.

Rendering Engine Foo could use that view name to look up a template in a database, returning an error if the template doesn't exist. Rendering Engine Bar could use that view name to look up a template on disk, returning an error if the template doesn't exist.

Anyway, thank you for getting back to me. I'm sure there's a lot of context I'm missing for these decisions. Feel free to close this, I'll work something out myself.

[dougwilson]

Hi @davidmerfield, I don't believe @hacksparrow was saying n to you, only trying to ask for more clarification on your request. Perhaps I can provide some better context on the current views system. I think it's fair to say this views system was written prior to any of the current maintainers joined Express, and as such, none of us were actually involved with the creation & definition of this part of Express. Essentially "it is what it is".

BUT that's absolutely not to say it cannot be different.

I understand where you are coming from, but of course can also say that's just not where the Express view engine is today. I think that possibility morphing the view engine into being able to work as you describe may be a better place.

So, @davidmerfield, to jump off this discussion, would you be up for formulating a proposal on how this view system should work, how it differs from the current Express view system, and perhaps even what it would take for us to get from the current to the proposal and what kinds of impact it would have to the current view engine ecosystem and perhaps even what the upgrade paths would be?

[dougwilson]

I could overwrite express' res.render with my own renderer. However in order to handle errors properly it seems I'd have to pass in next each time I called res.render . That seems repetitive.

As for this specific, technical point, you can use the same plumbing Express uses that prevents you from passing in next to res.render: the next function is actually always available as req.next, which you can call instead of requiring a callback.

[hacksparrow]

Maybe something to be discussed at pillarjs/discussions#2

[ScottFreeCode]

The other angle here is 404s. To my knowledge there's three ways for a matched route's handler to 404: it can call next() to let the following routes/handlers try (this isn't an immediate 404, but can become one based on whether the other handlers can serve the request), or it can call next with an error with a status property with the value 404, or it can send a 404 response itself. (Custom error handling for 404s would, I believe, be implemented in an error handler callback, which could be triggered in the event no routes match by a final always-matching route handler that just calls next with a 404 error.) Supposing I had an app.findView function to perform the same lookup that the view engine currently uses so I don't have to duplicate the view lookup logic to get in ahead of the view engine's 500 error, I could check that function and enact the 404 strategy of my choice instead of calling response.render. However, that doesn't solve letting the rendering engine do something other than file lookups with the view name/path, and getting rid of the 500 error to let the rendering engine use non-filesystem resources would be just as broken by such a pre-emptive 404 check as it is by the builtin 500 error.

Whatever solution is worked out here should really have the following characteristics:

• Whatever calls response.render should be able to choose whether a failure to find the view is a 500 error, a call to next() to try more route handlers, or a 404 error.
• The view engine should be able to choose whether the path is resolved against the filesystem or against something else, yet defer to the calling handler's choice of 500 error, next() or 404 error when the expected resource is missing (whatever "missing" means in the place where the resource is looked up).

An extra perk to such a system would be that the rendering engine also could theoretically 404 if any resources other than the named view are missing, e.g. if the view includes a partial and the partial is missing (analogous to how if a static HTML page links to a page that's missing that link is considered a 404 even though it's an error in the site's content).

[ScottFreeCode]

While looking for next("route") in the documentation, I noticed something in the documentation for express.static that led me to some further experimentation, and I've discovered that the simpler way to force a 404 and bypass the remaining routes is to pass next an error with the property status set to 404. This means my initial strategy for handling 404s in general needs a small adjustment, but it also means I can simplify my desired ideal for how the rendering engine allows for custom 404s. I've updated my preceding comment.

[dougwilson]

Hi @ScottFreeCode great comments, though I'm unclear how the 404 discussion relates to having the views use something other than the file system (the topic of this issue). The means that it is likely that we'll close this issue once non-fs views are implemented, and I don't want your comments to get lost in the shuffle.

[ScottFreeCode]

Hi @dougwilson; thanks! Mainly I am figuring that, if Express were to cease sending 500 errors when the view file isn't found so that rendering engines can look for stuff in places other than the filesystem, then it would be up to the rendering engine to send an error such as 404 if the file isn't found (or, if it's not using the filesystem, if whatever else it's looking for isn't matched) -- because right now the 500 error is basically a 404 (the file wasn't found) but with arguably the wrong number, and that automatic error is what's in the way of non-fs views. So, if as I expect the view engines of the future end up responsible for sending the not-found error, I'd like to not end up in a situation where every engine is making its own choices about how those errors are reported, but rather have a fairly standard way for the calling program to tell the engine whether it should send Express a 500 error as the view system does now, send Express a 404 error instead, or just call next(). Maybe that's better suited for discussion at wherever the plans are for view system improvement, though; or maybe it's not something Express can control anyway and just needs to be documented for developers of future rendering engines?

[dougwilson]

I'm still not clear on what you mean, I'm sorry. I can say yes, we give a 500, but only if you are not passing a callback to the render function to do any other behavior. We only 500 if you don't write the code to do something else...?

[ScottFreeCode]

Aaah, you're right; I had forgotten about the callback argument because I initially avoided it in order to avoid having to send the rendered response myself. My bad.