Architecture: Split UI and Backend

[carlosthe19916]

V2 is approaching but before, we might need to brainstorm ideas on how to tackle the 2 points below.

I wanted to bring this topic as this is an important architecture decision we might need to brainstorm before going GA on v2. Experience says that once things are done on a GA product, things get more difficult to change (and not necessarily because of its technical difficulty but for other reasons)

WDYT?

Performance

I noticed that when there are many "Pending" request to the REST API then all the other requests related to pulling HTML, Javacript, CSS, etc. files are blocked until the REST API requests are released. As a consequence, the user cannot navigate (in the browser) from one page to the other as navigating to other pages involve pulling additional Javascript, CSS, etc. files.

So the performance of the REST API has a direct impact on the server being able to serve static files CSS, HTML, JS, etc.

Ideally our application should still be able to serve static files regardless of any negative issue the backend might be impacted by.

The image below is what I see when navigating from the Dashboard UI to the Advisories page. You can see that the Advisories page renders a spinner and it can take a while before it is able to render the page. The reason is that the browser was not able to fetch javascript files because previous REST API calls are still "pending".

Testing

The repository https://github.com/trustification/trustify-tests will contains all the set of E2E tests UI+API for Trustify. And being able to execute those tests, at least, nightly is a good thing to have in order to discover errors at an early stage.

The problem I see is that the backend rarely points to the latest version of the UI available. Therefore, nightly executions of the E2E tests will trigger false positives frequently. Being able to reliable deploy the latest version of both UI and Server will help automation for early detection and continuous maintainability of the e2e tests.

If we were able to split the UI and backend in independent components then our helm chart, operator, etc. would only need to render the latest container image of each component (ui and backend) in order to reliable detect discrepancies between what is expected (tests) and is implemented (the code in both ui and backend).

[jcrossley3]

Ideally our application should still be able to serve static files regardless of any negative issue the backend might be impacted by.

Agree. Have you experimented with bumping up the value of HTTP_SERVER_WORKERS? Its default is the number of cores, which may be less than the number of static files being simultaneously requested.

[carlosthe19916]

No I haven't try it. I am using the Helm Chart.

@jcrossley3 Am I wrong assuming that even if i set HTTP_SERVER_WORKERS then it means that the static files will still be blocked if the REST APIs are occupying the available resources/workers? it will be a matter of opening 5 browsers and, given the number of calls the UI does (e.g in the dashboard page), then I think all workers will be quickly blocked again and not able to serve static files.

Let's use the "HTTP_SERVER_WORKERS" example: to solve the problem we must ensure 1 worker only serves static files and it is never used for anything else; and another "worker" is used exclusively for serving the REST API, this way we ensure the API does not block the static files. Can we do that setting HTTP_SERVER_WORKERS? Note: I am assuming "Workers" are kind of threads or cpus that we can use concurrently, I might be wrong though.

I guess we can also say that we can increase the number of pods/instances of the server (scale horizontally), but again, my naive thoughts make me think that the REST API will still block the static files (it is all a matter of opening a couple of browsers in parallel).

WDYT?

[ctron]

Right now it is not possible setting the HTTP_SERVER_WORKERS variable using the Helm charts. I added a tracking issue for that: guacsec/trustify-helm-charts#28

You can still "experiment" with that, by manually changing it.

However, I am not sure that really is the issue we are talking about. Doing a quick check on the docs: https://actix.rs/docs/server/#multi-threading

For this reason, any long, non-cpu-bound operation (e.g. I/O, database operations, etc.) should be expressed as futures or asynchronous functions. Async handlers get executed concurrently by worker threads and thus don't block execution:

Meaning, that everything waiting on the DB (which I'd expect to be most of the wait times) would not block anything related to the static UI files.

On the other side, the while OCP ingress stack has several components involed (like HAproxy), which we know from the past, can be a limiting factor, for long running HTTP requests.

So before making any suggestions on changing the architecture, let's understand the problem. Testing different settings for HTTP_SERVER_WORKERS is a good first step. Taking OCP out of the picture would be another. Assumptions are good, but should be validated.