Nethermind UI (initial) #8109

benaadams · 2025-01-27T08:55:25Z

Changes

First iteration of a Nethermind UI, so users don't need access to console, have to install Seq or Grafana to monitor what is going on with their node
Mapped when Health checks UI is mapped

Types of changes

What types of changes does your code introduce?

New feature (a non-breaking change that adds functionality)

Testing

Requires testing

No

Documentation

Requires documentation update

Yes

LukaszRozmej

Would be good to move from Nethermind.Runner to a plugin

src/Nethermind/Nethermind.Blockchain/BlockTree.cs

LukaszRozmej · 2025-01-27T09:57:55Z

src/Nethermind/Nethermind.Consensus/Processing/BlockchainProcessor.cs

@@ -63,6 +63,7 @@ public sealed class BlockchainProcessor : IBlockchainProcessor, IBlockProcessing
    private readonly Stopwatch _stopwatch = new();

    public event EventHandler<IBlockchainProcessor.InvalidBlockEventArgs>? InvalidBlock;
+    public event EventHandler<BlockStatistics>? NewProcessingStatistics;


Can we just pass through add/remove for simplicity?

LukaszRozmej · 2025-01-27T09:59:33Z

src/Nethermind/Nethermind.Runner/ConsoleHelpers.cs

@@ -41,3 +61,114 @@ public static void EnableConsoleColorOutput()
    [DllImport("kernel32.dll")]
    private static extern bool SetConsoleMode(IntPtr hConsoleHandle, uint dwMode);
 }
+
+
+public sealed class LineInterceptingTextWriter : TextWriter


Do we really need that? Doesn't NLog have something that would make it simpler for us?

Not really; that I could find :(

LukaszRozmej

Apart of my comments I'm fine with rest from C#.
Someone needs to review TypeScript, @rubo ?

LukaszRozmej · 2025-02-19T17:17:49Z

src/Nethermind/Nethermind.Runner/Ethereum/JsonRpcRunner.cs

@@ -76,6 +76,7 @@ public async Task Start(CancellationToken cancellationToken)
                    s.AddSingleton(_jsonRpcUrlCollection);
                    s.AddSingleton(_webSocketsManager);
                    s.AddSingleton(_rpcAuthentication);
+                    s.AddSingleton(_api);


I don't like this, we want to remove/hide api not inject it

Changed to inject them individually

src/Nethermind/Nethermind.Runner/Monitoring/DataFeed.cs

LukaszRozmej · 2025-02-19T17:18:54Z

src/Nethermind/Nethermind.Runner/Monitoring/DataFeed.cs

+        await ctx.Response.WriteAsync($"event: nodeData\n", cancellationToken: ct);
+        await ctx.Response.WriteAsync($"data: ", cancellationToken: ct);
+        await ctx.Response.Body.WriteAsync(GetNodeData(), cancellationToken: ct);
+        await ctx.Response.WriteAsync($"\n\n", cancellationToken: ct);
+
+        await ctx.Response.WriteAsync($"event: txNodes\n", cancellationToken: ct);
+        await ctx.Response.WriteAsync($"data: ", cancellationToken: ct);
+        await ctx.Response.Body.WriteAsync(TxPoolFlow.NodeJson, cancellationToken: ct);
+        await ctx.Response.WriteAsync($"\n\n", cancellationToken: ct);
+
+        await ctx.Response.WriteAsync($"event: log\n", cancellationToken: ct);
+        await ctx.Response.WriteAsync($"data: ", cancellationToken: ct);
+        await ctx.Response.Body.WriteAsync(JsonSerializer.SerializeToUtf8Bytes(ConsoleHelpers.GetRecentMessages(), JsonSerializerOptions.Web), cancellationToken: ct);
+        await ctx.Response.WriteAsync($"\n\n", cancellationToken: ct);


Wouldn't it be better to do that in 1 WriteAsync?

LukaszRozmej · 2025-02-19T17:19:10Z

src/Nethermind/Nethermind.Runner/Monitoring/DataFeed.cs

+            await ctx.Response.WriteAsync($"event: {entry.Type}\n", cancellationToken: ct);
+            await ctx.Response.WriteAsync($"data: ", cancellationToken: ct);
+            await ctx.Response.Body.WriteAsync(entry.Data, cancellationToken: ct);
+            await ctx.Response.WriteAsync($"\n\n", cancellationToken: ct);


LukaszRozmej · 2025-02-19T17:21:45Z

src/Nethermind/Nethermind.TxPool/TxPool.cs

+                Metrics.TransactionsSourcedPrivateOrderFlow += notInMempoool;
+                Metrics.TransactionsSourcedMemPool += transactionsInBlock - notInMempoool;


With DarkPoolRatio, do we need those 2 metrics?

src/Nethermind/Nethermind.TxPool/TxPool.cs

rubo

Although the UI concept is cool, I have some concerns about the approach. I'd prefer this to be a separate component instead of being incorporated into the Nethermind Runner. Please see the comments.

rubo · 2025-02-19T19:21:09Z

src/Nethermind/Nethermind.Runner/Nethermind.Runner.csproj

+  <Target Name="JsBuild" BeforeTargets="PreBuildEvent">
+    <Exec Command="yarn build:dev" Condition="'$(Configuration)'=='Debug'" />
+    <Exec Command="yarn build:release" Condition="'$(Configuration)'=='Release'" />
+  </Target>


It is highly undesirable to make yarn, hence node.js, a required tool to build Nethermind for a non-essential and optional component as the UI. The build failed for me just because I didn't have yarn.

A few suggestions:

Do not use node.js/yarn at all

Provide already minified versions (do we really need it, given its local usage only?)

Having the UI as a separate NuGet package, like the health checks. I think this is the best approach and similar to the plugin approach suggested by @LukaszRozmej. I can help with that.

Do not use node.js/yarn at all

Removed yarn; still uses node because need that to compile typescript, even with the dotnet build process

Having the UI as a separate NuGet package, like the health checks.

Would be annoying af to develop as the UI is directly dependent on the data the Runner is providing and will be evolving over time. Health checks sit on an agreed upon standardised api

node is only used in docker to build; its not included in the final container

I agree that maintaining a NuGet package is somewhat annoying, but bringing the Node.js as a new build dependency for some optional components is also annoying. I believe not everyone in the team uses or even has Node.js & stuff, let alone third parties that build from source. Moreover, with a NuGet package, you'll have more liberties regarding the organization and approaches, as there will be much less nitpicking.
Of course, this is just my opinion, but I think I'm not alone in that. I'd like more opinions here.

rubo · 2025-02-19T19:25:13Z

src/Nethermind/Nethermind.Runner/wwwroot/fonts/dm-mono-latin.woff2

Why have all these fonts in the repo instead of downloading them from HTML?

Web site has 0 requirement for an internet connection (just to your node) and doesn't send beacon requests to google

Maybe you want to put it on a big TV, but don't want to give that TV full access to the Internet just to download some fonts

I see, but on the other hand, binary stuff is not git-friendly. If it was a separate package, it would be much less of a concern.

src/Nethermind/Nethermind.Runner/wwwroot/favicon.webp

rubo · 2025-02-19T19:29:26Z

src/Nethermind/Nethermind.Runner/wwwroot/nethermind.svg

Better to put this in the logo directory with the other images and rename it to images. So, all the graphics are in the same place.

Is a special image and different dimensions; logos is 3rd party chain logos and all the same dimensions

Still, images can go into their separate directory, and there you can organize them however you wish. As mentioned above, if it was a separate package, I wouldn't care.

emlautarom1

Some comments on the TS side of things.

emlautarom1 · 2025-03-10T19:17:49Z

src/Nethermind/Nethermind.Runner/wwwroot/fonts/dm-mono-latin-ext.woff2

We probably need to include some LICENCE file for these fonts.

emlautarom1 · 2025-03-10T19:18:38Z

src/Nethermind/Nethermind.Runner/wwwroot/css/app.css

@@ -0,0 +1,272 @@
+/* latin-ext */


We probably want to put this file through some code formatter.

emlautarom1 · 2025-03-10T19:20:41Z

src/Nethermind/Nethermind.Runner/package.json

+    "ansi-to-html": "0.7.2",
+    "d3": "7.9.0",
+    "d3-sankey": "0.12.3"


We are listing these dependencies but we are also vendoring them (lib/*.js). Why is that?

emlautarom1 · 2025-03-10T19:33:02Z

src/Nethermind/Nethermind.Runner/scripts/utilities.ts

+  // Prepare day, month, year, hours, minutes, seconds
+  const day = String(date.getDate()).padStart(2, '0');
+  const month = months[date.getMonth()];
+  const year = date.getFullYear();
+  const hours = String(date.getHours()).padStart(2, '0');
+  const minutes = String(date.getMinutes()).padStart(2, '0');
+  const seconds = String(date.getSeconds()).padStart(2, '0');
+
+  // Return the formatted date string
+  // Example: "09 Feb 2025 13:45:00"
+  return `${day} ${month} ${year} ${hours}:${minutes}:${seconds}`;


Suggested change

// Prepare day, month, year, hours, minutes, seconds

const day = String(date.getDate()).padStart(2, '0');

const month = months[date.getMonth()];

const year = date.getFullYear();

const hours = String(date.getHours()).padStart(2, '0');

const minutes = String(date.getMinutes()).padStart(2, '0');

const seconds = String(date.getSeconds()).padStart(2, '0');

// Return the formatted date string

// Example: "09 Feb 2025 13:45:00"

return `${day} ${month} ${year} ${hours}:${minutes}:${seconds}`;

return date.toLocaleString(navigator.language, {

dateStyle: 'long',

timeStyle: 'medium',

hour12: false

});

If we want consistency regarding language we can force the locale to be en-UK instead of navigator-language. The date then looks like 10 March 2025 at 16:25:57 in all devices.

en-uk is invalid though. en is enough here ;)

Is it invalid semantically or do you get an error when trying to use it? en gets me the MM/DD/YYYY format which is common in the US while en-UK gets me DD/MM/YYYY.

I mean the correct locale for the UK is en-gb, although the browser somehow recognizes the other one. As we use a neutral locale for Nethermind everywhere else, en is the one to go with.

src/Nethermind/Nethermind.Runner/scripts/utilities.ts

emlautarom1 · 2025-03-10T20:23:48Z

src/Nethermind/Nethermind.Runner/scripts/app.ts

+  }
+  updateText(version, data.version);
+  updateText(network, networkName);
+  (document.getElementById("network-logo") as HTMLImageElement).src = `logos/${getNetworkLogo(data.network)}`;


Same as with previous getElementById

emlautarom1 · 2025-03-10T20:27:02Z

src/Nethermind/Nethermind.Runner/scripts/app.ts

+  updateTreemap<TransactionReceipt>(
+    document.getElementById("block"), // or d3.select("#treemap")
+    200,
+    parseInt(data.head.gasLimit, 16),
+      mergedData,
+      // keyFn
+      d => d.hash,
+      // orderFn
+      d => d.order,
+      // sizeFn
+      d => parseInt(d.gasUsed, 16),
+      // colorFn
+      d => parseInt(d.effectiveGasPrice, 16) * parseInt(d.gasUsed, 16)
+  );


Maybe refactor updateTreeMap to accept an object for named parameters instead of relying on comments?

Suggested change

updateTreemap<TransactionReceipt>(

document.getElementById("block"), // or d3.select("#treemap")

200,

parseInt(data.head.gasLimit, 16),

mergedData,

// keyFn

d => d.hash,

// orderFn

d => d.order,

// sizeFn

d => parseInt(d.gasUsed, 16),

// colorFn

d => parseInt(d.effectiveGasPrice, 16) * parseInt(d.gasUsed, 16)

);

updateTreemap<TransactionReceipt>(

{

element: document.getElementById("block"), // or d3.select("#treemap"),

height: 200,

totalSize: parseInt(data.head.gasLimit, 16),

data: mergedData,

keyFn: d => d.hash,

orderFn: d => d.order,

sizeFn: d => parseInt(d.gasUsed, 16),

colorFn: d => parseInt(d.effectiveGasPrice, 16) * parseInt(d.gasUsed, 16)

});

src/Nethermind/Nethermind.Runner/scripts/app.ts

emlautarom1 · 2025-03-10T20:40:24Z

src/Nethermind/Nethermind.Runner/scripts/app.ts

+      frag.appendChild(newEntry);
+    }
+    logs = [];
+    nodeLog.appendChild(frag);


Do we drain this nodeLog element at some point? Otherwise if we keep appending then we'll run OOM.

emlautarom1 · 2025-03-10T20:49:50Z

src/Nethermind/Nethermind.Runner/scripts/utilities.ts

+  "Mainnet": "ethereum-logo.svg",
+  "1": "ethereum-logo.svg",
+  "480": "world-logo.svg",
+  "8453": "base-logo.svg"


I see that we have logos for Base and Worldchain. No Optimism?

Copilot

Pull Request Overview

This PR introduces the initial iteration of a Nethermind UI to allow node monitoring without external tools, while adding TxPool metrics reporting and updating various parts of the codebase to support these features.

Introduces new TxPool monitoring classes (TxPoolFlow, Link, Node).
Enhances JSON RPC startup by mapping data feeds and enabling static file serving when health checks are enabled.
Adds console message interception for logging output and new processing statistics events.

Reviewed Changes

Copilot reviewed 57 out of 60 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Nethermind/Nethermind.Runner/Monitoring/TxPool/TxPoolFlow.cs	New TxPool metrics and state linking logic.
src/Nethermind/Nethermind.Runner/Monitoring/TxPool/Node.cs	Defines the Node class used in TxPoolFlow.
src/Nethermind/Nethermind.Runner/Monitoring/TxPool/Link.cs	Introduces immutable Link class for TxPool state tracking.
src/Nethermind/Nethermind.Runner/Monitoring/NethermindNodeData.cs	Maps node information for monitoring.
src/Nethermind/Nethermind.Runner/Monitoring/DataFeedExtensions.cs	Adds data feed mapping to endpoint routes.
src/Nethermind/Nethermind.Runner/JsonRpc/Startup.cs	Updates JSON RPC startup to support new endpoints and static files.
src/Nethermind/Nethermind.Runner/Ethereum/JsonRpcRunner.cs	Registers additional services for TxPool support.
src/Nethermind/Nethermind.Runner/ConsoleHelpers.cs	Enhances console output interception and event emission.
src/Nethermind/Nethermind.Runner.Test/StandardTests.cs	Temporarily comments out a metrics test.
src/Nethermind/Nethermind.Consensus/Processing/*	Adds new processing statistics events and related handlers.
src/Nethermind/Nethermind.Blockchain/*	Introduces ForkChoiceUpdated events and corresponding structural changes.

Files not reviewed (3)

Dockerfile: Language not supported
Dockerfile.chiseled: Language not supported
src/Nethermind/Nethermind.Runner/Dockerfile: Language not supported

Comments suppressed due to low confidence (1)

src/Nethermind/Nethermind.Blockchain/ReadOnlyBlockTree.cs:156

The empty add/remove accessors for the OnForkChoiceUpdated event may lead subscribers to receive no notifications; confirm that this no-op implementation is intentional or provide a proper backing event if notifications are expected.

event EventHandler<IBlockTree.ForkChoice> IBlockTree.OnForkChoiceUpdated { add { } remove { } }

Copilot · 2025-03-29T00:03:54Z

src/Nethermind/Nethermind.Runner/ConsoleHelpers.cs

+    {
+        lock (_recentMessages)
+        {
+            if (_recentMessages.Count > 100)


The condition may allow the queue to exceed 100 messages; using '>=' instead of '>' can ensure that the queue does not grow beyond the intended limit.

Suggested change

if (_recentMessages.Count > 100)

if (_recentMessages.Count >= 100)

Copilot · 2025-03-29T00:03:55Z

src/Nethermind/Nethermind.Runner/JsonRpc/Startup.cs

@@ -168,7 +180,7 @@ public void Configure(IApplicationBuilder app, IWebHostEnvironment env, IJsonRpc
                }
            }

-            if (method == "GET")
+            if (method == "GET" && !(ctx.Request.Headers.Accept[0].Contains("text/html", StringComparison.Ordinal)))


Accessing the first element of the Accept header without checking if the header is non-empty may lead to an index out-of-range exception; consider verifying that the header has at least one entry before indexing.

Suggested change

if (method == "GET" && !(ctx.Request.Headers.Accept[0].Contains("text/html", StringComparison.Ordinal)))

if (method == "GET" && ctx.Request.Headers.Accept.Count > 0 && !(ctx.Request.Headers.Accept[0].Contains("text/html", StringComparison.Ordinal)))

benaadams · 2025-04-09T21:42:07Z

Was having issue with github after rebase, so reopened at this PR #8503

benaadams added the ux label Jan 27, 2025

benaadams changed the title ~~Nethermind UI~~ Nethermind UI (initial) Jan 27, 2025

LukaszRozmej reviewed Jan 27, 2025

View reviewed changes

benaadams requested review from rubo and a team as code owners February 19, 2025 01:10

LukaszRozmej approved these changes Feb 19, 2025

View reviewed changes

rubo requested changes Feb 19, 2025

View reviewed changes

benaadams force-pushed the nethermind-ui branch from 18b3229 to 8f5ffe2 Compare February 21, 2025 12:43

emlautarom1 reviewed Mar 10, 2025

View reviewed changes

Nethermind UI

06aa86b

benaadams force-pushed the nethermind-ui branch from 2bf7b0c to 06aa86b Compare March 26, 2025 13:10

benaadams requested a review from Copilot March 29, 2025 00:03

Copilot AI reviewed Mar 29, 2025

View reviewed changes

benaadams closed this Apr 9, 2025

		Metrics.TransactionsSourcedPrivateOrderFlow += notInMempoool;
		Metrics.TransactionsSourcedMemPool += transactionsInBlock - notInMempoool;

	if (_recentMessages.Count > 100)
	if (_recentMessages.Count >= 100)

	if (method == "GET" && !(ctx.Request.Headers.Accept[0].Contains("text/html", StringComparison.Ordinal)))
	if (method == "GET" && ctx.Request.Headers.Accept.Count > 0 && !(ctx.Request.Headers.Accept[0].Contains("text/html", StringComparison.Ordinal)))

Nethermind UI (initial) #8109

Nethermind UI (initial) #8109

Conversation

benaadams commented Jan 27, 2025 • edited Loading

Changes

Types of changes

What types of changes does your code introduce?

Testing

Requires testing

Documentation

Requires documentation update

LukaszRozmej left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LukaszRozmej left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rubo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emlautarom1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Copilot AI Mar 29, 2025

Choose a reason for hiding this comment

Copilot AI Mar 29, 2025

Choose a reason for hiding this comment

benaadams commented Apr 9, 2025

benaadams commented Jan 27, 2025 •

edited

Loading