Noah/webgpu scatterbrain pt1 by froyo-np · Pull Request #246 · AllenInstitute/vis

froyo-np · 2026-04-27T17:10:28Z

WIP Steps

This section is optional if the PR is fully ready for review

Create the PR as a "draft" PR
Add [WIP] to the title
Use the template below to ask for specific feedback
@name: Could you look at ___? I'm trying to do... (etc.)

What

Replace this with 1-2 line Description of what feature, bugfix, chore does the PR contain the code for.

How

Replace this txt describing what kind of technical overlaying code changes were introduced here.

Screenshots

This section is optional if there are no visible changes

If possible add screenshots of the visible additions in the UI.
If there are changes in the UI, add Before and After Screenshots for quick overview.
If there was a Figma design, add a link to that here as well.
Hint : Drag and Drop any images you want to add to the PR. Also you can create a gif of an interactive version and add that!

PR Checklist

…ain (to bring it into connectomics quickly...) experimental, might try typeGPU again

…sformer plugin...

…, to compare it with the type-gpu version

froyo-np · 2026-04-27T17:12:59Z

+
+type Content<V extends Cacheable> = Record<string, V>
+
+export function buildScatterbrainCacheClient<V extends Cacheable>(


this file got moved - its mostly the same as the old webGL cache client! it did get a tiny tweak to make it agnostic to webGL / webGPU - you can use the same function (this one) to create either a webGL or webGPU cache-client, and they can even share the same cache, if you wanted

froyo-np · 2026-04-27T17:13:47Z

@@ -0,0 +1,32 @@
+<!DOCTYPE html>


this is the demo I used to debug my webGPU implementation - we should remove it, but for now starlight seems to have trouble with our webGPU utils (a 3rd party helper)

froyo-np · 2026-04-27T17:14:41Z

+} & QuantitativeFilterRanges
+
+
+export function generate(config: Config): string {


the guts of the shader that we generate to render a scatterplot in a particular configuration

@Jarbuckle interested in your thoughts on this part!

Alrighty, I shall take a look! 🙂

froyo-np · 2026-04-27T17:15:45Z

+    const toGpuBuffer = (buffer: ArrayBuffer, type: WebGLSafeBasicType) => {
+        if (type === 'uint16') {
+            // seems like uint16 is cursed - vertex buffers have to have a stride of at least 4...
+            const B = device.createBuffer({ size: buffer.byteLength * 2, usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.VERTEX });


here is the hack I use to deal with uint16 data - we expand it here when we cache it - its not slow, and we still get to keep our 16bit ints over the wire, but on the GPU we have to expand them to 32bit

note that uint16 data is supported in webGPU, but any buffer that is bound as a vertex attribute must declare an arrayStride of at least 4 (bytes). this means if that its easy to use u16 data in an interleaved buffer, but impossible to use via separate buffers as we do for scatterbrain style columnar vertex data

froyo-np · 2026-04-27T17:16:38Z

+        const bg1 = updateCategorical(categories);
+        const bg2 = updateGradient(viridis); // todo - dont do this every frame...
+
+        // so... the gad damn bindings - if you dont use a binding, it needs to be omitted from


this comment was made in anger, as I beat my head against a wall trying to get the shader to validate. remove!

froyo-np · 2026-04-27T17:17:19Z

+};
+
+
+export async function whatever() {


the bootstrap function for my demo - remote eventually

…with the webGPU stuff (startlight cant build wgpu-utils, site/ cant resolve webGPU types)

…as all my fault

…ain-pt1

… node-dependencies...

thinking through how to make better use of how webGPU works with regard to the nice way of thinking via REGL.

…ain-pt1

chrisj · 2026-05-18T16:34:06Z

+        // ugh, it also requires bindgroup layouts in non-auto mode - such a slog
+        // ok - seems like the most normal-person thing to do here would be to split out the per-qt-node and the per-frame uniforms into 2 groups
+        // then create the per-qt-node data when we load it, and stow it in the cache... that would be ok, although it does require
+        // a bindGroupLayout (non-automode) separate from the creation of the pipeline... thats gonna be a good idea anyway if anyone ever changes any settings for these...


The decision here is very sound, we should just clean up this comment

chrisj · 2026-05-18T16:37:04Z

+
+function columnsForItem<T extends object>(
+    config: Config,
+    col2shader: Record<string, string>,


I would still name it columnNameToShaderName here

chrisj · 2026-05-18T16:39:14Z

+    dataset: ScatterbrainDataset | SlideviewScatterbrainDataset,
+) {
+    const columns: Record<string, ColumnRequest> = {};
+    const s2c = Object.keys(col2shader).reduce(


shaderNameToColumnName

chrisj · 2026-05-19T15:56:06Z

+        }
+        const bg = device.createBindGroup({
+            label: 'single bg',
+            entries,


does createBindGroup succeed here if settings.categoricalFilters and quantitativeRangeFilters are empty?

I'm having some trouble running the demo

froyo-np added 10 commits April 22, 2026 12:10

thinking about what it would be like to use webGPU here for scatterbr…

6b57bba

…ain (to bring it into connectomics quickly...) experimental, might try typeGPU again

switch to vite (library mode) because it supports typeGPU's wgsl tran…

0b0d594

…sformer plugin...

minor import cleanup

a64a9d8

try typegpu until I go crazy, but it does (almost) work

71db373

fix the uint16 mystery. implement the raw webGPU scatterplot renderer…

8dd215c

…, to compare it with the type-gpu version

WIP total reorg

376e96c

fix oh so many mistakes

b04f894

continue to fight with stuff to make the examples work...

8a7ab4f

lots of cleanup

80532be

more cleanup

0051cd3

froyo-np commented Apr 27, 2026

View reviewed changes

froyo-np added 5 commits May 1, 2026 08:24

fmt

e93aa02

tsconfig webgpu types

3eaf74d

webgpu types

b04849f

put everything back to regl until we can figure out what is going on …

855c371

…with the webGPU stuff (startlight cant build wgpu-utils, site/ cant resolve webGPU types)

clean it up

c5bed41

froyo-np marked this pull request as ready for review May 1, 2026 16:11

froyo-np requested a review from a team as a code owner May 1, 2026 16:11

froyo-np requested review from TheMooseman and suyli7 May 1, 2026 16:11

froyo-np added 5 commits May 1, 2026 09:14

lint & fmt

3f12b87

fix a very silly hack that was totally ruining astro's build and it w…

da81118

…as all my fault

oh also this has float16array so thats good

47afe9a

cleanup experimental stuff

086a838

update lockfile

24d78a0

froyo-np added 2 commits May 4, 2026 09:45

Merge remote-tracking branch 'origin/main' into noah/webgpu-scatterbr…

9b3cda9

…ain-pt1

install

b582440

froyo-np requested review from Jarbuckle and removed request for suyli7 May 4, 2026 17:35

froyo-np added 7 commits May 4, 2026 10:44

use node 24 in ci actions, so that it doesnt choke on Float16Array in…

82d8fa3

… node-dependencies...

for some fun reason lodash does not play nice with parcel...

43d3ae7

thinking through how to make better use of how webGPU works with regard to the nice way of thinking via REGL.

TIL lodash-es, hooray!

26b03df

beat stuff up until it works

46c7a1a

make it build, confirm working webgpu example... discover little bug...

8a4c235

a note about per-draw uniform data...

8804591

Merge remote-tracking branch 'origin/main' into noah/webgpu-scatterbr…

c66c468

…ain-pt1

chrisj reviewed May 19, 2026

View reviewed changes


		type Content<V extends Cacheable> = Record<string, V>

		export function buildScatterbrainCacheClient<V extends Cacheable>(

		} & QuantitativeFilterRanges


		export function generate(config: Config): string {

Conversation

froyo-np commented Apr 27, 2026

WIP Steps

What

How

Screenshots

PR Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

froyo-np Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

froyo-np Apr 27, 2026 •

edited

Loading