Malt 1.0 UX Improvements (Feedback needed)

[pragma37]

As we get near the 1.0 release, it's time to re-evaluate some of the Malt design decisions.
This thread is for listing and discussing UX improvements that require breaking backward compatibility.

Feel free to submit your own.

For new feature requests that could be added after 1.0, please open a new thread:
https://github.com/bnpr/Malt/discussions/categories/feedback-feature-requests

For discussing proposed changes, write a direct reply instead of creating a new comment.

Important!

Breaking backward compatibility means old Malt scenes will have to be updated to work on the newer versions.
All the current changes from the Development branch will be merged into the Release branch, and the Development branch will be more unstable than it used to be.
From now on, I'd recommend using the Development version only for testing, and use the Release version for everything else.

[pragma37]

Default Material (Done)

Current:

The default material is an unlit yellow shader.

Proposed:

Use a light grey diffuse material as default.
Allow using a different default.
Leave yellow unlit only for compilation errors.

Rationale:

It’s common to start modeling objects without a material applied, so having a usable default material can be handy.
Incorrectly setup materials should be unambiguously presented as such.

[pragma37]

Scene and World parameters (Done)

Current:

Malt pipelines declare Scene and World parameters.

Proposed:

Move all Scene parameters to World.

Rationale:

Makes it easier to share render settings across Scenes and Blend files (by appending/linking) while still being able to override them.

[pragma37]

Vertex Layout

Current:

Malt loads to OpenGL vertex positions, normals, UVs (up to 4), precomputed tangents/bitantgents (up to 4) and vertex colors (up to 4)

Proposed:

Load precomputed tangents/bitangents for the first UV only and compute the rest on the pixel shader.
Load vertex weights (up to 4)

Rationale:

OpenGL has a limit of 16 per-vertex attributes, and precomputed tangents are (usually) only useful for low-poly models.
Blender is extremely slow at computing tangents.
Vertex groups are a commonly requested feature.

[pragma37]

PixelOutput shadow_color (Done)

Proposed:

Rename shadow_color to transparent_shadow_color

Rationale:

The current name is misleading, it’s only used for transparent materials.

[pragma37]

Pixel Units

Current:

Some features (like line width) are specified in pixels.

Proposed:

Use a different type of unit.

Options:

• Screen percentage (width or height? whatever is lower or higher? diagonal?)
• Keep using pixels, but use a global scaling multiplier or a “reference resolution”.
• Keep using pixels, but add extra functions/nodes to use the previous options.

Rationale:

Using pixels as units can heavily change the look of an artwork when rendering at different resolutions (ie. 1080 vs 4k) and by the time the user notices the problem, it may be hard to fix.

[aVersionOfReality]

I can confirm that this is very important for doing complex line art. We had this pixel problem in Freestyle and it was very limiting. (Although in Malt we can at least fix it since we can get the current resolution and adjust things more easily. FS needed tons of drivers.)

Having it work like the Blur node in the compositor, where you can choose either Pixels or Relative seems like the right idea.
So I think your 3rd bullet point is the way to go.

[MatchaChoco010]

Not only LineArt, but also like Bloom expressions using blur would be difficult to use if they are pixel-dependent, so I would be happy if relative values of draw area could be used.

In adding relative values of drawing area, I have a request.
My request is that the relative values should take into account the area that is shown in the camera when the camera is previewed with the 0 key.

What I want is not the relative value of the area shown in the next image.

In my opinion, the relative value of the area shown in the next image should be displayed.

[aVersionOfReality]

I think it'd be handy if we had more access in general to Final Render and Viewport render. I know there's some areas we can set different settings for Preview (which I assume means viewport?) but I'm not sure if that's something we can just get as a Fac in the nodes.
That sort of thing is probably the right solution for having an effect display outside the camera clipped area in the viewport.

[pragma37]

@MatchaChoco010 The way I wanted to handle this is by simply rendering ONLY the inner area. That would allow simplifying a few things on the rendering side.

@aVersionOfReality The Preview overrides are only applied in the Material Preview Viewport Shading Mode.
Final Render overrides are for F12.
The default value, if you override both the Preview and Final Render would be only used in the Rendered Viewport Shading Mode.

[pragma37]

IDs (Done)

Current:

IDs are used for detecting lines at object boundaries, computing bevels, and detecting self-shadows.
Each object has its own ID and it can be overridden from shaders.

Proposed:

Use 4 ID channels

Options:

• Object, Material, Mesh, and Custom
• Object, CustomA, CustomB, CustomC

Rationale:

It’s common to want different lines at different types of intersections.
You may want to use thick lines at Object boundaries and thinner lines at Custom boundaries (for example at clothing seams).
Or you may want to detect boundaries between leaves and a bark tree, but not between different leaves objects.

[HSDSZ]

Maybe I misunderstood how IDs works
Check the node. The PO.id has colors as input. In this case, the red part and the blue part have different Id. Shouldn't I see a outline at the color boundary?

[HSDSZ]

Here is another way I guess how the po.id works.
I assign different material to the left and right half of the cube. They have different po.id input. but I still don't have outline in between

[pragma37]

@HSDSZ There are several things going on there.

1. There was a bug that I just fixed in the ID code (only the first ID channel was getting assigned).
2. With that pack_8bit node you are storing your vec4 property into a single 32-bit uint, but IDs are stored in 16bit textures, so half of your data is lost.
   In your second example, if you use uvec4_join nodes with different values, it should work.
3. For some reason, lighting in the PRE_PASS (where the IDs are rendered) doesn't work in the Development branch. It does in Release.
   I have to figure out what's going on.
   Edit: Nope, it actually works.

Also, you seem to be using Blender 3.0, which is not yet supported.

[HSDSZ]

Well, I see. I misunderstood the uvec4 data type. I thought it was made for that pack_8bit function. But it actually it is not.

Here is how it should work I think (in blender 2.93)

Btw, the above nodetree also works in blender 3.00. Only that the vec3/vec4 property display in a vector form, which is acceptable.

[pragma37]

uvec stands for "unsigned integer vector".
So its a vector that holds positive integer numbers (you can use any number from 0 to 65536).

The idea is not to use the 4 channels as a single ID, but use each 1 for different things. By default the R channel is a unique per-object ID and the others are set to 0.
You could use, for example, one channel for material IDs, other to separate light and shadows and the last one for subsections inside a single mesh (think for example individual hair strands).
The cool thing is that it will be possible to use different types of line for the boundaries of each channel.

It's also best to leave the R channel to the default value, since that one is also used for detecting self-shadows. Although there could be situations where you want to use the same "object" ID for multiple meshes (for example, all the sub-meshes of a single character).

This is a somewhat better way to do the same thing you are doing there:

The pack_8bit node was made for packing up to 4 8-bit textures into a single AOV. There's an unpacking Render Layer node.
See #94 (comment)

[pragma37]

Libraries

Current:

Malt uses paths for selecting GLSL shaders, GLSL libraries and render pipelines.

Proposed:

Use global (always available) and local (selected per project) libraries.
Automatically scan and reload nodes, shaders and pipelines from those libraries.
Use searchable drop-down menus instead of raw paths.
Make them automatically available to include/import from glsl/python.

Use a pre-defined directory layout, so Malt knows where to find the relevant files, for example:

• Library
  • GLSL Library
    • Optional SubFolder
      • *glsl files
    • *glsl files
  • Pipeline Nodes Library
    • Optional Subfolder
      • *python files
    • *python files
  • Pipelines Library
    • CustomPipeline
      • CustomPipeline.py
      • GLSL Library
        • Optional SubFolder
          • *glsl files
        • *glsl files
      • Pipeline Nodes Library
        • Optional Subfolder
          • *python files
        • *python files

Rationale:

Encourages a cleaner and easier way to handle custom user libraries and makes them easier to share.

[pragma37]

More explicit Render Layer

Current:

The default Malt Pipeline is composed of several scene and screen passes. More info here:
https://github.com/bnpr/Malt/tree/Development/Malt/Pipelines/NPR_Pipeline#render-overview

These steps are hard-coded and the user has no control over them. Render Layer nodes only allows post-processing the results of the previous steps.

Proposed:

Rewrite each of these steps as configurable Render Layer nodes.

Rationale:

Allow more customization and extensibility. For example:

• Customize line rendering.
• Modify PRE_PASS normals (bevel) before sending them to the MAIN_PASS.
• Render multiple PRE_PASSes and MAIN_PASSes. For example, for inverted hull outlines.

[pragma37]

Simplify COMMON_PIXEL_SHADER Input/Output (Done)

Current:

At least 3 nodes are needed for the basic setup of a material:
COMMON_PIXEL_SHADER Input -> PixelOutput -> COMMON_PIXEL_SHADER Output

Proposed:

Always use the default output values unless a socket has a node attached, so COMMON_PIXEL_SHADER Output is enough.

Rationale:

Simplify shader nodes setup.

[Lonely-Goblin]

Could the math menus be simplified with a math node? Similar to the math node in cycles with the various math functions in a drop down? I find the current version with every math operation listed in menu to be too much especially when first learning the software. It would shrink down the add node menu and simplify it more.

[pragma37]

The way I see it, it would just move the same operations menu to another place.
It wouldn't be really more simple or more complex, just different.

Anyway, IMO the best way to do math in Malt nodes is using the Inline Code node.

[Lonely-Goblin]

I'll have to disagree, a math node would simplify the add node menu, which I think is more important for the overall experience of creating nodes. Inline code for math is interesting but I think that relies too much on the individual knowing what they want to use ahead of time. Whereas having a built in list allows for switching and experimentation. Not that I disagree with putting in a inline code node for those who want to do more advanced work though.

[HSDSZ]

This makes malt node logic flow more similar to the default material node in blender.
A wonderful update

[pragma37]

Improve the shader API and add default values for node sockets

Now that it's possible to set default values and optional sockets to nodes using META() macros in GLSL, it's a good moment to make a clean-up and improvement pass to the whole API.