Guidelines for bringing back visual scripting to Godot in the future

[aaronfranke]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Godot 3.x had visual scripting, but Godot 4.0 removed this. This is ultimately a good thing, because it would be better to bring it back as a GDExtension in the future.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Provide an official visual scripting system. See below for details.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I am here to propose an 8-part plan to bring back visual scripting to Godot to ensure the system is broadly useful in lots of ways, including GLTF interoperability, extensibility, and of course, ease of use. These 8 parts are guiding principles as opposed to steps that can be taken one at a time.

1. Not a part of the engine

Visual scripting should be done with GDExtension. It should not be a part of the engine for several reasons:

• Bloat: Many users do not want visual scripting. Especially considering we may want many high-level blocks, it's better to not burden the main repo with all of that custom behavior.
• Extensibility: Of course we will want to allow writing custom script blocks in GDScript, but users may want to customize the visual scripting system's C++ code. This is much easier if it's done in GDExtension.
• Independent development: This will allow the visual scripting system to be developed and updated independently of the engine. This way users don't need to wait for newer Godot versions to get new visual scripting features.

2. Speed is not vital

Of course, we don't want to make the visual scripting system be purposefully slow, so we should write it in a language like C++ or possibly Rust. But we need to ensure the visual scripting system is primarily high code quality, not high performance.

Depending on the operations you do, GDScript is about 5 to 20 times slower than C++, or about 2 to 10 times slower than C#. Big Godot projects may have hundreds of GDScript files, some of them may have hundreds or thousands of lines. However, GDScript is still fast enough for most users. Visual scripting would be used with much simpler scripts than GDScript, so it doesn't need to be as fast as GDScript.

Knowing this, even if visual scripting was 100 times slower than C++, that is still plenty fast. We should primarily target code quality and easy extensibility over performance.

3. Allow extending in GDScript

Visual scripting needs to be easy to extend. On a 2-person team of 1 artist and 1 high-level GDScript programmer, the GDScript programmer may wish to provide APIs for visual scripting to use.

Therefore we need to ensure it's easy to extend. We need to allow extending visual script block classes with virtual callback methods. We should also ensure the internal code quality is high, to allow users to extend the C++ code if needed.

Still, allowing extending in GDScript should not come at the cost of the performance of built-in script blocks. Having to allocate a Godot Object for every script block to allow exposing it would be quite slow and memory-intensive at runtime. Instead, all of the built-in blocks should be use lightweight structs to store the data internally, and we have a more complex "extension" block class that uses Godot Object (or possibly RefCounted?) to allow extending in GDScript. This does not hurt code quality while being a big win for performance.

4. Provide entire Godot API

One of the big problems with VisualScript in Godot 3.x is that it was missing some critical functionality, like setting values in Arrays. All operations available in GDScript should also be available in visual scripting, although not always exactly the same (ex: since visual script blocks can have multiple outputs for item in array: visual script blocks can provide both the index and the item).

Aside from that, we should ensure that every API call available in Godot is accessible via the visual scripting system. Every method needs to be callable, every property needs to be accessible, every class needs to be instantiable, etc.

5. Provide high-level blocks

In order for visual scripting to be useful for users to write complex scripts without their canvas turning into a mess of blocks and wires, we need to provide high-level script blocks. These are concepts that may not even make sense to have in GDScript.

For example, let's say we want a high-level script block to get the material properties of an object. The material properties may be located on the MeshInstance3D node, or on the Mesh resource contained in it. In GDScript you would have to check if the MeshInstance3D node is valid, check if it has material properties, check if it has a Mesh resource, check if that has material properties, etc. All that is just to find the material, then you would need to check if it's a BaseMaterial3D and grab each of the properties you need from it, like albedo color, roughness, etc. The high-level visual script block would be much simpler: It takes in an object, and it has many output ports for each of the properties on the material, all the processing to get those outputs is internal to the high-level script block. This does not make much sense in GDScript because on the level of GDScript you really do care about the fine details like where the material is located and what type the object is. It also does not make much sense in GDScript because all of those output ports would simply not work, GDScript functions do not have multiple return values and even if they did that would severely clutter your script with unused variables.

6. Learn from existing successes

A visual scripting system would be incomplete without getting experienced Unreal Blueprints users to participate in its development. Tons of people have a great time using Unreal Blueprints, we need to ensure we have all of the features that Unreal users are currently using to make successful AAA games using Blueprints (that doesn't imply that visual scripting will enable AAA games of course, just that your choice of scripting language should not limit you).

Unreal is of course not the only engine with a visual scripting system, but it is the most popular. We can learn from the successes of visual scripting in other engines as well.

EDIT: I have just been made aware that Godot Orchestrator exists https://github.com/Vahera/godot-orchestrator so this will either be a great source of inspiration or a great base.

7. Interoperability with GLTF interactivity

The Khronos group is working on an engine-agnostic scripting system in GLTF. This is super cool. Imagine being able to export a script from Godot and import that into Unreal, or export a script from Unreal and import that into Godot. Or imagine using Godot as a "game asset SDK" where you can write a simple behavior script in Godot and that can be imported at runtime into a "metaverse" style game, such as V-Sekai or The Mirror, two platforms made in Godot, without the security and portability concerns of just sending a Godot scene file with GDScript in it.

I think it would be a great idea to design our future visual scripting system to be compatible with GLTF interactivity to the greatest extent possible. This includes both import and export.

Of course, the Godot visual scripting system will need to be a superset of GLTF interactivity, as there will be many concepts in Godot that can't be available in an engine-agnostic format, and we will probably need to write code to "compile down" high-level blocks into simpler ones. But still, we should be able to export most simple behavior scripts, or most features of complex scripts, or import scripts from GLTF into Godot. This is good enough to cover lots of use cases. For complex scripts being brought between engines, if 10% of the visual script blocks do not survive the trip, that still means 90% did survive, and then it's easier to manually fill in the gaps instead of starting from scratch.

To be clear, the below image has nothing to do with what the proposed UI will look like. This is just an abstract representation of the interactivity graph data.

Work-in-progress GLTF interactivity PR: KhronosGroup/glTF#2293

8. Timeline

There is not really any rush in completing this, and in fact, it may be better to wait for several reasons.

For the goal of interoperability with Khronos GLTF interactivity, it would be helpful to wait for the Khronos group to finish standardizing that so we have a good base standard to structure visual scripting around.

For making an editor, @Geometror has been refactoring the GraphEdit system over the past year or so. It would be helpful to wait to work on the visual scripting system until the GraphEdit refactor is done, so that we will not need to redo the editor code to use the new GraphEdit APIs. You can view some of his excellent work here: https://github.com/godotengine/godot/issues?q=author%3AGeometror+label%3Atopic%3Agui+

For actually implementing, there are still some bugs and limitations with GDExtension that would be helpful to have fixed first. I don't have any specific bugs immediately in mind that are blocking visual scripting (since I have not started this, I have not run into them yet) but I know that there are many bugs and we will run into them at some point.

There are still other areas that are a much higher priority for engine contributors to work on. Of course since this will be done in GDExtension, the development cycle is not tied to the engine, but this will very likely be done by existing engine contributors (as evidenced by few people stepping up to maintain VisualScript in Godot 3.x). I would rather the engine contributors spend their time for now on important bug fixes than chase after a new shiny thing. I am personally interested in working on this, and I have lots of experience with developing a visual scripting system thanks to my time at The Mirror, but it is not a high priority for me.

I would expect that this visual scripting system could be completed sometime in the late 2020s (remember it is already 2024), but it is hard to say if that will be 2026 or 2029. Regardless I want to ensure we take the time do this right, and I have no plans to start working on this at all in 2024 due to all of the above reasons.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

This should be an add-on in the asset library! An officially endorsed one.

[synalice]

While I do think 5. Provide high-level blocks is a good idea whose absence would make using visual scripting unbearable, there absolutely must be a way to convert a visual script into a .gd file. The same way as we have the ability to convert visual shaders or materials into shader code.

Ensuring this will automatically solve 7. Interoperability with GLTF interactivity since you'd only need to support GDScript -> GLTF conversion (assuming I correctly understood what this GLTF thing is).

[aaronfranke]

@synalice GDScript -> GLTF conversion would be vastly worse than visual script -> GLTF conversion. So, visual script -> GDScript absolutely does not help with that. But it's a good idea for its own sake (when the desired end result is GDScript anyway).

[octanejohn]

@passivestar maybe related to your idea for universal geo nodes see gltf section, i know this is visual scripting but its simular

[AThousandShips]

One consideration would be to use the proposed IR directly instead of translating into GDScript

• Use an intermediate representation format for GDScript #8605

[passivestar]

Unreal is of course not the only engine with a visual scripting system, but it is the most popular. We can learn from the successes of visual scripting in other engines as well.

I used both Blueprints and Unity UVS, and I urge whoever designs this to take a look at Unity. One of my biggest problems with Blueprints was that it was impossible to create custom nodes that have persistent state (not without recompiling the engine)

Some built-in nodes have state, like for example the "Flip Flop" node that has a counter inside so that it can execute another branch next time it has control, but that's it. Being able to easily create your own nodes with state allows you to describe more complex behaviours in much less nodes. For example I was able to create a "Transition" node that outputs a value that lags behind of its input, which is very useful for a lot of things

[aaronfranke]

@passivestar Great suggestion, but I see no reason to prevent making a visual script block with state, so you'll be able to.

[FR3NKD]

I believe I'm a perfect candidate for being a visual scripting user let's see how a visual solution can solve most coding problems for a beginner like me:

Problem 1: Knowing the syntax

Nodes are just a disaster for readability in my opinion and it's time to think about visual scripting differently so forget about nodes, visual scripting is created using vertical blocks that stack on top of each other and can be collapsed or expanded, basically just like code but without needing to know the syntax, this would allow you to easily follow coding tutorials as a side effect.

Problem 2: Arbitrary knowledge required to use the engine classes / General discoverability of API's functions

A big ADD button would suggest new context-relevant blocks when pressed, blocks should be separated by categories and have icons similar to Unity components.

Blocks should be directly translated into GDScript text, you should be able to follow a GDScript tutorial with this visual solution even if it uses normal code, this way all GDScript tutorials would still be relevant and users might be able to learn GDScript.

High level blocks should be an option and not the norm, I should be able to work 100% like in GDScript or use blocks already created for me which are more high level and simple to use.
Creating these custom blocks should be easy either via blocks or directly with GDScript.
I can imagine a complete novice using only high-level blocks created by others, or an advanced user who assembles a script quickly by simply adding high-level blocks created previously.

A sidebar should show all available blocks divided by category and a small section should recommend blocks relevant to the current context, blocks can be saved to favorites for quick access.

Problem 3: Good design patterns

Not being a coder I don't have a clear solution to this problem but this visual language could direct you to somehow use the right design patterns.

How to do it?

My original idea was to interpret and convert pure GDScript code into a user interface, giving an interface to the code like modern GUIs have given an interface to OSes.

This should have 0 performance impact compared to GDScript because this conversion does not happen at runtime, so the final output is just GDScript code.

In my deepest fantasies this language can also be used by some medium level programmers to be more efficient and type less.

The most important part of this idea

This idea could be technically infeasible but the most important part of this idea it's undeniable and is the synergy between GDScript and the visual language, if you have to keep only one thing from my vision, let it be this!
Users could use the GDScript documentation, watch existing tutorials, get help from their colleagues who know GDScript, learn the language without immediately having to write code, the benefits are endless.

[passivestar]

Nodes are just a disaster for readability

Not sure about that. When you have branching, nodes do a better job at visualizing the execution flow than text or stacked blocks

Also being able to see in real time what graph branches run and what values are getting passed is convenient for debugging because you can see at a glance if some value is off or if some branch doesn't get reached

uvs.mp4

Also it'll have to be nodes if we want godot visual scripting to be a superset of GLTF interactivity (and we really want that)

[FR3NKD]

This kind of visualization is admittedly pretty cool (and useful). I just think spaghetti are unreadable and it is very easy to create spaghetti with nodes.

[passivestar]

It's about as easy to create spaghetti code in GDScript. It's just less obvious that you've created it 🙂

I've made a lot of blender tools in both python and geonodes. Not making a mess with nodes is easier than it seems. All of my graphs looks like this no matter the complexity, I don't have long noodles anywhere:

[FR3NKD]

I use nodes out of necessity (for shading and stuff) but I have always found them intimidating and visual scripting should be as intuitive as possible.

I also never liked how you have to zoom out to see the whole graph but then you can't read the text of the nodes.

I don't know.. I don't love them, but this is just my personal preference.

[FR3NKD]

I would like to add that we had node based visual scripting in Godot and today we have Orchestrator but It doesn't seem to be that popular, why not try something different? I don't think nodes should be the default option necessarily

[aaronfranke]

@FR3NKD You are welcome to try something different by developing an addon like what you want yourself. Godot is extensible, you can plug in new scripting languages, for example Orchestrator does this.

[FR3NKD]

I'm not a programmer, that's why I'm trying to contribute to this with a non-programmer point of view.

When you create a tools you should put yourself in the shoes of the end user, this can be difficult if you are an expert in the field you ar trying to make accessible.

We should seek feedback from people who understands programming while not actually knowing how to code (they are rare 😅).

I admit that my opinion about nodes vs blocks can be subjective (although I suspect many feel the same way) so you are welcome to ignore that but don't dismiss my feedback all together 😁