High-Level Language Support for Transient Storage

[ekpyron]

Comprehensive high-level language support for transient storage decomposes into the following tasks:

Tasks

Preview Give feedback

1. transient is not a keyword, so we need a parser hack to introduce it non-breakingly as a first step: Transient storage parser support: accept transient as a data location or as an identifier #15006
2. Support for value types in transient storage: Support transient storage state variables in legacy codegen #15257, IR codegen for transient state variables #15292
3. Support for auxiliary compiler output artifacts like a transient storage layout, etc: Generate transient storage layout #15247
4. Support for reference types (structs, arrays) in transient storage; no copies between data locations.
5. Iterative support of copying.
6. Independently of reference types: support for mapping in transient storage.

Starting from the second step each step decomposes into a Yul and legacy part. We currently plan to start with Yul implementations and to merely backport functionality to legacy on demand.

Questions to be settled during the process:

• Layout of transient storage variables.
  • current proposal: storage and transient storage have independent layouts (both starting at zero and overlapping), otherwise the layout (including packing) is the same between transient storage and storage.
• Autoclearing, no autoclearing, optional autoclearing.
  • So far we assume that despite the connected dangers unconditional compiler-generated autoclearing is not expected and also not possible for all types. If we were to decide for opt-in or opt-out autoclearing behaviour, we would need to decide syntax for it. We're open to community input on the question.
  • Given the design of transient storage there is unfortunately no optimal choice here; current tendency is to expose the bare EVM functionality directly in the first instance; only consider optional opt-in autoclearing separately.

[Philogy]

IMO it would be better to focus on features that make the languages/libraries more powerful/ergonomic rather than enshrining another new first-class citizen in the language.

There's a transient storage library I wrote that I am using for work: https://github.com/Philogy/transient-goodies that is already relatively ergonomic to use, behaving like normal storage in many ways:

• being assigned unique slots in the contract's storage slot
• being composable into mappings and structs out-of-the-box

it allows the following usage which besides the .get, .inc overhead is pretty close to native storage IMO:

import {tuint256} from "transient-goodies/src/TransientPrimitives.sol";

contract MyContract {
    mapping(bytes32 => tuint256) internal txValidatedHashes;

    function validate(bytes32[] calldata hashes) external {
        for (uint256 i = 0; i < hashes.length; i++) {
            bytes32 hash = hashes[i];
            require(txValidatedHashes[hash].get() == 0, "Already validated");
            txValidatedHashes[hash].inc(1);
        }
    }
}

I wonder if implementing features that would allow library devs like me to make such libraries even more ergonomic or useful wouldn't be more powerful and achieve more than focusing on the specific case of transient storage itself.

Things like:

• allowing you to customize the behaviour of assignment of custom types/structs
• allowing you to define implicit type casting for custom types/structs
• allowing you to define the behavior of other operations -=, +=, *=, etc. for custom types / structs
• generics (type parameters & type classes aka interfaces)

[mehtavishwa30]

Hey @Philogy!

Thank you for detailing this out so well. As mentioned in my response to your quote-tweet here, we agree that implementing features like these into a standard library for library devs to use would be quite useful. And this is indeed what we are aiming to do with Experimental Solidity (you can watch our talk from Solidity Summit 2024 here). However, building a language with proper generics that's logically accurate and tested thoroughly is a non-trivial and time-consuming process and we're afraid that the community would not want to hold off on support for these features at the EVM-level.

That being said, we agree that your proposal is still proves to be the better approach in the longer term and so we did not start working on transient storage support until the community pushed for it.

In conclusion, even though the broader goal with Experimental Solidity is exactly what you proposed, we cannot ignore the immediate community demand and will have to focus on high-level language support for transient storage for the time-being. Would love to hear your further thoughts. :)

[Philogy]

@mehtavishwa30 I've seen your point and appreciate the note. The reason I brought it up again here is because I wanted to be more specific on what features exactly I think could be implemented short-term without a larger rework like full generics (such as being able to override assignment behavior, defining -=, +=, etc. for custom types, type casting).

Now I have no knowledge of the Solidity compiler's internals so I can't judge the complexity of implementing these over transient storage itself but from the description of solc devs that I've read saying that high-level transient storage support is non-trivial, requiring several deep changes it seems like this features might be equally if not less complex while providing more short-term benefit.

Yes, the community has requested high-level transient storage support and it's great to see how the team is incorporating this feedback but I think there are different ways to achieve this.