immutable for libraries / dynamic libraries / high-level delegatecall

[chriseth]

I'm pretty sure there are issues about this scattered across the project, but this has come up recently, so I'll create a new one.

Since linking is a bit cumbersome, it has been brought up that libraries should be explicit immutable state variables, so that linking happens at construction time instead of before deployment. If we also open this up to non-immutable state variables for libraries, it will create a high-level interface for delegatecall and proxies, even though the library interface cannot be changed after compilation (i.e. for a truly generic proxy, we would need .delegatecall anyway).

Pros:

• easier recompilation (no issue about using the compiler to link or substituting manually)

Cons:

• it is more complicated for off-chain tools to get the library address (it is not directly in the metadata file, but has to be extracted from the constructor)
• "using for" does not work

[axic]

it is more complicated for off-chain tools to get the library address (it is not directly in the metadata file, but has to be extracted from the constructor)

Because of this property, I think this feature is rather about having "dynamically linked" (=deploy time) libraries, as opposed to "statically linked" (= compile time).

[nventuro]

I'm not sure the pros list highlights the radical impact that implementing this issue would have. It would let developers fully do away with the notion of linked libraries, which means being able to drop linking tooling, not worry about bogus creation code (pre-replacement), the strange library replacement metadata, more easily expose libraries in use, among others.

With immutable, we can now have a drop-in replacement that still provides all linked library benefits (notably, reducing runtime bytecode size, which is becoming a huge issue in all non-trivial projects), while avoiding all of the weirdness that comes with linking. Libraries can finally be just another dependency passed as a constructor argument, and all contracts would be 'ready' after compilation, making linking a thing of the past.

Seriously, I recently spent a considerable amount of time puzzling over whether it made sense to rely on assembly-level delegatecall rather than use linked libraries due to the sheer number of new edge cases those would force us to cover.

---

And that's saying nothing about mutable libraries, if we allowed for delegatecalls to be performed on any stack address, whether it came from an immutable variable, a storage one, or even an external call. There's plenty of use cases here that don't involve upgrades, such as allowing external entities to access a contract's storage and do whatever heavyweight processing is required in a view-like manner: this would be a fantastic feature for e.g. price oracles or any other data-storing contract.

But even if this was only for immutable, it would be an incredible win for the language and ecosystem.

it is more complicated for off-chain tools to get the library address (it is not directly in the metadata file, but has to be extracted from the constructor)

I disagree: this instead empowers the developer to provide better ways of querying this information (e.g. a simple public getter), instead of relying on an obscure piece of deta that is not often used for any other purpose.

[fvictorio]

As @nventuro said, the biggest pro here is simplicity. Completely removing this concept of linking seems like a huge win both for tools and for users learning the language.

[chriseth]

I think to avoid confusion, calling non-public (i.e. internal) functions on libraries via a library variable should be disallowed, even if it's immutable. You can still call it via the name of the library, though.

[fvictorio]

@chriseth Does it make sense to keep that behavior of libraries now that free functions exist? More conceptual simplification of libraries would be nice. (I realize this is off-topic for this particular issue; I can create a new one if this idea is not completely off.)

[chriseth]

@chriseth Does it make sense to keep that behavior of libraries now that free functions exist? More conceptual simplification of libraries would be nice. (I realize this is off-topic for this particular issue; I can create a new one if this idea is not completely off.)

We still need #9211 in order to be able to phase out the call behaviour of libraries, but in general I would agree.

[hrkrshnn]

As I understand this, libraries are also deployed during construction time. Is that correct? In that case, for users who rely on libraries to get around contract-size limit would have problems, right?

[chriseth]

The constructor does not automatically deploy any libraries.

[hrkrshnn]

Does this mean that the library address needs to be passed as an argument during the constructor?

[viraj124]

on first thought as a solidity dev I believe for a contract using multiple libraries would need to have all those libraries deployed first before deploying the contract right might make deployment cumbersome would be good to know more details

[chriseth]

Maybe to clarify: This is just about libraries with public functions, internal functions are a whole different thing. Calling functions on the libraries discussed here will use delegatecall, and not a direct jump.

[hrkrshnn]

During the call, we briefly mentioned the idea of a library interface. The difference being that calls would be delegatecall. I think this might be another approach for this, and can be used as a replacement for some proxy designs that rely on low-level delegate calls.

Syntax could look like:

interface library L {
    function f() external;
    function g() external;
}

Now L can be treated similarly to how regular interfaces can be treated:

1. they can be stored both as immutable state variable as well as regular state variable (to get upgradability).
2. new L() isn't well-defined and cannot type check.
3. L(<address>).f() would be a delegate call instead of call or staticcall.

[cameel]

Do we also want to also document the calling convention for stuff that's not available in contract external functions? E.g. mappings or storage arguments?

We also discussed new L() (in case where it's a library rather than interface). I'm not really sure if that has a good use case. I mean, why would someone deploy multiple copies of a library if it cannot have any state? And if that's just a single copy, then isn't it strictly better to just deploy it rather than deploy a contract that contains its bytecode and can deploy it?

BTW, for anyone who wants to brainstorm this, there's also a topic on the forum: Dynamic linking for Libraries.