Why are Maps "write-only" operations failable?

[sprintermax]

Currently in Verse, writting set MyMap[MyKey] = Value is a failable operation. But, it is actually impossible to fail, causing confusion on some users, and requiring "workarounds" to ignore that "impossible failure", as shown bellow:

A "pure" set/override only map operation (with =) can't fail, since if the Key does not exist, it will be created or inserted, and, if the key already exists, it will just override it. There is no cases where it can ever fail. Besides this, the LSP/Compiler still marks these operations as failable, requiring us to do workarounds such as:

(set MyMap[MyKey] = Value) or Err("Unreachable")

# or

option{set MyMap[MyKey] = Value]}

This makes extra annoying to write code due to always needing to create these (unnecessary) "catch" syntax to ignore the impossible failure.

---

The only cases where a map set can fail, is on other operations, such as: +=, -=, /=, *=. This is due to these operations, before actually setting the value, it first try to reads the value at that position, to just then do the increment/operation and setting it back, example:

set MyMap[MyKey] += 10

This is correctly failable, because if MyKey does not exist, it can't read from it to increment 10 to store the operation result.

---

The Idea/Proposal:

• The Map operations such as +=, -=, /=, *= should keep being failable, since it correctly describes their behaviors.
• The Map operation =, which is a pure and unfailable set/override, should not be marked as failable, since is impossible to fail.

Example signature would be:

# Note no <decides>
operator'set ='(MyMap:[t]t2 where t:comparable, t2:any)<transacts>:[t]t2 = { ... }

[sprintermax]

Extra note:
This is only talking about Maps. Other types such as Arrays continue the same behavior, since a = operation are still failable on them (Can't insert at inexisting index).

I also am not sure if the failure is attached to the = operator, or on the [] characters of these cases. If that is the case, it would be a limiting factor about this proposed QoL change...

[janvitek]

Here is what AndrewS wrote on our internal slack regarding this (someone else asked there):

It's a long-standing wart that the compiler thinks the expression as a whole can fail. It should be infallible. What we want to do in the future is to allow set StringToIntMap["a"] = MaybeInt?, and if MaybeInt? fails it will delete the StringToIntMap["a"] key from the map. But the overall expression would still succeed.

[sprintermax]

What about splitting the left and right parts of the expression?
Some examples can be:

# Failable during unwrapping the MaybeInt, and progagate the failure:
set StringToIntMap["a"] = MaybeInt?

# Unfailable, will always set (right side has a `or` to catch the failure - both are the same):
set StringToIntMap["a"] = MaybeInt? or 0
set StringToIntMap["a"] = (MaybeInt? or 0)

# Failable during the unwrapping, the set expression fails then `or` catches it. Does not propagate
(set StringToIntMap["a"] = MaybeInt?) or void

We already had failure on the right side of map set in the past, but it was removed later after some update, I am unsure why that happened, would love to know more about if possible... That change made us needing to write more workaround syntax that were not needed before.

In my mind, the pure map assignments such as = should evaluate as right-to-left (no need to read, just directly write to it).
Only read-assignment operations such as +=, then is ok to evaluate left-to-right (read -> increment -> write)

[janvitek]

Seems fine with me.

[sprintermax]

Update on this:

@KnightBreaker commented out a important topic with me:

Currently in Verse, set operations of multiple nested maps may fail if any of the accessors on the chain is non existent. Example:

var MyMap : [int][int][int]int = map:
    4 => map:
        8 => map{}

# Fail: "4" exists, but "6" does not, so can't set/insert key "1" inside "6" with value "50"
set MyMap[4][6][1] = 50

# Succeed: "4" exists, "6" exists, then set/insert key "1" inside "6" with value "50"
set MyMap[4][8][1] = 50

This can be a conflicting point about the "unfailable map set" idea, however, there is some things to also note:

In Verse, nested map set operations mutates the entire outer (MyMap) by making copies of all inner maps with their current and/or updated values as needed.
With that into count, a simple fix would be, since is a set operation, and on that operation we are already providing the keys for all the nested levels, it could safely create the missing ones, for example taking the same example as before:

set MyMap[4][6][1] = 50

# "4" exists, but "6" does not exist (then create/insert it), then also set/create key "1" inside that "6" with value "50"
# After the operation, the map would become:
map:
    4 => map:
        8 => map{}
        6 => map:
            1 => 50
# (set operation set all keys across the nested path inserting them if not existing, being unfailable)

This would only work due to us providing all the keys on that set = operation, which still would mean "override or insert if not exist yet".
Other operations such as set += would still be failable since it still requires reading the previous value first.

Note that this is a opinionalized though about how the language could behave, and not a definitive rule - Currently different languages already choose different reasons and interpretations for this same usage.

---

Another case to consider is with interfaces or classes, since they get passed as references and not copies:

my_class := class:
    var MyVar : int = 10

MapA : [string]my_class = map{}
var MapB : [string]my_class = map{}

# Example 1:
# MapA is not mutable, but the class field is mutable, this may fail or succed if the "apple" key exists on the map:
set MapA["apple"].MyVar = 30

# Example 2:
# MapB itself is mutable, but even with that, the "set" here is not mutating the map, but instead "MyVar" inside the class;
# It also may fail or succeed depending if the key exists or not.
set MapB["apple"].MyVar = 30

# Example 3:
# Here we are mutating the MapB directly, inserting a class into it, this is unfailable. 
set MapB["orange"] = my_class{}

Then, for these cases, we could still handle the possible failure for nested classes, and "impossible to fail" cases:

Compiler and LSP can already detect if we are making a set operation on a nested map class (Examples 1 and 2), which the map operation is just a "read" where it can fail, or if we are making a set operation on the map itself (Example 3), which is just a "write" operation, without reading the current value, that is unfailable (if no value, it will be created).

This "knowledge" can be expanded to the interpreter to be smart about where and how the map access are being used, and with this, giving us more flexibility to not needing to catch false/wrong "impossible-to-fail" errors in the code.

[KnightBreaker]

Here are some ideas I have in my in regards of that topic.

Possible solutions:
A) The compiler provides set guarantees only on the deepest map value (e.g. set Map[0][1](2) = 3).
B) The intrinsic map type could be extended to provide a conditional non-failable subscript operations when the value type is known to be concrete.

To visually explain (B) we will perform a simple pseudo substituation to demonstrate the desired effect.

set Map[0][1][2] = 3 # type is `[int]([int]([int]int))`

The read or write access to a map's key-value pair is performed the syntactically expressed [$] operation. In reality it's no different from a type bound methods such as SubscriptGet or SubscriptSet.

This gives us the following structure:

set Map.SubscriptGet[0].SubscriptGet[1].SubscriptSet[2] = 3

So far we only changed the syntax from from being Map[0][1][2]. Notice how only the outer most (deepest) operations is actually SubscriptSet while the rest are SubscriptGet operations. For the sake of simplicity we will ignore concrete implementation details of things like (im)mutability here, because a subscript operation is often closely tied to read/write operations anyways. Furthermore we will shorten the above method to just Subscript.

set Map.Subscript[0].Subscript[1].Subscript[2] = 3

(B) proposes that there's an additional non-failable Subscript when the value type in [key]value is concrete. The hypothetical implementation of such a non failable operation would likely have a 'mutating get' path where when the value does not exist, it would create one on demand!

set Map.Subscript(0).Subscript(1).Subscript(2) = 3
        ^~~~~~~~~~~~ # creates empty `[int]([int]int)` on demand
                     ^~~~~~~~~~~~ # creates empty `[int]int` on demand
                                  ^~~~~~~~~~~~~~~~ # regular non-failable set operation as `value` is provided

---

In the following example I'm going to use Swift to fully demonstrate how this could work:

// protocol that mimics verse' `<concrete>` effect
protocol Concrete {
  init()
}

// `Int` is retroactively made concrete: works because it already provides `init()`.
extension Int: Concrete {}

// Just an error that we will throw when our operation fails.
struct MapError: Error {}

// `Map` type mimics the access of verse' `map` and it's also marked as concrete.
struct Map<Key, Value>: Concrete where Key: Hashable {
  // some inner storage
  var storage: [Key: Value] = [:]

  // Swift cannot express effect on the `subscript.set` accessor, therefore
  // we will split that into `set` and `mutate` methods.

  // Like in Verse, nothing really is failable, but we marked it anyways.
  mutating func `set`(key: Key, value: Value) throws {
    storage[key] = value
  }

  mutating func mutate(key: Key, _ deepMutate: (inout Value) throws -> Void) throws {
    if var value = storage[key] {
      try deepMutate(&value)
      storage[key] = value 
    } else {
      throw MapError()
    }
  }
}

// Conditional subscript operation as proposed by (B)
extension Map where Value: Concrete {
  // conditional set no longer is failable
  mutating func conditionalSet(key: Key, value: Value) {
    storage[key] = value
  }

  // conditionalMutate set no longer is failable
  // (it does re-throws, only if `deepMutate` throws)
  mutating func conditionalMutate<E: Error>(
    key: Key,
    _ deepMutate: (inout Value) throws(E) -> Void
  ) throws(E) {
    var value = storage[key] ?? Value()
    try deepMutate(&value)
    storage[key] = value 
  }

  // simpler version of the above:
  subscript (key: Key) -> Value {
    mutating get {
      if let value = storage[key] {
        return value
      } else {
        let value = Value() // init new concrete value
        storage[key] = value
        return value
      }
    }
    set {
      storage[key] = newValue
    }
  }
}

// Simulating `var Map: [int][int][int]int = map {}`
var map: Map<Int, Map<Int, Map<Int, Int>>> = Map()

print("performing failable example")
// Verse: `set Map[0][1][2] = 3`
do {
  try map.mutate(key: 0) { innerMap in 
    try innerMap.mutate(key: 1) { innerMap in 
      try innerMap.set(key: 2, value: 3)
    }
  }
} catch {
  print("failure") // executes
}

print("performing non-failable example with concrete")

// Since `Int` and `Map` are both concrete, we can use the
// conditional `*Mutate` and `*Set` methods now without any
// failable context.
map.conditionalMutate(key: 0) { innerMap in 
  innerMap.conditionalMutate(key: 1) { innerMap in 
    innerMap.conditionalSet(key: 2, value: 3)
  }
}

print(map[0][1][2] == 3) // prints true

// `subscript` in Swift uses `[$]` syntax.
// However we are using the above conditional `subscript` from 
// `extension Map where Value: Concrete`.
//
// this would be `set Map(0)(1)(2) = 4` in verse as proposed by (B)
map[0][1][2] = 4

print(map[0][1][2] == 4) // prints true

This demonstrates that when the compiler knows that Value is Concrete the chained subscript operation can perform a mutating get that would only instantiate and inject a new concrete value if it didn't existed before.

[sprintermax]

Implicitly creating new instances of types just because being concrete is not good, also goes agains the philosophy of verse that aims to "what you write is what it executes" without "extra steps"

my_class := class<concrete>:
    var MyVar : int = 10

var MyMap : [string]my_class = map{}

OtherMap : [string]my_class = map:
        "apple" => my_class{}


# fail or succeed depending if the key exists or not.
# no implicit creation of `my_class{}` ever happens, regardless of it being concrete or not
set MyMap["apple"].MyVar = 30

# The set mutation operation is not on the map but on the clas field, on this case the map is only read but never set
set OtherMap["apple"].MyVar = 30

[KnightBreaker]

The set OtherMap["apple"].MyVar = 30 is correct, but in my example above the side-effect of the absense of the concrete object would only be explicitly triggered via non-failable subscript overload set OtherMap("apple").MyVar = 30. If and only iff there was no object for the "apple" key, the concrete object would be created.

set OtherMap["apple"].MyVar = 30 # only sets `MaVar` no behavior changed
set OtherMap("apple").MyVar = 30 # IFF the object did not exist, create it, then set `MyVar`

This would be syntactically still distinct and it would also require documentation of that side-effect. I would not argue that it goes against something if it was a documented side-effect.

[sprintermax]

That seems fine logic wise, but idk, I still don't like much implicit class instancing and allocation (specially since they can chain reaction into other parts of the code due to block constructions and so on)

[KnightBreaker]

That's a fair point against it. However I personally do not account construction side effects as being a problem of the discussed feature. map { unique_class_with_side_effect {} => unique_class_with_side_effect {} } this is already valid code.

[KnightBreaker]

We can even demonstrate how Swift's Dictionary extension can have similar effects, however there's an issue with overloading the existing subscript as dictionary failure is silent and absence of value is denoted with an optional return type. Therefore we have to qualify using a key label for the sake of demonstration:

protocol Concrete {
  init()
}

extension Int: Concrete {}
extension Dictionary: Concrete {}
extension Array: Concrete {}

extension Dictionary where Value: Concrete {
  subscript (key key: Key) -> Value {
    mutating get {
      if let value = self[key] {
        return value
      } else {
        let value = Value()
        self[key] = value
        return value
      }
    }
    set {
      self[key] = newValue
    }
  }
}

// Verse: `var Map: [int][int][]int = map`
var map: [Int: [Int: [Int]]] = [:]

// Verse: `set Map[0][1] = array { 42 }`
map[0]?[1]? = [42] // fails silently in Swift

// Verse: `set Map[0][1] += array { -1 }`
map[0]?[1]? += [-1] // fails silently in Swift

// WITH PROPOSED (B) IDEA:

// Verse: `set Map(0)(1) += array { 1 }`
map[key: 0][key: 1] += [1]

// Verse: `set Map(0)(1) += array { 2 }`
map[key: 0][key: 1] += [2]

// Verse: `for (Element: Map(0)(1))`
for element in map[key: 0][key: 1] {
  print(element) // prints only 1 and 2
}

As you can see, it does enables very convenient use cases with other operators such as +=, -= etc. This approach is not limited to just =.

[KnightBreaker]

Noticed that for (Element: Map(0)(1)) would not work in verse as it cannot mutate on its own without set.