typeclass-based collections implementations

[armanbilge]

This is a recurring idea, most recently by @johnynek in #4118 (comment).

i.e. we should work towards having a typeclass based implementation of HashSet, SortedSet, HashMap, SortedMap that use only typeclasses, not the instance methods.

Although the issue comes up in Cats since this is where we house instances for Scala stdlib, this might be a better PR for cats-collections.

[djspiewak]

Definitely better for cats-collections. 🙂 This has been a dream for a while, but it's meaningfully hard to do correctly, and it's not clear to me that people would actually use them over the standard library.

[armanbilge]

it's meaningfully hard to do correctly

I thought somewhere someone suggested we could basically copy the implementations out of Scala itself. But I am very afraid of the collections hierarchy 😆

it's not clear to me that people would actually use them over the standard library.

yeah, fair ...

[rossabaker]

Precedent: scalaz.Maybe and scalaz.IList got used successfully where people didn't care about interop with stdlib, and were cursed mightily wherever people did.

[johnynek]

People definitely won't use what we don't have.

I have definitely sunk more time into code with zero users. I think it is worth it, even if it is niche. Someone could also learn a lot about HAMT by doing it, so it is a great learning experience.

One last note: cats-collections has two immutable heaps. The standard library has zero. No one generally uses them, but I have and I bet a few others. Safe immutable types are a big part of our mission. Let's not give into "no one will use this" defeatism. They may not, but that is not how we measure success.

[armanbilge]

A comment by @durban in typelevel/cats-effect#2464 reminded me of this issue:

What I find very strange is that it uses universal equality/hash for the keys, while there is a Hash typeclass in Cats.

• Incorporate MapRef into Std cats-effect#2464 (comment)

Understandably, it is difficult to provide even a Ref-based MapRef implementation using Hash when we are lacking a Map based on Hash!

So even if a Cats' Map is something "nobody" would use, MapRef certainly is.

[johnynek]

well, if you don't worry to much about performance, here is how:

class Hashed[K](implicit hash: Hash[K]) {

  class Key(val unwrap: K) {
    override val hashCode: Int = hash.hashOf(unwrap)
      that match {
        case hk: HashKey => hash.eqv(unwrap, hk.unwrap)
        case _ => false 
     }
  }

  def toMap[V](kvs: Iterable[K, V]): Map[Key, V] =
    kvs.iterator.map { case (k, v) => (new Key(k), v) }.toMap
}

val keySpace = new Hashed[Foo]

val myMap: Map[keySpace.Key, String] = keySpace.toMap(List(Foo(1) -> "1"))

etc....

so, in other words, you box with something that delegates hashCode and equals to Hash.

[johnynek]

I mean, we could add these methods and inner types to Hash[K] now... and have toMap and toSet and it might be pretty usable.

[DavidGregory084]

I think I am going to have a crack at this one unless someone has already taken it?

My plan is to do a pretty conservative port of the HashMap and HashSet from the Scala 2.13 collections initially.

e.g. for HashSet. something like this:

abstract class HashSet[A] {
  final def contains(value: A)(implicit hash: Hash[A]): Boolean

  final def add(value: A)(implicit hash: Hash[A]): HashSet[A] 

  final def remove(value: A)(implicit hash: Hash[A]): HashSet[A] 

  // etc.
}

Essentially, the interface of the existing Scala collections but with the Hash constraint . :)

I have not really looked into any kind of caching of hash code values or the many optimizations that have been added to the 2.13.x collections since the initial CHAMP port as I think it's probably best to do the simplest possible implementation to begin with.

[armanbilge]

Hmm, shouldn't the Hash be part of the constructor, instead of passed to each method? That also creates potential for inconsistency between calls.

[DavidGregory084]

I am not really sure - I have seen both approaches and I guess it's just as surprising both ways if the Hash that is used locally and the one that was used when constructing the Set are not consistent - both will produce strange results.
In general though it seems that we use method-level constraints in cats, see e.g. Chain's == and contains.

[djspiewak]

In general, passing it to each method is superior because it allows call sites to more correctly reflect their constraints. For example, imagine the constructor of MapRef[F, K, V]. You shouldn't need a Hash[K] to construct a MapRef, only to write key/value pairs. This makes the type signatures of both the functions using the constructor and the functions using the accessors much more parametric in terms of the constraints they declare, which in turn makes it easier to infer their semantics.

[armanbilge]

Ah, ok, thanks both. That makes sense, and I see that cats-collections also uses method-level constraints. 👍