Clarify the scoping nature of fields

[leonerd]

The RFC as it stands does not give enough clarity on the scoping nature of field names. In particular, all of the examples simply use unique field ... names at the level of the class block itself, so they don't sufficiently explain various cornercases.

Lets for now entirely ignore the generated method names for :reader or :writer attributes, or the constructor behaviour of a :param, and think purely about the field as an internal storage mechanism.

I can imagine any of several different models.

1. my-like

Since already we're saying that fields are private within a given class; that methods even in a directly derived subclass cannot see them, I wonder if they are lexically scoped within the block that declares them. I could see a model in which they scope the same as my variables. Compare the analogy of:

package Red {
  { my $x = 1; sub one { return $x; } }
  { my $x = 2; sub two { return $x; } }
}

class Blue {
  { field $f { 1 }  method one { return $f } }
  { field $f { 2 }  method two { return $f } }
}

This has a notable upside, in that if an individual method wanted to do some sort of per-instance memoization, it can hide a field variable in its own little block to act as a storage cache; in a similar way to the way regular subs can:

class Example {
  {
    field $cache;
    method calculate_thing {
      return $cache //= do { expensive code goes here };
    }
  }
  # other code and methods here can't see $cache
}

That variable is now hidden from all the other methods, and name clashes don't matter.

I like this model because it means that these fields (names beginning with $ or other sigils) scope the same way as lexical variables. Similar looking things behave similarly.

2. method-like

Alternatively, I could imagine that the set of field names declared on a class is visible once throughout the class. By analogy to how package subs are callable even between physically-separate parts of the same package, I could imagine that fields are similarly visible:

package Red { sub func { ... } }
# now we're back in main package

package Red { func() } # calls the function defined earlier

class Blue { field $f; }

class Blue { method m { $f++ } }

In this arrangement, it doesn't matter what block-level scope a field is declared in. Once created for a given class it now exists, just once, in any code that's part of that class.

I don't like this model because it makes these fields scope the same as methods, even though they look superficially like lexicals. It's a subtle and complex model to try to explain to people.

3. Some other model?

There may be some other ideas, but so far I can't think of another good one.

[leonerd]

Given the questions above, there are now a few ideas for test cases. In each case, what should the correct behaviour be? Should it be a compiletime failure? If not and it does at least compile, what would the correct result be?

Duplicated fields in the same block

class Test1 {
   field $x;
   field $x;
}
ok(Test1->new);

Duplicated fields in their own block

class Test2 {
  { field $y; method one { $y++; return $y; } }
  { field $y; method two { $y++; return $y; } }
}
my $obj = Test2->new;
$obj->one;
is($obj->two, 1, 'Each method had its own private $y');

Class code split across two locations

class Test3 {
  field $z;
  ADJUST { $z = "value"; }
}

class Test3 {
  method z { return $z; }
}
is(Test3->new->z, "value", 'Field $z visible across scopes');

Given the competing concerns, they can't all pass - the model implied by test 2 passing means that test 3 must fail and vice versa.

So what do we prefer?

[leonerd]

My personal feeling is that we should use model 1 (my-like). This would make Test2 pass as is written. Test1 should probably compile OK but yield a warning, something about "duplicate declaration of field at same scope", similar to the warning you get from { my $x; my $x; }. This means Test3 would be a compiletime error, because $z is not visible in the second class block.

[Ovid]

I much prefer the my-like solution. I would probably write it like this:

class Example {
    method calculate_thing {
       field $cache;
       return $cache //= do { expensive code goes here };
    }
    # other code and methods here can't see $cache
}

But use an outer block if more than one method needs it:

class Example {
    {
        field $cache;
        method calculate_thing {
            return $cache //= do { expensive code goes here };
        }
        method needs_the_cache {
            my $foo = $cache->stuff;
        }
    }
    # other code and methods here can't see $cache
}

[Ovid]

For duplicated fields in the same block:

class Test1 {
   field $x;
   field $x;
}

At minimum, that should be a "redefined" warning, but I think it might be better if it's fatal. Much safer that way because it's easy to overlook/swallow warnings.

For duplicated fields in their own block:

class Test2 {
  { field $y; method one { $y++; return $y; } }
  { field $y; method two { $y++; return $y; } }
}

With your proposal for the my-like syntax, I think the above is quite reasonable, though again, I might prefer this:

class Test2 {
    method one { field $y; $y++; return $y; }
    method two { field $y; $y++; return $y; }
}

I think that gives us a powerful new mechanism for managing state, and even allowing it to not be exposed to other parts of the class which doesn't need it. Encapsulated encapsulation.

That being said, what does this mean?

method one { field $y :param; }
method two { field $y :param; }

To my mind, any field not declared in the outer scope should not have a :param, :reader or :writer attributes. In fact, maybe no attributes should be allowed (though the postfix default block should be, I think).

For class code split across two locations: definitely a compile-time error.

[Ovid]

Also, this is the full list of field attributes suggested in the RFC: https://github.com/Ovid/Cor/blob/master/rfc/attributes.md

I realize that not all will be implemented.

[Ovid]

I wrote this:

---

For duplicated fields in the same block:

class Test1 {
   field $x;
   field $x;
}

At minimum, that should be a "redefined" warning, but I think it might be better if it's fatal. Much safer that way because it's easy to overlook/swallow warnings.

---

Now that I think about it, it must be an error because a warning would be a disaster. For regular Perl, we have this:

my $x = ...
# lots of code here
my $x = ...

And that generates a warning, but it's usually harmless. In fact, you can usually just remove the my from the second declaration and you're good to go (though maybe you wanted a different variable name).

The reason this is harmless is because in procedural code, this is just a control flow issue. You've gone straight from the first to the second $x. In OO code, it's not procedural and we don't know the order in which the code is going to execute. Sure, we assume that the second field $x is going to overwrite the first, but let's look at this:

class Example {
    use Some::Other::Class;
    field $answer :param :reader {42};

    method frobnicate ($val) {
        return $val < $answer ? "frob" : "nicate";
    }

    # much later in the same class
    field $x { Some::Other::Class->new }

    method uh_oh ($thing) {
        if ( $x->handles($thing) ) {
            ...
        }
    }
}

What does that second $x do? It's clearly not semantically equivalent to the first. If it was scoped (e.g., my-like), we could wrap it in a block and make it work, but as it stands now, what does $instance->x return? Or do we still have a reader? Is it allowed in the constructor?

If we make this fatal instead of a warning and need to back it out later, no one will be relying on that behavior. However, if we make it a warning and not fatal, someone might rely on that behavior and if it turns out to be a serious design flaw, we might be stuck with it.

I recommend playing it safe and making that fatal.

[leonerd]

@Ovid

I much prefer the my-like solution. I would probably write it like this:
{code with field inside method

Yeah that makes sense. Once you can hide a field inside any scope, it feels natural to hide it even within a single method. It has parallels to state variables in that case.

To my mind, any field not declared in the outer scope should not have a :param, :reader or :writer attributes. In fact, maybe no attributes should be allowed (though the postfix default block should be, I think).

Certainly I think those ones shouldn't be permitted - :param, or any of the others that create accessor methods. Some third-party attributes would be fine though - :isa for example. I think it would need a case-by-case decision on whether they can be permitted.

[HaraldJoerg]

Allowing several fields in a class to have the same name like this feels ... itchy. With Object::Pad an object can have several fields with the same name - iff these fields come from different parent classes or roles. This is an enhancement compared to Moo*, and it makes some sense. It is still possible to describe an object's state in words if you think of the field name as its "given name" and the class/role holding its definition as its "surname".

What Perl's traditional OO (including Moo*, but not inside-out) has given us for free is an incredibly easy and robust way to serialize objects, and I am probably not the only one who uses that a lot. Part of it is already gone with Object::Pad: Data::Dumper can not round-trip Object::Pad objects. Even if it could be taught to serialize (that can be viewed as "describing an object's state"), de-serialization is no longer that simple.

You can, of course, go Meta with Object::Pad for serialization, and I have some hope this will be available for Corinna.

But can you? How do you call $metafield = $metaclass->get_field( $name ) if $name isn't unique?

Anecdotal evidence: I had to go Meta for serialization to be able to convert between different versions of an Object::Pad class. Storable sort of works sometimes, but fails in interesting ways if the order of field declarations in the class is changed.

[Ovid]

Certainly I think those ones shouldn't be permitted - :param, or any of the others that create accessor methods. Some third-party attributes would be fine though - :isa for example. I think it would need a case-by-case decision on whether they can be permitted.

I think the general rule would be: if a field attribute exposes the field outside of the class, it's only allowed in the top-level class scope. Thus, :param, :name, :reader, :writer, :predicate, and :handles would be disallowed. :common would also be disallowed because that shares the field across :classes. :isa would be fine, as would :lazy if there's any way that could be implemented.

[Ovid]

But can you? How do you call $metafield = $metaclass->get_field( $name ) if $name isn't unique?

That's a thorny question. At first blush, I would suggest that it only pulls top-level fields, but if you want to use the MOP to clone something, you're stuck unless we find a clean way to declare scope in the MOP and that's not good.

For now, I would suggest that duplicate names are forbidden until we hash this out. It's the simplest way to resolve the issue (though I can still see the scope issues).

class Blue { field $f; }

class Blue { method m { $f++ } }

But let's look at procedural code:

package Blue {
    my $f = 3;
    sub foo { say $f }
}
package Blue {
    sub bar { say $f }
}

That won't work and I'd suggest that, by analogy, it wouldn't work in Corinna, either. Lexical scope is lexical scope.

But yeah, this is a problem:

class Example {
    field $x {2};
    {
        field $y {2};
        method foo {
            # can access $x and $y
        }
        method bar {
            # can access $x and $y
        }
    }
    method baz {
        # can access $x, but not $y
    }
}

Thus, in the MOP, we'd need some way of saying which methods can access those fields. Either "all" (the common case) or a list of methods, identified by signature (because eventually we're likely to get multisubs and (hope, hope, hope), return values can be included in the signatures).

[HaraldJoerg]

For now, I would suggest that duplicate names are forbidden until we hash this out.

I agree (even if you drop "For now"). The benefits of using duplicate names are too small.

Thus, in the MOP, we'd need some way of saying which methods can access those fields.

And then we might want lexical methods so that method names are no longer unique. I think the chances that this gets into the way is bigger than the chances that it actually helps to implement interesting things.

I also would forbid more than one class block with the same name. A package is just a namespace, so there's nothing Perl can do about it, but IMHO a class should be "sealed" when the declaring block is closed (unless Meta wizardry is applied). In that case there are no scoping issues with fields.

[Ovid]

I also would forbid more than one class block with the same name. A package is just a namespace, so there's nothing Perl can do about it ...

For now. In the future, it's entirely possible (even preferable) we can disentangle classes and packages. @leonerd Thoughts?

[haarg]

I think for now it's fine to make it an error to define a field with a given name multiple times per class. But to make sure everything else in the implementation would allow it, so we don't paint ourselves in a corner. Things like anonymous classes or roles may be desirable in the future, which could easily make it impossible to have a unique 'name' for a given field.

[aquanight]

Might I toss in a 3rd possible way to look at field declarations?

3. our-like

package Red {
    { our $x; sub one { return $x; } }
    { our $x; sub two { return $x; } }
    eval q{ sub three { return $x; } }; # Global symbol '$x' requires explicit package name ($x not in scope here)
    # these are the same $x (specifically $Red::x
}

class Blue {
    { field $x; sub one { return $x; } }
    { field $x; sub two { return $x; } }
    # These are the same $x (the field '$x' within Blue objects) but $x is only visible within the block it appears
   eval q{ sub three { return $x; } };  # Global symbol '$x' requires explicit package name ($x not in scope here)
}

package Yellow;
our $x;

class Blue;
field $y; # Creates the field $y in Blue and links the lexical $y to it. Note that $y has file visibility.

class Other;
sub bark {
    say $x; # $Yellow::x
    eval q{ say $y; }; # '$y' is unavailable (in-scope lexical attempting to refer to a field of another class)
}

class Blue;
sub four { return $y; } # Legal again since we are back in the bounds of class Blue.

Where you have multiple 'fields' in the same class, they're really all the same field just available in different lexical scopes. If none of them have attributes, there's no problem. However, attributes in general should probably only appear on at most one definition but there's not any obvious reason it needs to be any specific definition other than to be able to distinguish "I meant for there to be no attributes" from "I just didn't put attributes here".