From 4e1eb08cdc1b3a04437cf438650fe1f5b38deabb Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Wed, 15 Sep 2021 20:53:02 +0300 Subject: [PATCH 01/12] Add explicit backing fields proposal --- proposals/explicit-backing-fields.md | 273 +++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 proposals/explicit-backing-fields.md diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md new file mode 100644 index 000000000..7855f1744 --- /dev/null +++ b/proposals/explicit-backing-fields.md @@ -0,0 +1,273 @@ +# Explicit Backing Fields + +- **Type**: Design Proposal +- **Author**: ? +- **Contributors**: Svetlana Isakova, Kirill Rakhman, Dmitry Petrov, Roman Elizarov, Ben Leggiero, Matej Drobnič, Mikhail Glukhikh, Nikolay Lunyak +- **Status**: Implemented in FIR +- **Prototype**: Implemented +- **Initial YouTrack Issue**: [KT-14663](https://youtrack.jetbrains.com/issue/KT-14663) +- **Initial Proposal**: [private_public_property_types#122](https://github.com/Kotlin/KEEP/pull/122) + +## Summary + +**Note**: initial proposal contents have been partially copied down here for convenience. Despite the different approach, the already shown use cases are still relevant. + +> Common pattern in java is having full type accessible as private property and then only exposing required interface in the public getter: +> +> ```java +> class MyClass { +> // Use full type for private access +> private final ArrayList data = new ArrayList<>(); +> +> // Only expose what is needed in public getter +> public List getData() { +> return data; +> } +> } +> ``` +> +> This pattern allows easy hiding of implementation details and allows for easy clean external interfaces to the classes. But there is no idiomatic Kotlin way to achieve this pattern. Best thing you can do is define two separate properties to mimic how Java does it: +> +> ```kotlin +> class MyClass { +> private val _data = ArrayList() +> val data: List +> get() = _data +> } +> ``` + +With the proposed syntax in mind, the above code snipped could be rewritten as follows: + +```kotlin +class MyClass { + val data: List + field = ArrayList() +} +``` + +## Use Cases + +### Motivation + +> - There is no clean idiomatic way to do this pattern in Kotlin +> - Current best approach throws away all benefits of Kotlin properties and forces developer to write Java-like code with separate private field and public getter +> - Current best approach forces developer to assign two different names to single property (or pad private property, for example adding `_` prefix and then using this prefix everywhere in code) +> - This is another place where Java pattern could be introduced into Kotlin with less boilerplate + +### Read-Only from Outside + +> We often do not want our data structures to be modified from outside. Unlike Java, this can be easily achieved in Kotlin by just exposing read-only `List`. But as already ilustrated in above example, exposing different type of a property is a bit messy. + +```kotlin +private val _items = mutableListOf() +val item : List by _items +``` + +And the new syntax allows us to write: + +```kotlin +val items: List + private field = mutableListOf() +``` + +### Android Architecture Components + +> Proper way to do Architecture components is to use `MutableLiveData` (`LiveData` implementation that allows caller to change its value) privately inside View Model classes and then only expose read-only `LiveData` objects outside. + +Sample code [from an Android app](https://github.com/elpassion/crweather/blob/9c3e3cb803b7e4fffbb010ff085ac56645c9774d/app/src/main/java/com/elpassion/crweather/MainModel.kt#L14): + +```kotlin +private val mutableCity = MutableLiveData().apply { value = "" } +private val mutableCharts = MutableLiveData>().apply { value = emptyList() } +private val mutableLoading = MutableLiveData().apply { value = false } +private val mutableMessage = MutableLiveData() + +val city: LiveData = mutableCity +val charts: LiveData> = mutableCharts +val loading: LiveData = mutableLoading +val message: LiveData = mutableMessage +``` + +Becomes: + +```kotlin +val city: LiveData + field = MutableLiveData().apply { value = "" } +val charts: LiveData> + field = MutableLiveData().apply { value = emptyList() } +val loading: LiveData + field = MutableLiveData().apply { value = false } +val message: LiveData + field = MutableLiveData() +``` + +### RX Observable and Subjects + +> Data can often be pushed into reactive world using subjects. However, exposing `Subject` would allow consumer of the class to push its own data into it. That is why it is good idea to expose all subjects as read-only `Observable`: + +```kotlin +class MyClass { + private val _dataStream = PublishSubject.create() + val dataStream: Observable + get() = _dataStream +} +``` + +Turns into: + +```kotlin +class MyClass { + val dataStream: Observable + field = PublishSubject.create() +} +``` + +## Design + +The proposed design consists of two new ideas. + +### Explicit Backing Fields + +```kotlin +val it: A + [visibility] field[: B] = initializer +``` + +The above `field` declaration is referred to as an _explicit backing field declaration_. + +#### Accessors + +- if `A :> B`, the compiler can provide a default getter +- if `A <: B`, the compiler can provide a default setter +- If an accessor is required, it must be provided either by the compiler or the user + +```kotlin +public val flow: SharedFlow + field: MutableSharedFlow? = null + get() { + return field ?: run { ... } + } +``` + +#### Visibility + +Only the `private` and the `internal` visibilities are allowed for explicit backing fields now. The default visibility is `private`. + +```kotlin +val mutableWithinModule: List + internal field = mutableListOf() +``` + +Right now, there is no special syntax to explicitly access the backing field from outside the property accessors, but we will see how it can be accessed implicitly via the ["second idea"](#smart-type-narrowing). + +#### Lateinit + +If a property has an explicit backing field declaration, and it needs to be `lateinit`, the modifier must be placed at the `field` declaration. + +```kotlin +var someStrangeExample: Int + lateinit field: String + get() = field.length + set(value) { + field = value.toString() + } +``` + +#### Initializer + +If there is an explicit backing field, it must have an initializer, and the property must not declare an initializer. + +#### Forbidden + +Explicit backing fields are not allowed inside interfaces or abstract properties. + +Explicit backing fields are not allowed for delegated properties. + +### Smart Type Narrowing + +#### Rules + +If: + +1. the compiler can guarantee the property getter returns the same instance as the one stored in the backing field +2. the type of that instance is compatible with the property type +3. the backing field visibility allows a hypothetical direct access +4. there is no ambiguity in such an access + +it can then narrow down the returned instance type at the call site without loss of functionality. This is what is meant by the words _smart type narrowing_. + +The formal checks corresponding to the above rules are: + +1. the property does not have a custom getter, only the default one +2. `A :> B` +3. accessing: + 1. a private backing field of a class member property within the class + 2. a private backing field of a top-level property within the file + 3. an internal backing field within the module +4. property must be final, otherwise an overridden version would be able to provide a custom getter that may return a different instance + +#### Example + +```kotlin +class MyClass { + val items: List + field = mutableListOf("a", "b") + + fun registerItem(item: String) { + items.add(item) // Viewed as MutableList + } +} + +fun test() { + val it = MyClass() + it.items // Viewed as a List +} +``` + +## Alternatives + +### Initial Proposal + +Initially, the following syntax was suggested: + +```kotlin +public val items = mutableListOf() + private get(): List +``` + +> In above example `items` is `MutableList` when accessed privately inside class and read-only `List` when accessed from outside class. + +In fact, the above syntax brings in an incorrect mental model: it says _'There is a private property `it` with some *part* that declares its public behavior'_. + +Attempt to add support for the above syntax led to multiple redundant complications (see the [problems section](https://github.com/matejdro/KEEP/blob/private_public_property_types/proposals/private_public_property_types.md#questions-to-consider) + unclear override mechanics). + +## Future Enhancements + +### Direct Backing Field Access + +It might be handy to allow the direct access to the property backing field via syntax like `myProperty#field`. + +### Allow `protected field` + +This way, smart type narrowing would become possible inside subclasses as well. + +### Use of `field` Before Property Type Is Known + +Right now, any attempt to use `field` inside a property accessor in cases when the property type is not yet known (depends on the getter return value) will result into `UNRESOLVED_REFERENCE`. + +Since we now can have explicit backing field declarations, we may use them to find out the type for backing fields: + +```kotlin +var thing + field = SomeThing() + get() { + return convertSomehow(field) + } + set(value) { + field = convertTheOtherWayAround(value) + } +``` + +### Custom Getters + +Hypothetically, we could perform more analysis to find out whether smart type narrowing is possible for the given getter. \ No newline at end of file From 0c661ca31d32e567ba7f43abab94bec3de4aab45 Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Thu, 16 Sep 2021 11:15:03 +0300 Subject: [PATCH 02/12] Mention backing fields mutability --- proposals/explicit-backing-fields.md | 48 ++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 7855f1744..702f73fa5 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -36,7 +36,7 @@ > } > ``` -With the proposed syntax in mind, the above code snipped could be rewritten as follows: +With the proposed syntax in mind, the above code snippet could be rewritten as follows: ```kotlin class MyClass { @@ -177,6 +177,10 @@ var someStrangeExample: Int If there is an explicit backing field, it must have an initializer, and the property must not declare an initializer. +#### Mutability + +For now, we assume `val` properties have immutable backing fields, and `var` properties have mutable ones. + #### Forbidden Explicit backing fields are not allowed inside interfaces or abstract properties. @@ -270,4 +274,44 @@ var thing ### Custom Getters -Hypothetically, we could perform more analysis to find out whether smart type narrowing is possible for the given getter. \ No newline at end of file +Hypothetically, we could perform more analysis to find out whether smart type narrowing is possible for the given getter. + +### Mutability + +Letting a `val` property have a mutable backing field may be useful. Consider the [following example](https://github.com/JetBrains/kotlin/blob/master/libraries/stdlib/jvm/src/kotlin/util/LazyJVM.kt#L57): + +```kotlin +private class SynchronizedLazyImpl( + initializer: () -> T, + lock: Any? = null +) : Lazy, Serializable { + private var initializer: (() -> T)? = initializer + @Volatile private var _value: Any? = UNINITIALIZED_VALUE + // final field is required to enable safe publication of constructed instance + private val lock = lock ?: this + + override val value: T + get() { + val _v1 = _value + if (_v1 !== UNINITIALIZED_VALUE) { + @Suppress("UNCHECKED_CAST") + return _v1 as T + } + + return synchronized(lock) { + val _v2 = _value + if (_v2 !== UNINITIALIZED_VALUE) { + @Suppress("UNCHECKED_CAST") (_v2 as T) + } else { + val typedValue = initializer!!() + _value = typedValue + initializer = null + typedValue + } + } + } + + ... +} +``` + From c0ebff1c1ce92ce87237c480c3cd3c73f3b88e26 Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Thu, 16 Sep 2021 19:53:53 +0300 Subject: [PATCH 03/12] Paraphrase things --- proposals/explicit-backing-fields.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 702f73fa5..c53c9880c 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -129,17 +129,20 @@ The proposed design consists of two new ideas. ### Explicit Backing Fields ```kotlin -val it: A - [visibility] field[: B] = initializer +val it: P + [visibility] field[: F] = initializer ``` The above `field` declaration is referred to as an _explicit backing field declaration_. +Explicit backing fields is a FIR-only feature. + #### Accessors -- if `A :> B`, the compiler can provide a default getter -- if `A <: B`, the compiler can provide a default setter -- If an accessor is required, it must be provided either by the compiler or the user +- if `P :> F`, the compiler can provide a default getter +- if `P <: F`, the compiler can provide a default setter + +If the compiler can not provide a getter, the user must declare it explicitly. The same applies to setters in case of `var` properties. ```kotlin public val flow: SharedFlow @@ -175,7 +178,9 @@ var someStrangeExample: Int #### Initializer -If there is an explicit backing field, it must have an initializer, and the property must not declare an initializer. +If there is an explicit backing field, the property must not declare an initializer. + +If the explicit backing field is not `lateinit`, it must have an initializer. For `lateinit` properties initializers are forbidden. #### Mutability @@ -235,8 +240,8 @@ fun test() { Initially, the following syntax was suggested: ```kotlin -public val items = mutableListOf() - private get(): List +private val items = mutableListOf() + public get(): List ``` > In above example `items` is `MutableList` when accessed privately inside class and read-only `List` when accessed from outside class. From 0cc9fa2721e178e68f822fcfa20c8327ca41cc3b Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Fri, 17 Sep 2021 20:07:43 +0300 Subject: [PATCH 04/12] Fix some things --- proposals/explicit-backing-fields.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index c53c9880c..c0b6490c6 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -145,7 +145,7 @@ Explicit backing fields is a FIR-only feature. If the compiler can not provide a getter, the user must declare it explicitly. The same applies to setters in case of `var` properties. ```kotlin -public val flow: SharedFlow +public val flow: SharedFlow field: MutableSharedFlow? = null get() { return field ?: run { ... } @@ -254,7 +254,7 @@ Attempt to add support for the above syntax led to multiple redundant complicati ### Direct Backing Field Access -It might be handy to allow the direct access to the property backing field via syntax like `myProperty#field`. +It might be handy to allow the direct access to the property backing field via syntax like `myProperty#field` (just a hypothetical syntax). ### Allow `protected field` From 8367fb3369fec7194271aad25ffa7f92c8209206 Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Fri, 17 Sep 2021 21:31:28 +0300 Subject: [PATCH 05/12] Rearrange the text and fix the indents --- proposals/explicit-backing-fields.md | 114 ++++++++++++++------------- 1 file changed, 59 insertions(+), 55 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index c0b6490c6..60b7d2096 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -12,54 +12,34 @@ **Note**: initial proposal contents have been partially copied down here for convenience. Despite the different approach, the already shown use cases are still relevant. -> Common pattern in java is having full type accessible as private property and then only exposing required interface in the public getter: -> -> ```java -> class MyClass { -> // Use full type for private access -> private final ArrayList data = new ArrayList<>(); -> -> // Only expose what is needed in public getter -> public List getData() { -> return data; -> } -> } -> ``` -> -> This pattern allows easy hiding of implementation details and allows for easy clean external interfaces to the classes. But there is no idiomatic Kotlin way to achieve this pattern. Best thing you can do is define two separate properties to mimic how Java does it: -> -> ```kotlin -> class MyClass { -> private val _data = ArrayList() -> val data: List -> get() = _data -> } -> ``` +Sometimes, Kotlin programmers need to declare two properties which are conceptually the same, but one is part of a public API and another is an implementation detail. This is known as [backing properties](https://kotlinlang.org/docs/properties.html#backing-properties): + +```kotlin +class C { + private val _elementList = mutableListOf() + + val elementList: List + get() = _elementList +} +``` With the proposed syntax in mind, the above code snippet could be rewritten as follows: ```kotlin -class MyClass { - val data: List - field = ArrayList() +class C { + val elementList: List + field = mutableListOf() } ``` ## Use Cases -### Motivation - -> - There is no clean idiomatic way to do this pattern in Kotlin -> - Current best approach throws away all benefits of Kotlin properties and forces developer to write Java-like code with separate private field and public getter -> - Current best approach forces developer to assign two different names to single property (or pad private property, for example adding `_` prefix and then using this prefix everywhere in code) -> - This is another place where Java pattern could be introduced into Kotlin with less boilerplate - ### Read-Only from Outside > We often do not want our data structures to be modified from outside. Unlike Java, this can be easily achieved in Kotlin by just exposing read-only `List`. But as already ilustrated in above example, exposing different type of a property is a bit messy. ```kotlin -private val _items = mutableListOf() +internal val _items = mutableListOf() val item : List by _items ``` @@ -67,7 +47,7 @@ And the new syntax allows us to write: ```kotlin val items: List - private field = mutableListOf() + internal field = mutableListOf() ``` ### Android Architecture Components @@ -92,13 +72,45 @@ Becomes: ```kotlin val city: LiveData - field = MutableLiveData().apply { value = "" } + field = MutableLiveData().apply { value = "" } val charts: LiveData> - field = MutableLiveData().apply { value = emptyList() } + field = MutableLiveData().apply { value = emptyList() } val loading: LiveData - field = MutableLiveData().apply { value = false } + field = MutableLiveData().apply { value = false } val message: LiveData - field = MutableLiveData() + field = MutableLiveData() +``` + +### Common Java Pattern + +> Common pattern in java is having full type accessible as private property and then only exposing required interface in the public getter: +> +> ```java +> class MyClass { +> // Use full type for private access +> private final ArrayList data = new ArrayList<>(); +> +> // Only expose what is needed in public getter +> public List getData() { +> return data; +> } +> } +> ``` +> +> This pattern allows easy hiding of implementation details and allows for easy clean external interfaces to the classes. +> +> - There is no clean idiomatic way to do this pattern in Kotlin +> - Current best approach throws away all benefits of Kotlin properties and forces developer to write Java-like code with separate private field and public getter +> - Current best approach forces developer to assign two different names to single property (or pad private property, for example adding `_` prefix and then using this prefix everywhere in code) +> - This is another place where Java pattern could be introduced into Kotlin with less boilerplate + +The proposed syntax allows to achieve the same functionality while keeping the same level of simplicity: + +```kotlin +class MyClass { + val data: List + field = ArrayList() +} ``` ### RX Observable and Subjects @@ -118,7 +130,7 @@ Turns into: ```kotlin class MyClass { val dataStream: Observable - field = PublishSubject.create() + field = PublishSubject.create() } ``` @@ -130,7 +142,7 @@ The proposed design consists of two new ideas. ```kotlin val it: P - [visibility] field[: F] = initializer + [visibility] field[: F] = initializer ``` The above `field` declaration is referred to as an _explicit backing field declaration_. @@ -176,21 +188,13 @@ var someStrangeExample: Int } ``` -#### Initializer - -If there is an explicit backing field, the property must not declare an initializer. - -If the explicit backing field is not `lateinit`, it must have an initializer. For `lateinit` properties initializers are forbidden. - -#### Mutability - -For now, we assume `val` properties have immutable backing fields, and `var` properties have mutable ones. +#### Restrictions -#### Forbidden +If there is an explicit backing field, the property must not declare an initializer. If the explicit backing field is not `lateinit`, it must have an initializer. For `lateinit` properties, initializers are forbidden. -Explicit backing fields are not allowed inside interfaces or abstract properties. +For now, we assume `val` properties have immutable backing fields, and `var` properties have mutable ones. That is, assignment to `field` inside a `val` property getter results in an error. -Explicit backing fields are not allowed for delegated properties. +Explicit backing fields are not allowed inside interfaces or abstract properties, as well as they are forbidden for delegated properties. ### Smart Type Narrowing @@ -220,7 +224,7 @@ The formal checks corresponding to the above rules are: ```kotlin class MyClass { val items: List - field = mutableListOf("a", "b") + field = mutableListOf("a", "b") fun registerItem(item: String) { items.add(item) // Viewed as MutableList @@ -241,7 +245,7 @@ Initially, the following syntax was suggested: ```kotlin private val items = mutableListOf() - public get(): List + public get(): List ``` > In above example `items` is `MutableList` when accessed privately inside class and read-only `List` when accessed from outside class. From 6bdd59e5538309d65f34f298ecb9220e538f7497 Mon Sep 17 00:00:00 2001 From: Nikolay Lunyak Date: Mon, 20 Sep 2021 13:37:36 +0300 Subject: [PATCH 06/12] Replace `A :> B` with `P :> F` --- proposals/explicit-backing-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 60b7d2096..704f31374 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -212,7 +212,7 @@ it can then narrow down the returned instance type at the call site without loss The formal checks corresponding to the above rules are: 1. the property does not have a custom getter, only the default one -2. `A :> B` +2. `P :> F` 3. accessing: 1. a private backing field of a class member property within the class 2. a private backing field of a top-level property within the file From c27a5b2d858d5e688d165bc5b7d4c9f7557b65c4 Mon Sep 17 00:00:00 2001 From: Roman Elizarov Date: Mon, 24 Jan 2022 19:46:36 +0300 Subject: [PATCH 07/12] Updated explicit backing fields proposal * Use-cases are orothogonalized (similar) use-cases are grouped together. * Additional use-cases are explained to cover all the major pieces of the proposed design. * Design is worked out in more details, with grammar and various restrictions more explicitly spelled out. * This list of future enhancements is narrowed down to the ones that are feasible in the near future. * Table of Contents is added for reference. --- proposals/explicit-backing-fields.md | 355 +++++++++++++++------------ 1 file changed, 192 insertions(+), 163 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 704f31374..5848c2f5b 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -1,18 +1,17 @@ # Explicit Backing Fields - **Type**: Design Proposal -- **Author**: ? +- **Author**: Nikolay Lunyak, Roman Elizarov - **Contributors**: Svetlana Isakova, Kirill Rakhman, Dmitry Petrov, Roman Elizarov, Ben Leggiero, Matej Drobnič, Mikhail Glukhikh, Nikolay Lunyak -- **Status**: Implemented in FIR -- **Prototype**: Implemented +- **Status**: Prototype implemented in FIR - **Initial YouTrack Issue**: [KT-14663](https://youtrack.jetbrains.com/issue/KT-14663) - **Initial Proposal**: [private_public_property_types#122](https://github.com/Kotlin/KEEP/pull/122) ## Summary -**Note**: initial proposal contents have been partially copied down here for convenience. Despite the different approach, the already shown use cases are still relevant. - -Sometimes, Kotlin programmers need to declare two properties which are conceptually the same, but one is part of a public API and another is an implementation detail. This is known as [backing properties](https://kotlinlang.org/docs/properties.html#backing-properties): +Sometimes, Kotlin programmers need to declare two properties which are conceptually the same, +but one is part of a public API and another is an implementation detail. +This pattern is known as [backing properties](https://kotlinlang.org/docs/properties.html#backing-properties): ```kotlin class C { @@ -32,11 +31,39 @@ class C { } ``` +## Table of contents + + + +* [Use Cases](#use-cases) + * [Expose read-only subtype](#expose-read-only-subtype) + * [Decouple storage type from external representation](#decouple-storage-type-from-external-representation) + * [Expose read-only view](#expose-read-only-view) + * [Access field from outside of getter and setter](#access-field-from-outside-of-getter-and-setter) +* [Design](#design) + * [Explicit Backing Fields](#explicit-backing-fields) + * [Restrictions](#restrictions) + * [Accessors](#accessors) + * [Visibility](#visibility) + * [Lateinit](#lateinit) + * [Smart Type Narrowing](#smart-type-narrowing) +* [Alternatives](#alternatives) + * [Initial Proposal](#initial-proposal) +* [Future Enhancements](#future-enhancements) + * [Direct Backing Field Access](#direct-backing-field-access) + * [Protected Fields](#protected-fields) + * [Mutable Fields for Read-only Properties](#mutable-fields-for-read-only-properties) + + + ## Use Cases -### Read-Only from Outside +This proposal caters to a variety of use-cases that are currently met via a backing property pattern. -> We often do not want our data structures to be modified from outside. Unlike Java, this can be easily achieved in Kotlin by just exposing read-only `List`. But as already ilustrated in above example, exposing different type of a property is a bit messy. +### Expose read-only subtype + +We often do not want our data structures to be modified from outside. It is customary in Kotlin to have +a read-only (e.g. `List`) and a mutable (e.g. `MutableList`) interface to the same data structure. ```kotlin internal val _items = mutableListOf() @@ -50,11 +77,13 @@ val items: List internal field = mutableListOf() ``` -### Android Architecture Components - -> Proper way to do Architecture components is to use `MutableLiveData` (`LiveData` implementation that allows caller to change its value) privately inside View Model classes and then only expose read-only `LiveData` objects outside. +This use-case is also widely applicable to architecture of reactive applications: + +* Android `LiveData` has a `MutableLiveData` counterpart. +* Rx `Observable` has a mutable `Subject` counterpart. +* Kotlin coroutines `SharedFlow` has a `MutableSharedFlow`, etc. -Sample code [from an Android app](https://github.com/elpassion/crweather/blob/9c3e3cb803b7e4fffbb010ff085ac56645c9774d/app/src/main/java/com/elpassion/crweather/MainModel.kt#L14): +For example, sample code [from an Android app](https://github.com/elpassion/crweather/blob/9c3e3cb803b7e4fffbb010ff085ac56645c9774d/app/src/main/java/com/elpassion/crweather/MainModel.kt#L14): ```kotlin private val mutableCity = MutableLiveData().apply { value = "" } @@ -80,100 +109,155 @@ val loading: LiveData val message: LiveData field = MutableLiveData() ``` + +> `private` is a default access for a `field` declaration. -### Common Java Pattern - -> Common pattern in java is having full type accessible as private property and then only exposing required interface in the public getter: -> -> ```java -> class MyClass { -> // Use full type for private access -> private final ArrayList data = new ArrayList<>(); -> -> // Only expose what is needed in public getter -> public List getData() { -> return data; -> } -> } -> ``` -> -> This pattern allows easy hiding of implementation details and allows for easy clean external interfaces to the classes. -> -> - There is no clean idiomatic way to do this pattern in Kotlin -> - Current best approach throws away all benefits of Kotlin properties and forces developer to write Java-like code with separate private field and public getter -> - Current best approach forces developer to assign two different names to single property (or pad private property, for example adding `_` prefix and then using this prefix everywhere in code) -> - This is another place where Java pattern could be introduced into Kotlin with less boilerplate - -The proposed syntax allows to achieve the same functionality while keeping the same level of simplicity: +In this use-case, an read access to the field from inside the corresponding classes/modules +(where the field is visible) automatically gives access to mutable type, which we call [Smart Type Narrowing](#smart-type-narrowing), +but is seen as a property with read-only type to the outside code. + +### Decouple storage type from external representation + +Sometimes a property must be internally represented by a different type for storage-efficiency or architectural reason, +while having a different outside type. For example, API requirements might dictate that the property type +is `String`, but if we know that it always represents a decimal integer, then it can be efficiently stored as such +with custom getter and custom setter. ```kotlin -class MyClass { - val data: List - field = ArrayList() -} +val number: String + field: Int = 0 + get() = field.toString() + set(value) { field = value.toInt() } ``` -### RX Observable and Subjects +### Expose read-only view + +In some application it is desirable to expose not just a read-only subtype, but a specially constructed read-only +view that protects the data structure from casting into mutable type. For example a `MutableStateFlow` has +[`asStateFlow`](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/as-state-flow.html) +extension for that purpose. -> Data can often be pushed into reactive world using subjects. However, exposing `Subject` would allow consumer of the class to push its own data into it. That is why it is good idea to expose all subjects as read-only `Observable`: +With the proposed syntax it is possible to declare a custom getter: ```kotlin -class MyClass { - private val _dataStream = PublishSubject.create() - val dataStream: Observable - get() = _dataStream -} +val state: StateFlow + get() = field.asStateFlow() + field = MutableStateFlow(State.INITIAL) ``` -Turns into: +For this use-case, the TBD syntax of [Direct Backing Field Access](#direct-backing-field-access) will need to be added. + +### Access field from outside of getter and setter + +Kotlin allows property field to be accessed from the property's getter and setter using a `field` variable. +This proposal is designed with an idea to provide an explicit syntax to access a property's field from anywhere +inside the corresponding class when the field is declared explicitly: ```kotlin -class MyClass { - val dataStream: Observable - field = PublishSubject.create() +class Component { + var status: Status + field // explicit field declaration + set(value) { + field = value + notifyStatusChanged() + } } ``` +This way, all the code inside the class can change the field of the property directly, without invoking the setter . +However, the actual access syntax of such [Direct Backing Field Access](#direct-backing-field-access) is TBD. + ## Design -The proposed design consists of two new ideas. +The proposed design consists of two new ideas: explicit backing fields and smart type narrowing. ### Explicit Backing Fields -```kotlin -val it: P - [visibility] field[: F] = initializer +The grammar for `propertyDeclaration` is extended to support an optional _explicit backing field declaration_ in addition +to the optional `getter` and `setter` (in any order between them). + ``` +propertyDeclaration ::= + modifiers? ('val' | 'var') typeParameters? + (receiverType '.')? + (multiVariableDeclaration | variableDeclaration) + typeConstraints? (('=' expression) | propertyDelegate)? ';'? + ( (getter? (semi? setter)? (semi? field)?) + | (setter? (semi? getter)? (semi? field)?) + | (getter? (semi? field)? (semi? setter)?) + | (setter? (semi? field)? (semi? getter)?) + | (field? (semi? getter)? (semi? setter)?) + | (field? (semi? setter)? (semi? getter)?) + +getter ::= + modifiers? 'get' ('(' ')' (':' type)? functionBody)? + +setter ::= + modifiers? 'set' ('(' functionValueParameterWithOptionalType ','? ')' (':' type)? functionBody)? + +field ::= + modifiers? 'field` (':' type)? ('=' expression)? +``` + +Explicit backing field declaration has an optional visibility, an optional type, and an optional +initialization expression. + +#### Restrictions + +There are the following additional semantic restrictions on the property declaration grammar: + +* A property with an explicit field declaration cannot have its own initializer. + A property with an explicit field must be initialized with the initialization expression for its + field to clarify the fact, that property initialization goes directly to the field and does not + call property's setter. The property without field initialization expression is considered + uninitialized and is allowed only when it is `lateinit` (see [Lateinit](#lateinit) section for details). +* A property with an explicit backing field must always explicitly specify the type of the property itself. +* Explicit backing field declaration is not allowed for interface properties, for `abstract` properties, and for delegated properties. -The above `field` declaration is referred to as an _explicit backing field declaration_. +A backing field type is not required to be explicitly specified: -Explicit backing fields is a FIR-only feature. +* If both backing field type and initialization expression are not specified, then the field type is the same as the property type. +* If backing field type is not specified, but there is an initialization expression, then + the field type is inferred from the type of its backing field initialization expression. +* When both field type and initialization expression are specified, then the type of the former must be + assignable to the latter. + +Backing field assignability is the same as it is now for `field` references in getters and setters: + +* `var` properties have mutable backing fields. +* `val` properties have read-only backing fields. That is, assignment to `field` inside a `val` property getter results in an error. #### Accessors + +When explicit backing field with type `F` for a property with type `P` is declared explicitly, +then compiler can derive getter and, for `var` properties, setter implementation automatically: -- if `P :> F`, the compiler can provide a default getter -- if `P <: F`, the compiler can provide a default setter +* If `P :> F`, the compiler can provide a default getter. +* If `P <: F`, the compiler can provide a default setter. -If the compiler can not provide a getter, the user must declare it explicitly. The same applies to setters in case of `var` properties. +If the compiler can not provide a getter, the user must declare it explicitly. +The same applies to setters in case of `var` properties. ```kotlin public val flow: SharedFlow field: MutableSharedFlow? = null - get() { + get() { // It is an error if getter is not explicitly specified here return field ?: run { ... } } ``` #### Visibility -Only the `private` and the `internal` visibilities are allowed for explicit backing fields now. The default visibility is `private`. +Only the `private` and `internal` visibilities are allowed for explicit backing fields. +The default field visibility is `private`. ```kotlin val mutableWithinModule: List internal field = mutableListOf() ``` -Right now, there is no special syntax to explicitly access the backing field from outside the property accessors, but we will see how it can be accessed implicitly via the ["second idea"](#smart-type-narrowing). +The special syntax to explicitly access the backing field from outside the code of property accessors is TBD, +but the field can be implicitly access when it is visible via the [Smart Type Narrowing](#smart-type-narrowing). #### Lateinit @@ -188,54 +272,41 @@ var someStrangeExample: Int } ``` -#### Restrictions - -If there is an explicit backing field, the property must not declare an initializer. If the explicit backing field is not `lateinit`, it must have an initializer. For `lateinit` properties, initializers are forbidden. - -For now, we assume `val` properties have immutable backing fields, and `var` properties have mutable ones. That is, assignment to `field` inside a `val` property getter results in an error. - -Explicit backing fields are not allowed inside interfaces or abstract properties, as well as they are forbidden for delegated properties. - ### Smart Type Narrowing -#### Rules - -If: +The idea behind smart type narrowing is to implicitly access the underlying field, as opposed to the property, +when the field is in scope and when it is safe to do so. For example, expanding on +[Expose read-only subtype](#expose-read-only-subtype) use-case, one can write: -1. the compiler can guarantee the property getter returns the same instance as the one stored in the backing field -2. the type of that instance is compatible with the property type -3. the backing field visibility allows a hypothetical direct access -4. there is no ambiguity in such an access +```kotlin +class Component { + val items: List + field = mutableListOf() + + fun addItem(item: Item) { + items += item // works + } +} -it can then narrow down the returned instance type at the call site without loss of functionality. This is what is meant by the words _smart type narrowing_. +// outside code +component.items += item // does not compile; cannot add to List +``` -The formal checks corresponding to the above rules are: +The code above works, because `items` there implicitly refers to the field with type `MutableList`. -1. the property does not have a custom getter, only the default one -2. `P :> F` -3. accessing: - 1. a private backing field of a class member property within the class - 2. a private backing field of a top-level property within the file - 3. an internal backing field within the module -4. property must be final, otherwise an overridden version would be able to provide a custom getter that may return a different instance +Smart type narrowing works when trying to read the value of the property and all the following conditions are met: -#### Example +* The backing field is visible from the point of access. +* The property is final (that is, it is not open `open`). +* The property does not have an explicit getter. -```kotlin -class MyClass { - val items: List - field = mutableListOf("a", "b") - - fun registerItem(item: String) { - items.add(item) // Viewed as MutableList - } -} +> The last rule automatically guarantees that the getter was automatically generated. Together with +> the requirement that the field is not `open`, it means that the compiler knows that the field stores the same +> instance as returned by the getter and that the type of the field is narrower than the type of +> the property (see [Accessors](#accessors) section). -fun test() { - val it = MyClass() - it.items // Viewed as a List -} -``` +In this case, the type of property read expression is narrowed by the compiler from the type of the property +to the type of its field. ## Alternatives @@ -250,77 +321,35 @@ private val items = mutableListOf() > In above example `items` is `MutableList` when accessed privately inside class and read-only `List` when accessed from outside class. -In fact, the above syntax brings in an incorrect mental model: it says _'There is a private property `it` with some *part* that declares its public behavior'_. +In fact, the above syntax brings in an incorrect mental model: it says _'There is a private property `items` with some *part* that declares its public behavior'_. -Attempt to add support for the above syntax led to multiple redundant complications (see the [problems section](https://github.com/matejdro/KEEP/blob/private_public_property_types/proposals/private_public_property_types.md#questions-to-consider) + unclear override mechanics). +Attempt to add support for the above syntax led to multiple redundant complications (see the [problems section](https://github.com/matejdro/KEEP/blob/private_public_property_types/proposals/private_public_property_types.md#questions-to-consider) and unclear override mechanics). ## Future Enhancements ### Direct Backing Field Access + +We plan to add support for a syntax to explicitly access the property's backing field when the field was +explicitly declared and is accessible via some TBD syntax. -It might be handy to allow the direct access to the property backing field via syntax like `myProperty#field` (just a hypothetical syntax). - -### Allow `protected field` - -This way, smart type narrowing would become possible inside subclasses as well. - -### Use of `field` Before Property Type Is Known +### Protected Fields + +The set of visibilities for an explicitly declared field can be extended to include `protected` +(in addition to `private` and `internal`). This way, subclasses can explicitly or implicitly +(via the [smart type narrowing](#smart-type-narrowing)) reference the field. -Right now, any attempt to use `field` inside a property accessor in cases when the property type is not yet known (depends on the getter return value) will result into `UNRESOLVED_REFERENCE`. +### Mutable Fields for Read-only Properties -Since we now can have explicit backing field declarations, we may use them to find out the type for backing fields: +Letting a `val` property have a mutable backing field may be useful. +Consider the following snippet from the +[implementation of `lazy` in the Kotlin standard library](https://github.com/JetBrains/kotlin/blob/master/libraries/stdlib/jvm/src/kotlin/util/LazyJVM.kt#L57): ```kotlin -var thing - field = SomeThing() +// backing field pattern +private var _value: Any? = UNINITIALIZED_VALUE + +override val value: T get() { - return convertSomehow(field) - } - set(value) { - field = convertTheOtherWayAround(value) + // initializes _value backing field on the first access } ``` - -### Custom Getters - -Hypothetically, we could perform more analysis to find out whether smart type narrowing is possible for the given getter. - -### Mutability - -Letting a `val` property have a mutable backing field may be useful. Consider the [following example](https://github.com/JetBrains/kotlin/blob/master/libraries/stdlib/jvm/src/kotlin/util/LazyJVM.kt#L57): - -```kotlin -private class SynchronizedLazyImpl( - initializer: () -> T, - lock: Any? = null -) : Lazy, Serializable { - private var initializer: (() -> T)? = initializer - @Volatile private var _value: Any? = UNINITIALIZED_VALUE - // final field is required to enable safe publication of constructed instance - private val lock = lock ?: this - - override val value: T - get() { - val _v1 = _value - if (_v1 !== UNINITIALIZED_VALUE) { - @Suppress("UNCHECKED_CAST") - return _v1 as T - } - - return synchronized(lock) { - val _v2 = _value - if (_v2 !== UNINITIALIZED_VALUE) { - @Suppress("UNCHECKED_CAST") (_v2 as T) - } else { - val typedValue = initializer!!() - _value = typedValue - initializer = null - typedValue - } - } - } - - ... -} -``` - From 29f7bc70462f559d295e9df787dd01df04bda7c0 Mon Sep 17 00:00:00 2001 From: Roman Elizarov Date: Thu, 27 Jan 2022 10:27:52 +0300 Subject: [PATCH 08/12] Update proposals/explicit-backing-fields.md Co-authored-by: ilya-g --- proposals/explicit-backing-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 5848c2f5b..48fbdda2f 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -164,7 +164,7 @@ class Component { } ``` -This way, all the code inside the class can change the field of the property directly, without invoking the setter . +This way, all the code inside the class can change the field of the property directly, without invoking the setter. However, the actual access syntax of such [Direct Backing Field Access](#direct-backing-field-access) is TBD. ## Design From 35ca7802cd0e1491450cd931176e52d906dd350b Mon Sep 17 00:00:00 2001 From: Alex Sokol Date: Wed, 26 Jan 2022 16:32:37 +0300 Subject: [PATCH 09/12] Fixed type The variable should be var to have setter --- proposals/explicit-backing-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 48fbdda2f..bd2b18d39 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -124,7 +124,7 @@ is `String`, but if we know that it always represents a decimal integer, then it with custom getter and custom setter. ```kotlin -val number: String +var number: String field: Int = 0 get() = field.toString() set(value) { field = value.toInt() } From 31f6f8eae5b4c1438a57d1430928ef7791e10cc5 Mon Sep 17 00:00:00 2001 From: Roman Elizarov Date: Wed, 2 Feb 2022 20:01:55 +0300 Subject: [PATCH 10/12] Fixes various issues after review * Fix subtype -> supertype. * Clarify the advantages of the new syntax of the backing property pattern. * Clarify repeated allocations in Expose read-only view use-case. * Fix Access field from outside of getter and setter use-cases, don't require explicit field declaration. * Retrieve property delegate reference use-case added. --- proposals/explicit-backing-fields.md | 85 ++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 12 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index bd2b18d39..141ce6ad3 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -36,10 +36,11 @@ class C { * [Use Cases](#use-cases) - * [Expose read-only subtype](#expose-read-only-subtype) + * [Expose read-only supertype](#expose-read-only-supertype) * [Decouple storage type from external representation](#decouple-storage-type-from-external-representation) * [Expose read-only view](#expose-read-only-view) * [Access field from outside of getter and setter](#access-field-from-outside-of-getter-and-setter) + * [Retrieve property delegate reference](#retrieve-property-delegate-reference) * [Design](#design) * [Explicit Backing Fields](#explicit-backing-fields) * [Restrictions](#restrictions) @@ -58,16 +59,17 @@ class C { ## Use Cases -This proposal caters to a variety of use-cases that are currently met via a backing property pattern. +This proposal caters to a variety of use-cases that are currently met via a +[backing property pattern](https://kotlinlang.org/docs/properties.html#backing-properties). -### Expose read-only subtype +### Expose read-only supertype We often do not want our data structures to be modified from outside. It is customary in Kotlin to have a read-only (e.g. `List`) and a mutable (e.g. `MutableList`) interface to the same data structure. ```kotlin internal val _items = mutableListOf() -val item : List by _items +val item: List by _items ``` And the new syntax allows us to write: @@ -77,6 +79,9 @@ val items: List internal field = mutableListOf() ``` +> While the number of lines in this code does not change, the related properties become better grouped together and +repetition of the property name multiple times is avoided. + This use-case is also widely applicable to architecture of reactive applications: * Android `LiveData` has a `MutableLiveData` counterpart. @@ -145,18 +150,22 @@ val state: StateFlow field = MutableStateFlow(State.INITIAL) ``` -For this use-case, the TBD syntax of [Direct Backing Field Access](#direct-backing-field-access) will need to be added. +> Note, that `field.asStateFlow()` allocates a wrapper read-only view object and it is called here every time this +property is accessed, allocating a new instance. For some applications this is an acceptable and even preferable +behavior. For others, it is crucial to create this view object once and cache it in a separate field. We don't plan +to address the later use-cases — that's where the backing-field pattern will have to continue to be used. + +For this use-case to become actually usable, the TBD syntax of [Direct Backing Field Access](#direct-backing-field-access) +will need to be added. That's because such applications need to modify the mutable data structure that is stored in the field, +but the rules of [Smart Type Narrowing](#smart-type-narrowing) will not make it directly available. ### Access field from outside of getter and setter Kotlin allows property field to be accessed from the property's getter and setter using a `field` variable. -This proposal is designed with an idea to provide an explicit syntax to access a property's field from anywhere -inside the corresponding class when the field is declared explicitly: ```kotlin class Component { var status: Status - field // explicit field declaration set(value) { field = value notifyStatusChanged() @@ -164,8 +173,60 @@ class Component { } ``` -This way, all the code inside the class can change the field of the property directly, without invoking the setter. -However, the actual access syntax of such [Direct Backing Field Access](#direct-backing-field-access) is TBD. +This proposal is designed with an idea to provide an explicit syntax to access a property's field from anywhere +inside the corresponding class as if the backing field was declared with `private` access. In this particular example, +any code inside the class `Component` will be able to change the field of the `status` property directly, without invoking the setter. +However, the actual access syntax of such [Direct Backing Field Access](#direct-backing-field-access) is TBD. + +### Retrieve property delegate reference + +Delegated properties in Kotlin have a backing field that stores a reference to the delegate object and +automatically generate getter and setter implementations that forward accesses to the delegate. +The following Kotlin code: + +```kotlin +class Data { + val expensive: T by lazy { /*...*/ } +} +``` + +is basically compiled into the following code: + +```kotlin +class Data { + private val expensive$delegate: Lazy = SynchronizedLazyImpl { /*...*/ } + val expensive: T + get() = expensive$delegate.value +} +``` + +As you can see, `expensive$delegate` here is effectively a backing for the property with the delegate. +The are many use-cases where the direct access to the delegate is need. For example, a `Lazy` type +has [`isInitialized`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-lazy/is-initialized.html) function. +The existing syntax for its access is cumbersome and not type-safe: + +```kotlin +fun isExpensiveInitialized() = + (this::value.getDelegate() as Lazy).isInitialized() +``` + +In practice, it means that +[developers have to resort to an backing field pattern](https://stackoverflow.com/questions/42522739/kotlin-check-if-lazy-val-has-been-initialised) +to make it look nice: + +```kotlin +class Data { + private val expensiveDelegate: Lazy = SynchronizedLazyImpl { /*...*/ } + val expensive: T + get() = expensiveDelegate.value + fun isExpensiveInitialized() = + expensiveDelegate.isInitialized() +} +``` + +It is expected that TBD [Direct Backing Field Access](#direct-backing-field-access) syntax is going to +provide a direct and simpler way to access the delegate of a delegated property without having to +resort to the backing field pattern. ## Design @@ -329,8 +390,8 @@ Attempt to add support for the above syntax led to multiple redundant complicati ### Direct Backing Field Access -We plan to add support for a syntax to explicitly access the property's backing field when the field was -explicitly declared and is accessible via some TBD syntax. +We plan to add support for a syntax to explicitly access the property's backing field as well as for property +delegate via some TBD syntax. ### Protected Fields From cc38eca4ec04a6864319fd3ed7a4cb7da0d66e26 Mon Sep 17 00:00:00 2001 From: Roman Elizarov Date: Fri, 10 Jun 2022 10:45:49 +0300 Subject: [PATCH 11/12] Prototype in K2 preview with Kotlin 1.7.0 --- proposals/explicit-backing-fields.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 141ce6ad3..3475f75f2 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -3,7 +3,7 @@ - **Type**: Design Proposal - **Author**: Nikolay Lunyak, Roman Elizarov - **Contributors**: Svetlana Isakova, Kirill Rakhman, Dmitry Petrov, Roman Elizarov, Ben Leggiero, Matej Drobnič, Mikhail Glukhikh, Nikolay Lunyak -- **Status**: Prototype implemented in FIR +- **Status**: Prototype implemented in K2 compiler, in preview since 1.7.0 - **Initial YouTrack Issue**: [KT-14663](https://youtrack.jetbrains.com/issue/KT-14663) - **Initial Proposal**: [private_public_property_types#122](https://github.com/Kotlin/KEEP/pull/122) @@ -54,6 +54,8 @@ class C { * [Direct Backing Field Access](#direct-backing-field-access) * [Protected Fields](#protected-fields) * [Mutable Fields for Read-only Properties](#mutable-fields-for-read-only-properties) +* [Change log](#change-log) + * [Prototype in K2 preview with Kotlin 1.7.0](#prototype-in-k2-preview-with-kotlin-170) @@ -414,3 +416,15 @@ override val value: T // initializes _value backing field on the first access } ``` + +## Change log + +This section records changes to this KEEP. + +### Prototype in K2 preview with Kotlin 1.7.0 + +Prototype of this proposal has been delivered in Kotlin 1.7.0 as a part of K2 compiler preview. +In order to try out his new feature you need to enable the K2 compiler with +`-Xuse-k2` command line option and enable this language feature with `-XXLanguage:+ExplicitBackingFields`. +The implementation is not stable yet and will generate pre-release binary. +There is no IDE support yet. From 1880e68d4af64bc3d97eef07c44e7e9862b64991 Mon Sep 17 00:00:00 2001 From: Roman Efremov Date: Mon, 5 Feb 2024 14:58:53 +0100 Subject: [PATCH 12/12] Update KEEP on Explicit backing fields feature --- proposals/explicit-backing-fields.md | 637 +++++++++++++++------------ 1 file changed, 359 insertions(+), 278 deletions(-) diff --git a/proposals/explicit-backing-fields.md b/proposals/explicit-backing-fields.md index 3475f75f2..254cfc21b 100644 --- a/proposals/explicit-backing-fields.md +++ b/proposals/explicit-backing-fields.md @@ -1,11 +1,12 @@ # Explicit Backing Fields - **Type**: Design Proposal -- **Author**: Nikolay Lunyak, Roman Elizarov -- **Contributors**: Svetlana Isakova, Kirill Rakhman, Dmitry Petrov, Roman Elizarov, Ben Leggiero, Matej Drobnič, Mikhail Glukhikh, Nikolay Lunyak -- **Status**: Prototype implemented in K2 compiler, in preview since 1.7.0 +- **Authors**: Nikolay Lunyak, Roman Elizarov, Roman Efremov +- **Contributors**: Mikhail Zarechenskiy, Marat Akhin, Alejandro Serrano Mena, Anastasia Nekrasova, Svetlana Isakova, Kirill Rakhman, Dmitry Petrov, Ben Leggiero, Matej Drobnič, Mikhail Glukhikh +- **Status**: Design discussion +- **Discussion**: [KEEP-278](https://github.com/Kotlin/KEEP/issues/278) - **Initial YouTrack Issue**: [KT-14663](https://youtrack.jetbrains.com/issue/KT-14663) -- **Initial Proposal**: [private_public_property_types#122](https://github.com/Kotlin/KEEP/pull/122) +- **Previous Proposal**: [explicit-backing-fields.md, commit cc38eca](https://github.com/Kotlin/KEEP/blob/cc38eca4ec04a6864319fd3ed7a4cb7da0d66e26/proposals/explicit-backing-fields.md) ## Summary @@ -14,20 +15,18 @@ but one is part of a public API and another is an implementation detail. This pattern is known as [backing properties](https://kotlinlang.org/docs/properties.html#backing-properties): ```kotlin -class C { - private val _elementList = mutableListOf() - - val elementList: List - get() = _elementList +class SomeViewModel : ViewModel() { + private val _city = MutableLiveData() + val city: LiveData get() = _city } ``` With the proposed syntax in mind, the above code snippet could be rewritten as follows: ```kotlin -class C { - val elementList: List - field = mutableListOf() +class SomeViewModel : ViewModel() { + val city: LiveData + field = MutableLiveData() } ``` @@ -35,209 +34,346 @@ class C { -* [Use Cases](#use-cases) - * [Expose read-only supertype](#expose-read-only-supertype) - * [Decouple storage type from external representation](#decouple-storage-type-from-external-representation) - * [Expose read-only view](#expose-read-only-view) - * [Access field from outside of getter and setter](#access-field-from-outside-of-getter-and-setter) - * [Retrieve property delegate reference](#retrieve-property-delegate-reference) +* [Use cases](#use-cases) + * [Use cases targeted by the explicit backing field feature](#use-cases-targeted-by-the-explicit-backing-field-feature) + * [Expose read-only supertype](#expose-read-only-supertype) + * [Expose different object](#expose-different-object) + * [Use cases not supported by the explicit backing field feature](#use-cases-not-supported-by-the-explicit-backing-field-feature) + * [`var` backing property](#var-backing-property) + * [`lateinit` backing property](#lateinit-backing-property) + * [Delegation](#delegation) + * [Non-private visibility of property](#non-private-visibility-of-property) * [Design](#design) * [Explicit Backing Fields](#explicit-backing-fields) - * [Restrictions](#restrictions) - * [Accessors](#accessors) - * [Visibility](#visibility) - * [Lateinit](#lateinit) - * [Smart Type Narrowing](#smart-type-narrowing) -* [Alternatives](#alternatives) - * [Initial Proposal](#initial-proposal) -* [Future Enhancements](#future-enhancements) - * [Direct Backing Field Access](#direct-backing-field-access) - * [Protected Fields](#protected-fields) - * [Mutable Fields for Read-only Properties](#mutable-fields-for-read-only-properties) -* [Change log](#change-log) - * [Prototype in K2 preview with Kotlin 1.7.0](#prototype-in-k2-preview-with-kotlin-170) + * [Visibility](#visibility) + * [Resolution](#resolution) + * [Accessors](#accessors) + * [Explicit backing field in combination with property initializer](#explicit-backing-field-in-combination-with-property-initializer) + * [Accessing field inside accessors and initializers](#accessing-field-inside-accessors-and-initializers) + * [Type inference](#type-inference) + * [Other restrictions](#other-restrictions) +* [Technical details](#technical-details) + * [Grammar changes](#grammar-changes) + * [Java interoperability](#java-interoperability) + * [Reflection](#reflection) +* [Risks](#risks) + * [Unintentionally exposing a field as a return value](#unintentionally-exposing-a-field-as-a-return-value) +* [Future enhancements](#future-enhancements) + * [Underscore operator in type parameters](#underscore-operator-in-type-parameters) -## Use Cases +## Use cases -This proposal caters to a variety of use-cases that are currently met via a +This proposal caters to a variety of use-cases that are currently met via a [backing property pattern](https://kotlinlang.org/docs/properties.html#backing-properties). -### Expose read-only supertype +To better understand pattern usage and to help in decision-making, +for each pattern occurrence statistics were collected based on open repositories on GitHub. + +### Use cases targeted by the explicit backing field feature -We often do not want our data structures to be modified from outside. It is customary in Kotlin to have -a read-only (e.g. `List`) and a mutable (e.g. `MutableList`) interface to the same data structure. +#### Expose read-only supertype + +We often do not want our data structures to be modified from outside. It is customary in Kotlin to have +a read-only (e.g. `List`) and a mutable (e.g. `MutableList`) interface to the same data structure. Here is an example: ```kotlin -internal val _items = mutableListOf() -val item: List by _items +private val _items = mutableListOf() +val item: List get() = _items ``` -And the new syntax allows us to write: +This use-case is also widely applicable to architecture of reactive applications: + +* [Android `LiveData`](https://developer.android.com/topic/libraries/architecture/livedata) has a `MutableLiveData` counterpart ([code example](https://github.com/elpassion/crweather/blob/9c3e3cb803b7e4fffbb010ff085ac56645c9774d/app/src/main/java/com/elpassion/crweather/MainModel.kt#L14-L22)) +* [Kotlin coroutines `SharedFlow`](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-shared-flow/) has a `MutableSharedFlow`, `StateFlow` has a `MutableStateFlow`, etc. +* [Rx `Observable`](https://github.com/ReactiveX/RxJava/wiki/Subject) has a mutable `Subject` counterpart ([example](https://github.com/gojek/courier-android/blob/40a161cf881bd918cf6580bd5b62b51240295394/chuck-mqtt/src/main/java/com/gojek/chuckmqtt/internal/presentation/base/fragment/FoodMviBaseFragment.kt#L18-L19)). + +_Statistics_: this pattern found in more than 100k files in open repositories on GitHub. + +#### Expose different object + +In this pattern, the primary property does not directly return the object stored in the backup property. +Instead, the value of the backing property is transformed in some way. + +There are two options here. The value of primary property can be evaluated once and stored, or computed on every call. ```kotlin -val items: List - internal field = mutableListOf() +private val _exampleWithStored = ... +val exampleWithStored = _exampleWithStored.someTransformation() + +private val _exampleWithComputed = ... +val exampleWithComputed get() = _exampleWithComputed.someTransformation() ``` -> While the number of lines in this code does not change, the related properties become better grouped together and -repetition of the property name multiple times is avoided. +_Statistics_: found in 70k files for stored property and 12k for computed. -This use-case is also widely applicable to architecture of reactive applications: - -* Android `LiveData` has a `MutableLiveData` counterpart. -* Rx `Observable` has a mutable `Subject` counterpart. -* Kotlin coroutines `SharedFlow` has a `MutableSharedFlow`, etc. +Below are some use cases for this pattern. + +##### Returning read-only view -For example, sample code [from an Android app](https://github.com/elpassion/crweather/blob/9c3e3cb803b7e4fffbb010ff085ac56645c9774d/app/src/main/java/com/elpassion/crweather/MainModel.kt#L14): +In some application it is desirable to expose not just a read-only subtype, but a specially constructed read-only +view that protects the data structure from casting into mutable type. For example, a `MutableStateFlow` has +[`asStateFlow`](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/as-state-flow.html) +extension for that purpose. ```kotlin -private val mutableCity = MutableLiveData().apply { value = "" } -private val mutableCharts = MutableLiveData>().apply { value = emptyList() } -private val mutableLoading = MutableLiveData().apply { value = false } -private val mutableMessage = MutableLiveData() - -val city: LiveData = mutableCity -val charts: LiveData> = mutableCharts -val loading: LiveData = mutableLoading -val message: LiveData = mutableMessage +private val _city = MutableStateFlow("") +val city: StateFlow = _city.asStateFlow() ``` -Becomes: +It is usually more desirable for the primary property to be stored, as in the code snippet above or [this example](https://github.com/wikimedia/apps-android-wikipedia/blob/604007f38e834667f037c475b05f362b92a5575c/app/src/main/java/org/wikipedia/talk/template/TalkTemplatesViewModel.kt#L26-L30). +However, computed property pattern is also quite popular ([example](https://github.com/jellyfin/jellyfin-androidtv/blob/b46f1acc99fc848abb9ef896c9bb2941e9c6e3ff/playback/core/src/main/kotlin/PlayerState.kt#L75-L88)). + +##### Convenient type cast without spelling the full type + +Also, conversion method like `asStateFlow` is used when we want to cast backing property to supertype, +but we don't want to explicitly spell the full type (which is also was [one of the reasons of introducing `asStateFlow`](https://github.com/Kotlin/kotlinx.coroutines/issues/1973#issuecomment-621660861)): ```kotlin -val city: LiveData - field = MutableLiveData().apply { value = "" } -val charts: LiveData> - field = MutableLiveData().apply { value = emptyList() } -val loading: LiveData - field = MutableLiveData().apply { value = false } -val message: LiveData - field = MutableLiveData() +private val _item = MutableStateFlow(ExtremelyLongTypeName.DefaultValue) +val item = _city.asStateFlow() +// shorter than +val item: StateFlow = _city +// or +val item = _city as StateFlow ``` - -> `private` is a default access for a `field` declaration. -In this use-case, an read access to the field from inside the corresponding classes/modules -(where the field is visible) automatically gives access to mutable type, which we call [Smart Type Narrowing](#smart-type-narrowing), -but is seen as a property with read-only type to the outside code. +Apart from rewriting it using explicit backing fields, this use case could be reduced to the simpler form of [previous use-case](#expose-read-only-supertype), +if [underscore operator in type parameters](#underscore-operator-in-type-parameters) was supported. -### Decouple storage type from external representation +##### Hide complex value storage logic -Sometimes a property must be internally represented by a different type for storage-efficiency or architectural reason, -while having a different outside type. For example, API requirements might dictate that the property type -is `String`, but if we know that it always represents a decimal integer, then it can be efficiently stored as such -with custom getter and custom setter. +Sometimes we want to provide a public API for getting the immediate value of a variable, +and hide complex storage logic inside the implementation. +For example, we may store `AtomicInt` and provide api for obtaining instant `Int` value. ```kotlin -var number: String - field: Int = 0 - get() = field.toString() - set(value) { field = value.toInt() } +private val _itemCount = atomic(0) +val itemCount: Int get() = _itemCount.value ``` -### Expose read-only view +It is also possible with other containers: -In some application it is desirable to expose not just a read-only subtype, but a specially constructed read-only -view that protects the data structure from casting into mutable type. For example a `MutableStateFlow` has -[`asStateFlow`](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/as-state-flow.html) -extension for that purpose. +- Other atomic classes ([example](https://github.com/Sorapointa/Sorapointa/blob/31578cb9e460bb4516b51fcef4f0c4af5600caa9/sorapointa-core/src/main/kotlin/org/sorapointa/game/PlayerAvatarComp.kt#L42-L82)) +- `StateFlow`, `LiveData`, etc ([example](https://github.com/c5inco/Compose-Modifiers-Playground/blob/2c7e8c55eeaab15e38696397a84b57475a84a6a3/ideaPlugin/src/main/kotlin/intellij/SwingColors.kt#L44-L52)) +- `ThreadLocal` ([example](https://github.com/corda/corda/blob/a95b854b1eda8a1e9284c0db8ff43b78ea0eb290/core/src/main/kotlin/net/corda/core/serialization/SerializationAPI.kt#L56)) +- `WeakReference` ([example](https://github.com/expo/expo/blob/a502f90d6d9345648157aceb16a78f044669842e/packages/expo-dev-menu/android/src/main/java/expo/modules/devmenu/devtools/DevMenuDevToolsDelegate.kt#L21-L32)) + +##### Access backing property inside -With the proposed syntax it is possible to declare a custom getter: +It is worth emphasizing that in most cases the primary property is only for public use and is not +referenced within the class. +There is no point in doing so, since the backing property usually provides +all the same functionality, as a primary property and even more. + +### Use cases not supported by the explicit backing field feature + +Use cases listed below are not supported to be rewritten with a new feature. +However, they are left in the list to provide a more complete picture of pattern use in the wild, +as well as to illustrate statistic-driven rationale for decision making. + +#### `var` backing property + +It's possible that we want mutable backing property: ```kotlin -val state: StateFlow - get() = field.asStateFlow() - field = MutableStateFlow(State.INITIAL) +private var _prop: T? = null +val prop: T get() = _prop!! ``` -> Note, that `field.asStateFlow()` allocates a wrapper read-only view object and it is called here every time this -property is accessed, allocating a new instance. For some applications this is an acceptable and even preferable -behavior. For others, it is crucial to create this view object once and cache it in a separate field. We don't plan -to address the later use-cases — that's where the backing-field pattern will have to continue to be used. +While in some cases such code can be rewritten with one `lateinit var` + `private set`, +in the following situations we have to use the pattern above: + +1. `T` is primitive type (because `lateinit` on primitive types is not allowed). +2. `prop` must be `open` (because `private set` is not allowed in `open` properties). +3. We want a custom getter or setter. +For example, we want to return some default value in getter while backing property is initialized. +4. Eventually value must be set back to `null` again. For example, [Fragment view bindings in Android](https://developer.android.com/topic/libraries/view-binding#fragments), + which must be initialized in `onCreate()` and set to `null` in `onDestroy()`. +5. We want more permissive visibility of setter (e.g. protected property, internal setter). + +_Statistics_: found in 236k files (mostly case 4). + +#### `lateinit` backing property -For this use-case to become actually usable, the TBD syntax of [Direct Backing Field Access](#direct-backing-field-access) -will need to be added. That's because such applications need to modify the mutable data structure that is stored in the field, -but the rules of [Smart Type Narrowing](#smart-type-narrowing) will not make it directly available. +If for some of the mentioned above reasons it's needed to use `var` backing property, +but it's expected that property is initialized before first use and is always non-null, +`lateinit var` backing property could be used +([example](https://github.com/ProtonMail/proton-mail-android/blob/215edbf4f9a80efc2f683005a4df36f511d272a9/app/src/main/java/ch/protonmail/android/ui/adapter/ClickableAdapter.kt#L79-L85)). -### Access field from outside of getter and setter +_Statistics_: found in 5179 files. -Kotlin allows property field to be accessed from the property's getter and setter using a `field` variable. +#### Non-private visibility of property + +It's possible that backing property has visibility other than `private`. Here are some use-cases: + +- protected - providing common functionality base class ([example](https://github.com/Automattic/pocket-casts-android/blob/c59b40d7acd0ba58e6d8266e880bccea3f666705/modules/services/views/src/main/java/au/com/shiftyjelly/pocketcasts/views/multiselect/MultiSelectHelper.kt#L40-L41)) +- internal - to be able to mutate in neighbour class ([example](https://github.com/meganz/android/blob/21a8dd8da80dfdf1dfed58ad1734a2c76fadac6f/app/src/main/java/mega/privacy/android/app/presentation/photos/timeline/viewmodel/TimelineViewModel.kt#L111-L112)) +- internal - visible for testing ([example](https://github.com/stripe/stripe-android/blob/2f93daa301f57e2690b07d258a872e4541c497e9/identity/src/main/java/com/stripe/android/identity/viewmodel/IdentityViewModel.kt#L179-L180)) + +_Statistics_: found in 3.7k files + +Primary property also might be `protected` or `internal` +([example](https://github.com/meganz/android/blob/21a8dd8da80dfdf1dfed58ad1734a2c76fadac6f/app/src/main/java/mega/privacy/android/app/presentation/logout/LogoutViewModel.kt#L27-L28)). + +#### Delegation + +It's possible that either backing property ([example](https://github.com/b-lam/Resplash/blob/4b13d31134d1c31bd331e92ffe8d410984529212/app/src/main/java/com/b_lam/resplash/ui/upgrade/UpgradeViewModel.kt#L32-L40)) +or primary property ([example](https://github.com/google/accompanist/blob/0d0198d0cab599295f7601cb386d5989d72cc8cd/pager/src/main/java/com/google/accompanist/pager/PagerState.kt#L145)) +is delegated. + +_Statistics_: backing property delegation found in 2693 files, primary property delegation in 1010 files. + +## Design + +### Explicit Backing Fields + +By analogy with the already existing term ["implicit backing fields"](https://kotlinlang.org/docs/properties.html#backing-properties), it is proposed to introduce a syntax +for declaring "explicit backing fields" of properties. +Syntax consists of a keyword `field`, optional type definition and initialization expression +(more detailed [Grammar changes](#grammar-changes) are in the separate section). ```kotlin -class Component { - var status: Status - set(value) { - field = value - notifyStatusChanged() - } -} +private val _city = MutableLiveData() +val city: LiveData get() = _city ``` -This proposal is designed with an idea to provide an explicit syntax to access a property's field from anywhere -inside the corresponding class as if the backing field was declared with `private` access. In this particular example, -any code inside the class `Component` will be able to change the field of the `status` property directly, without invoking the setter. -However, the actual access syntax of such [Direct Backing Field Access](#direct-backing-field-access) is TBD. +Could be rewritten as follows: -### Retrieve property delegate reference +```kotlin +val city: LiveData + field = MutableLiveData() +``` + +### Visibility + +`private` is the default and the only allowed visibility for explicit backing fields +(supporting other visibilities might be considered, see [Future enhancements](#future-enhancements)). -Delegated properties in Kotlin have a backing field that stores a reference to the delegate object and -automatically generate getter and setter implementations that forward accesses to the delegate. -The following Kotlin code: +Visibility of property must be more permissive than explicit backing field visibility. + +### Resolution + +Calls of properties with explicit backing field are resolved to +- the backing field, if property is accessed from the same scope it is declared in + (actually, follows `private` visibility rules as per [specification](https://kotlinlang.org/spec/declarations.html#declaration-visibility)), +- getter, otherwise. ```kotlin -class Data { - val expensive: T by lazy { /*...*/ } +class SomeViewModel { + val city: LiveData + field = MutableLiveData("") + + fun updateCity(newCity: String) { + city.value = newCity // visible as MutableLiveData, calling field + } } + +fun outside(vm: SomeViewModel) { + vm.city // visible as LiveData, calling getter +} ``` -is basically compiled into the following code: +There is no possibility to call a getter instead of field +when the property is accessed from the same scope it is declared in +(might be reconsidered in [Future enhancements](#future-enhancements)). + +### Accessors + +Property, which has explicit backing field, can also specify getter and/or setter. ```kotlin -class Data { - private val expensive$delegate: Lazy = SynchronizedLazyImpl { /*...*/ } - val expensive: T - get() = expensive$delegate.value -} +val counter: Int + field = atomic(0) + get() = field.value ``` -As you can see, `expensive$delegate` here is effectively a backing for the property with the delegate. -The are many use-cases where the direct access to the delegate is need. For example, a `Lazy` type -has [`isInitialized`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-lazy/is-initialized.html) function. -The existing syntax for its access is cumbersome and not type-safe: +When explicit backing field with type `F` for a property with type `P` is declared, +then compiler can derive getter and, for `var` properties, setter implementation automatically: + +* If `F` is a subtype of `P`, the compiler can provide a default getter. +* If `P` is a subtype of `F`, the compiler can provide a default setter. + +If the compiler can not provide a getter, the user must declare it explicitly. +The same applies to setters in case of `var` properties. ```kotlin -fun isExpensiveInitialized() = - (this::value.getDelegate() as Lazy).isInitialized() +val city: String + field = MutableLiveData() + get() = field.value ?: "-" // It is an error if getter is not explicitly specified here ``` -In practice, it means that -[developers have to resort to an backing field pattern](https://stackoverflow.com/questions/42522739/kotlin-check-if-lazy-val-has-been-initialised) -to make it look nice: +### Explicit backing field in combination with property initializer + +It's allowed to have both property initializer and explicit backing field specified. ```kotlin -class Data { - private val expensiveDelegate: Lazy = SynchronizedLazyImpl { /*...*/ } - val expensive: T - get() = expensiveDelegate.value - fun isExpensiveInitialized() = - expensiveDelegate.isInitialized() -} +val city: StateFlow = field.asStateFlow() + field = MutableStateFlow("") ``` -It is expected that TBD [Direct Backing Field Access](#direct-backing-field-access) syntax is going to -provide a direct and simpler way to access the delegate of a delegated property without having to -resort to the backing field pattern. +In this case, the property initializer expression (`field.asStateFlow()` in example above) is evaluated once +immediately after the backing field is initialized and then returned every time the getter of property is called. -## Design +If property with explicit backing field has initializer, it's prohibited to declare accessors. -The proposed design consists of two new ideas: explicit backing fields and smart type narrowing. +> This use case stands out a little from the rest, +since it implies the simultaneous existence of two different stored values for one property, +which sounds dissonant with the current mental image of properties in Kotlin. +However, this is a case where the desire for consistency gives way to the importance of supporting a popular pattern +(see [Expose different object](#expose-different-object)). -### Explicit Backing Fields +### Accessing field inside accessors and initializers + +It's possible to access explicit backing field by calling `field` from inside accessors (like it works now for ordinary +properties) and property initialization expression. + +Backing field assignability is the same as it is now for `field` references in getters and setters: + +* `var` properties have mutable backing fields. +* `val` properties have read-only backing fields. That is, assignment to `field` inside a `val` property getter results in an error. + +The following additional restrictions apply: + +* It's prohibited to reference `field` inside explicit backing field initializer expression. +* Explicit backing field must be referenced through `field` in property initializer, setter and getter (to prevent feature abuse). + +### Type inference + +Explicit type declaration on explicit backing field is optional. If it is not specified, it's inferred from the explicit backing field initialization expression. + +Explicit type declaration on property, which has explicit backing field, is optional. If it is not specified, it's inferred from +the property initialization or from getter. If none of them are specified, the property type is equal to the explicit backing field type. + +It's a warning if the property type is equal to the explicit backing field type. + +### Other restrictions + +There are the following additional semantic restrictions on the property declaration. +Property with explicit backing field: + +* must be final (otherwise calling a field instead of a getter would be undesirable behavior) +* can't be `const` +* can't be `expect` or `external` +* must be initialized right away. It can't be initialized from `init` block + (otherwise we can't differentiate whether initializer expected here or not) +* can't be `var` if it has property initializer (to prevent ambiguity of what should be mutable here) +* can't be extension property + +Neither property with explicit backing field: +* nor its accessors can be `inline` +* nor explicit backing field itself can be `lateinit` (added to [Future enhancements](#future-enhancements)) +* nor explicit backing field itself can be delegated (added to [Future enhancements](#future-enhancements)) + +It is not prohibited for a top-level property to have an explicit backing field. + +## Technical details + +### Grammar changes The grammar for `propertyDeclaration` is extended to support an optional _explicit backing field declaration_ in addition -to the optional `getter` and `setter` (in any order between them). +to the optional `getter` and `setter` (in any order between them). ``` propertyDeclaration ::= @@ -250,181 +386,126 @@ propertyDeclaration ::= | (getter? (semi? field)? (semi? setter)?) | (setter? (semi? field)? (semi? getter)?) | (field? (semi? getter)? (semi? setter)?) - | (field? (semi? setter)? (semi? getter)?) + | (field? (semi? setter)? (semi? getter)?)) -getter ::= - modifiers? 'get' ('(' ')' (':' type)? functionBody)? - -setter ::= - modifiers? 'set' ('(' functionValueParameterWithOptionalType ','? ')' (':' type)? functionBody)? - field ::= - modifiers? 'field` (':' type)? ('=' expression)? + modifiers? 'field' (':' type)? '=' expression ``` -Explicit backing field declaration has an optional visibility, an optional type, and an optional -initialization expression. - -#### Restrictions - -There are the following additional semantic restrictions on the property declaration grammar: +### Java interoperability -* A property with an explicit field declaration cannot have its own initializer. - A property with an explicit field must be initialized with the initialization expression for its - field to clarify the fact, that property initialization goes directly to the field and does not - call property's setter. The property without field initialization expression is considered - uninitialized and is allowed only when it is `lateinit` (see [Lateinit](#lateinit) section for details). -* A property with an explicit backing field must always explicitly specify the type of the property itself. -* Explicit backing field declaration is not allowed for interface properties, for `abstract` properties, and for delegated properties. +For JVM target, property with explicit backing field is compiled into field(-s) and accessors. +Property can be accessed from Java through them in accordance with the following visibility rules: -A backing field type is not required to be explicitly specified: - -* If both backing field type and initialization expression are not specified, then the field type is the same as the property type. -* If backing field type is not specified, but there is an initialization expression, then - the field type is inferred from the type of its backing field initialization expression. -* When both field type and initialization expression are specified, then the type of the former must be - assignable to the latter. - -Backing field assignability is the same as it is now for `field` references in getters and setters: - -* `var` properties have mutable backing fields. -* `val` properties have read-only backing fields. That is, assignment to `field` inside a `val` property getter results in an error. - -#### Accessors - -When explicit backing field with type `F` for a property with type `P` is declared explicitly, -then compiler can derive getter and, for `var` properties, setter implementation automatically: - -* If `P :> F`, the compiler can provide a default getter. -* If `P <: F`, the compiler can provide a default setter. - -If the compiler can not provide a getter, the user must declare it explicitly. -The same applies to setters in case of `var` properties. +* accessors are assigned visibility of property +* field is assigned `private` visibility of explicit backing field ```kotlin -public val flow: SharedFlow - field: MutableSharedFlow? = null - get() { // It is an error if getter is not explicitly specified here - return field ?: run { ... } - } +protected val name: LiveData + field = MutableLiveData() ``` -#### Visibility +So, the property above is visible in Java as: -Only the `private` and `internal` visibilities are allowed for explicit backing fields. -The default field visibility is `private`. +```java +private final MutableLiveData name = MutableLiveData(); -```kotlin -val mutableWithinModule: List - internal field = mutableListOf() +protected final LiveData getName() { + return this.name; +} ``` -The special syntax to explicitly access the backing field from outside the code of property accessors is TBD, -but the field can be implicitly access when it is visible via the [Smart Type Narrowing](#smart-type-narrowing). +`@JvmField` is prohibited on properties with explicit backing field, because in this case a field type becomes exposed, +which makes use of the feature completely pointless. -#### Lateinit +#### Property with initializer compilation -If a property has an explicit backing field declaration, and it needs to be `lateinit`, the modifier must be placed at the `field` declaration. +Property with explicit backing field and initializer is represented by additional +auxiliary synthetic field which is needed for storing the value from property initializer. +Auxiliary field is given the name `propertyName$init` and is `private` regardless of property visibility. +Accessing this field directly from code is not possible (and is not planned), as it must remain an implementation detail. ```kotlin -var someStrangeExample: Int - lateinit field: String - get() = field.length - set(value) { - field = value.toString() - } +val city: StateFlow = field.asStateFlow() + field = MutableStateFlow("") ``` -### Smart Type Narrowing +The example above is compiled into: -The idea behind smart type narrowing is to implicitly access the underlying field, as opposed to the property, -when the field is in scope and when it is safe to do so. For example, expanding on -[Expose read-only subtype](#expose-read-only-subtype) use-case, one can write: +```java +private final MutableStateFlow city = MutableStateFlow(""); +private /* synthetic */ final StateFlow city$init = this.city.asStateFlow(); -```kotlin -class Component { - val items: List - field = mutableListOf() - - fun addItem(item: Item) { - items += item // works - } +public final StateFlow getCity() { + return this.city$init; } - -// outside code -component.items += item // does not compile; cannot add to List ``` -The code above works, because `items` there implicitly refers to the field with type `MutableList`. - -Smart type narrowing works when trying to read the value of the property and all the following conditions are met: +### Reflection -* The backing field is visible from the point of access. -* The property is final (that is, it is not open `open`). -* The property does not have an explicit getter. +Callable reference to property with explicit backing field has type `KProperty` (or its subtypes) where `V` +is type of property (not backing field) regardless of wherever it is accessed from. -> The last rule automatically guarantees that the getter was automatically generated. Together with -> the requirement that the field is not `open`, it means that the compiler knows that the field stores the same -> instance as returned by the getter and that the type of the field is narrower than the type of -> the property (see [Accessors](#accessors) section). +On JVM backing field can be obtained using [`javaField` extension](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect.jvm/java-field.html). -In this case, the type of property read expression is narrowed by the compiler from the type of the property -to the type of its field. +There is no API in Reflection to check whether property has an explicit backing field or obtain any information about it. +Adding such an API might be considered in [Future enhancements](#future-enhancements). -## Alternatives +## Risks -### Initial Proposal +### Unintentionally exposing a field as a return value -Initially, the following syntax was suggested: +The ability to call both a getter and a field with the same name depending on the context of call +introduces a possible error with the unwanted exposing field value (where getter value was meant), +when property is returned in function, especially when function return type is omitted: ```kotlin -private val items = mutableListOf() - public get(): List -``` +class MyViewModel { + val city = field.asStateFlow() + field = MutableStateFlow("") + + fun updateAndReturn(newValue: String): StateFlow { + city.value = newValue + return city // exposed MutableStateFlow instead of read-only wrapper + } -> In above example `items` is `MutableList` when accessed privately inside class and read-only `List` when accessed from outside class. + fun updateAndReturn2(newValue: String) /* MutableStateFlow inferred */ = + city.also { it.value = newValue } +} +``` -In fact, the above syntax brings in an incorrect mental model: it says _'There is a private property `items` with some *part* that declares its public behavior'_. +Although it would be nice to have warnings from the tooling in such cases, +it is unclear in which cases such behavior is desirable and in which it is not. -Attempt to add support for the above syntax led to multiple redundant complications (see the [problems section](https://github.com/matejdro/KEEP/blob/private_public_property_types/proposals/private_public_property_types.md#questions-to-consider) and unclear override mechanics). +## Future enhancements -## Future Enhancements +We strive to keep the design simple and uncluttered, so at this point we put aside functionality, which is rarely used or would complicate language too much. +Of course, we may revise some decisions in the future based on community feedback. +Here is a list of the most obvious future enhancements for the feature: -### Direct Backing Field Access - -We plan to add support for a syntax to explicitly access the property's backing field as well as for property -delegate via some TBD syntax. +1. Make it possible to access getter instead of explicit backing field when the property is accessed from the same scope the property is declared in. This is not supported because usually there is no need for calling getter (see [Access backing property inside](#access-backing-property-inside)). +2. Support other visibilities of explicit backing field (`protected` or `internal`). +3. Support delegation of explicit backing field or property. +4. Support `lateinit` explicit backing field. +5. Support combining mutable explicit backing field and non-mutable property. +Despite its popularity (see [`var` backing property](#var-backing-property)), this use case is not supported in this proposal, because it is impossible for one property to behave as mutable + nullable and +at the same time non-mutable and non-nullable. +6. Add API in Reflection to retrieve information about explicit backing field. -### Protected Fields - -The set of visibilities for an explicitly declared field can be extended to include `protected` -(in addition to `private` and `internal`). This way, subclasses can explicitly or implicitly -(via the [smart type narrowing](#smart-type-narrowing)) reference the field. +### Underscore operator in type parameters -### Mutable Fields for Read-only Properties +It's proposed to make [underscore operator](https://kotlinlang.org/docs/generics.html#underscore-operator-for-type-arguments) +more powerful and support it in type parameters. -Letting a `val` property have a mutable backing field may be useful. -Consider the following snippet from the -[implementation of `lazy` in the Kotlin standard library](https://github.com/JetBrains/kotlin/blob/master/libraries/stdlib/jvm/src/kotlin/util/LazyJVM.kt#L57): +This is a separate language feature, yet worth mentioning here, +as it can help rewrite properties with explicit backing field in a better way in some cases +(see [Convenient type cast without spelling the full type](#convenient-type-cast-without-spelling-the-full-type)). ```kotlin -// backing field pattern -private var _value: Any? = UNINITIALIZED_VALUE +val item = field.asStateFlow() // if we don't need read-only wrapper... + field = MutableStateFlow(ExtremelyLongTypeName.Default) -override val value: T - get() { - // initializes _value backing field on the first access - } +// ...we could rewrite it like that without losing conciseness +val item: StateFlow<_> // inferred StateFlow + field = MutableStateFlow(ExtremelyLongTypeName.Default) ``` - -## Change log - -This section records changes to this KEEP. - -### Prototype in K2 preview with Kotlin 1.7.0 - -Prototype of this proposal has been delivered in Kotlin 1.7.0 as a part of K2 compiler preview. -In order to try out his new feature you need to enable the K2 compiler with -`-Xuse-k2` command line option and enable this language feature with `-XXLanguage:+ExplicitBackingFields`. -The implementation is not stable yet and will generate pre-release binary. -There is no IDE support yet.