[RCP-45] Legacy and Deprecated Data Elements

[darnjo]

Discussed in #36

^{Originally posted by paulhethmon August 1, 2022}
Back in Confluence last month I asked:

I’m looking for a way to handle and express lookups that are no longer in use but exist in older listing data. As an example the field BusinessType might have a lookup of Video Store. 20 years ago that was important and used, today, not so much. I would like to expose the Video Store lookup in my metadata as a searchable item (albeit legacy) but I can’t allow it to be used on a new listing.
My search-fu this morning didn’t give me any leads on this, but it certainly seems like a situation many implementations would have.
Is there something in the standard I missed? Do we need to fill the gap?

Assigned To: @paulhethmon

[alifemove]

Slightly related I think, I'd be interested to know if we can specify somehow whether a field is filterable or not.

[darnjo]

Good question, @alifemove. We have added a field for this called SearchableYN in the new Field Resource proposal (RCP-042). See: #76

[alifemove]

Bringing this from the discussion

DeprecatedDate. A date value of when a field became deprecated

I'm in favor of this, but I'd like to change the definition to A date value of when a field became or will become deprecated. We're already versioning between DD versions idk that we want to also version between our own schema versions and this feels like an easier way to communicate changes than a new schema version.

After that the only reason to have the mentioned FieldStatus would be for upcoming because everything else can be inferred (deprecated and date in the future? going away soon, deprecated and date in the past? already gone, not deprecated? its active), so maybe it's not needed at that point?
I get wanting to inform consumers that a field will become available in the future, but if were doing that then it'd be nice to have another date to specify when it will become available, otherwise 'upcoming' is arbitrary and means different things for different companies/people. Upcoming next quarter? Next year? Next fiscal year? Next month?
At that point feels like the dates are just becoming a LastModified field or something...

[grispin]

On the certification side of this, We should exclude deprecated fields from certification process for off market listings to reduce the impact of large historical datasets to future DD version migrations. All active listings should be included in certification and should not have deprecated used.

This would decrease the migration impact and effort to clients and data providers by allowing legacy data to remain untouched. This keeps the timestamp updates and now the entity event resource from being flooded with updates with no new good content.

[darnjo]

If data elements show up in any records from a provider, they need to be declared in metadata regardless if they're deprecated. Deprecated only means that no new records will use those values. Not that they aren't part of the data set.

…

On Wed, Oct 23, 2024, 08:21 Geoff Rispin ***@***.***> wrote: On the certification side of this, We should exclude deprecated fields from certification process for off market listings to reduce the impact of large historical datasets to future DD version migrations. All active listings should be included in certification and should not have deprecated used. This would decrease the migration impact and effort to clients and data providers by allowing legacy data to remain untouched. This keeps the timestamp updates and now the entity event resource from being flooded with updates with no new good content. — Reply to this email directly, view it on GitHub <#97 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAECWPWRYDELWL2ZF4M7CHTZ465G5AVCNFSM6AAAAABQPCU2WOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZSGYYDENZUG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[grispin]

Completely agree. This just about how bring explicit about deprecation means within the testing rules of certification. We have not documented how deprecation would impact certification testing in detail yet.
We should explicitly state that legacy models, fields and lookups values that were removed/changed in the RESO standards between versions will not fail certification if they are flagged as deprecated in the metadata and no active resource records are using the deprecated elements.