[SR-15757] [Windows] Cannot inherit from class 'InputStream'

[lxbndr]

Previous ID	SR-15757
Radar	None
Original Reporter	@lxbndr
Type	Bug

Attachment: Download

Environment

Windows 10

compnerd.org Swift version 5.5.2-dev (LLVM d461fd52e58eca6, Swift e6979d24987b522)

Additional Detail from JIRA

Votes	0
Component/s	Compiler, Foundation
Labels	Bug
Assignee	@compnerd
Priority	Medium

md5: 349f7f16ee058b9b9b9d1effa84062f2

Issue Description:

It is impossible to subclass Foundation.InputStream on the 5.5 release branch (as well as on the main branch). The issue is reproducible with a few lines of code:

import Foundation
public class CustomStream: InputStream {}

Running compiler directly:

swiftc -parse-as-library -emit-library -sdk %SDKROOT% x06.swift -o x06.dll

Produces the following error:

x06.swift:3:14: error: cannot inherit from class 'InputStream' because it has overridable members that could not be loaded
public class CustomStream: InputStream {}
             ^

Additional observations

• Rebuilding Foundation with -enable-library-evolution makes it possible to compile the sample code

• Rebuilding Foundation with enabled testability also helps

[compnerd]

+@millenomi

[compnerd]

My suspicion is that we are seeing some behavioural differences due to the @_implementationOnly import of CoreFoundation which was something that we did as CoreFoundation should not be exposed. However, in the process some internals may have accidentally been removed from serialization into the interface and the vtable entry therefore devirtualized, resulting in the failure to subclass InputStream.

[millenomi]

What's happening here is that Windows is the first platform where CoreFoundation is not exposed, and thus entirely `_implementationOnly`. There is a fileprivate constructor in InputStream that takes a CF type, which apparently ends up in the vtable 🙁

The solution here is to turn any constructor of the form:

fileprivate init(_readStream: CFReadStream) { 
   self._something = process(_readStream)
}

into:

fileprivate init(_readStream: AnyObject) { 
   self._something = process(unsafeBitCast(_readStream, to: CFReadStream.self))
}

[millenomi]

This occurs outside of Windows, it's just masked by source compatibility requiring us to expose the CoreFoundation module publicly.

[compnerd]

Exploring this a bit further: I think that there is a set of cases that are impacted here. Rather than trying to think through this, I decided for an empirical approach. I believe that the following should identify the full set of issues.

 import Foundationpublic class SNSAffineTransform: NSAffineTransform {}
public class SBundle: Bundle {}
public class SByteCountFormatter: ByteCountFormatter {}
#if false
public class SDateComponentsFormatter: DateComponentsFormatter {}
#endif
public class SDateFormatter: DateFormatter {}
public class SDateIntervalFormatter: DateIntervalFormatter {}
public class SEnergyFormatter: EnergyFormatter {}
public class SFileHandle: FileHandle {}
public class SPipe: Pipe {}
public class SFileManager: FileManager {}
public class SDirectoryEnumerator: FileManager.DirectoryEnumerator {}
public class SFormatter: Formatter {}
public class SHost: Host {}
public class SISO8601DateFormatter: ISO8601DateFormatter {}
public class SJSONEncoder: JSONEncoder {}
public class SJSONDecoder: JSONDecoder {}
public class SJSONSerialization: JSONSerialization {}
public class SLengthFormatter: LengthFormatter {}
public class Srmatter: MassFormatter {}
#if false
public class SMeasurementFOrmatter: MeasurementFormatter {}
#endif
public class SNSArray: NSArray {}
public class SNSMutableArray: NSMutableArray {}
public class SNSAttributedString: NSAttributedString {}
public class SNSCache<KeyType: AnyObject, ObjectType: AnyObject>: NSCache<KeyType, ObjectType> {}
public class SNSCalendar: NSCalendar {
    required convenience init?(coder aDecoder: NSCoder) {}
}
public class SNSCharacterSet: NSCharacterSet {}
public class SNSMutableCharacterSet: NSMutableCharacterSet {}
public class SNSCoder: NSCoder {}
#if false
public class SNSComparisonPredicate: NSComparisonPredicate {}
#endif
public class SNSCompoundPredicate: NSCompoundPredicate {}
public class SNSData: NSData {}
public class SNSMutableData: NSMutableData {}
public class SNSDate: NSDate {}
public class SNSDateInterval: NSDateInterval {}
public class SNSDateComponents: NSDateComponents {}
public class SNSDeciamlNumber: NSDecimalNumber {
    required convenience init(floatLiteral value: Double) {}
    required convenience init(booleanLiteral value: Bool) {}
    required convenience init(integerLiteral value: Int) {}
    required convenience init(bytes buffer: UnsafeRawPointer, objCType type: UnsafePointer<Int8>) {}
}
public class SNSDecimalNumberHandler: NSDecimalNumberHandler {}
public class SNSDictionary: NSDictionary {}
public class SNSMutableDictionary: NSMutableDictionary {}
public class SNSEnumerator: NSEnumerator {}
public class SNSError: NSError {}
#if false
public class SNSExpression: NSExpression {}
#endif
public class SNSIndexPath: NSIndexPath {
    required convenience init?(coder aDecoder: NSCoder) {}
}
public class SNSIndexSet: NSIndexSet {}
public class SNSMutableIndexSet: NSMutableIndexSet {}
public class SNSKeyedArchiver: NSKeyedArchiver {}
public class SNSKeyedUnarchiver: NSKeyedUnarchiver {}
public class SNSLocale: NSLocale {}
public class SNSLock: NSLock {}
public class SNSConditionLock: NSConditionLock {}
public class SNSRecursiveLock: NSRecursiveLock {}
public class SNSCondition: NSCondition {}
public class SNSMeasurement: NSMeasurement {}
public class SNSNotification: NSNotification {}
public class SNotificationCenter: NotificationCenter {}
public class SNSNull: NSNull {}
public class SNSNumber: NSNumber {
    required convenience init?(coder aDecoder: NSCoder) {}
    required convenience init(integerLiteral value: Int) {}
    required convenience init(booleanLiteral value: Bool) {}
    required convenience init(floatLiteral value: Double) {}
    required convenience init(bytes buffer: UnsafeRawPointer, objCType: UnsafePointer<Int8>) {}
}
public class SNSObject: NSObject {}
public class SNSOrderedSet: NSOrderedSet {
    required convenience init?(coder aDecoder: NSCoder) {}
    required convenience init(arrayLiteral elements: Any...) {}
}
public class SNSMutableOrderedSet: NSMutableOrderedSet {
    required convenience init?(coder aDecoder: NSCoder) {}
    required convenience init(arrayLiteral elements: Any...) {}
}
public class SNSPersonNameComponents: NSPersonNameComponents {}
public class SNSPredicate: NSPredicate {}
public class SNSRegularExpression: NSRegularExpression {}
public class SNSSet: NSSet {}
public class SNSMutableSet: NSMutableSet {}
public class SNSCountedSet: NSCountedSet {}
public class SNSSortDescriptor: NSSortDescriptor {}
public class SNSString: NSString {
    required convenience init?(coder aDecoder: NSCoder) {}
    required convenience init(string aString: String) {}
    required convenience init(unicodeScalarLiteral value: StaticString) {}
    required convenience init(extendedGraphemeClusterLiteral value: StaticString) {}
}
public class SNSMutableString: NSMutableString {
    required convenience init?(coder aDecoder: NSCoder) {}
    required convenience init(unicodeScalarLiteral value: StaticString) {}
    required convenience init(extendedGraphemeClusterLiteral value: StaticString) {}
}
public class SNSTextCheckingResult: NSTextCheckingResult {}
#if false
public class SNSOrthography: NSOrthography {}
#endif
public class SNSTimeZone: NSTimeZone {
    required convenience init?(coder aDecoder: NSCoder) {}
}
public class SNSURL: NSURL {}
public class SNSURLComponents: NSURLComponents {}
public class SNSURLQueryItem: NSURLQueryItem {}
public class SNSUUID: NSUUID {}
public class SNSValue: NSValue {
    required convenience init(bytes value: UnsafeRawPointer, objCType type: UnsafePointer<Int8>) {}
    required convenience init?(coder aDecoder: NSCoder) {}
}
public class SNotificationQueue: NotificationQueue {}
public class SNumberFormatter: NumberFormatter {}
public class SOperation: Operation {}
public class SBlockOperation: BlockOperation {}
public class SOperationQueue: OperationQueue {}
#if false
public class SPersonNameComponentsFormatter: PersonNameComponentsFormatter {}
#endif
public class SPort: Port {}
#if false
public class SMessagePort: MessagePort {}
#endif
#if false
public class SNSMachPort: NSMachPort {}
#endif
public class SSocketPort: SocketPort {}
public class SPortMessage: PortMessage {}
public class SProcess: Process {}
public class SProcessInfo: ProcessInfo {}
public class SProgress: Progress {}
public class SPropertyListEncoder: PropertyListEncoder {}
public class SPropertyListDecoder: PropertyListDecoder {}
public class SPropertyListSerialization: PropertyListSerialization {}
public class SRunLoop: RunLoop {}
public class SScanner: Scanner {}
public class SStream: Stream {}
public class SInputStream: InputStream {}
public class SOutputStream: OutputStream {}
public class SThread: Thread {}
public class STimer: Timer {}
public class SUnitConverter: UnitConverter {}
public class SUnitConverterLinear: UnitConverterLinear {}
public class SUnit: Unit {}
public class SDimension: Dimension {}
public class SUserDefaults: UserDefaults {}

This identified the following cases where problems occur:

 reduced.swift:5:14: error: cannot inherit from class 'Bundle' because it has overridable members that could not be loaded
public class SBundle: Bundle {}
             ^
reduced.swift:10:14: error: cannot inherit from class 'DateFormatter' because it has overridable members that could not be loaded
public class SDateFormatter: DateFormatter {}
             ^
reduced.swift:11:14: error: cannot inherit from class 'DateIntervalFormatter' because it has overridable members that could not be loaded
public class SDateIntervalFormatter: DateIntervalFormatter {}
             ^
reduced.swift:19:14: error: cannot inherit from class 'ISO8601DateFormatter' because it has overridable members that could not be loaded
public class SISO8601DateFormatter: ISO8601DateFormatter {}
             ^
reduced.swift:32:14: error: cannot inherit from class 'NSCalendar' because it has overridable members that could not be loaded
public class SNSCalendar: NSCalendar {
             ^
reduced.swift:35:14: error: cannot inherit from class 'NSCharacterSet' because it has overridable members that could not be loaded
public class SNSCharacterSet: NSCharacterSet {}
             ^
reduced.swift:42:14: error: cannot inherit from class 'NSData' because it has overridable members that could not be loaded
public class SNSData: NSData {}
             ^
reduced.swift:43:14: error: cannot inherit from class 'NSMutableData' because it has overridable members that could not be loaded
public class SNSMutableData: NSMutableData {}
             ^
reduced.swift:44:14: error: cannot inherit from class 'NSDate' because it has overridable members that could not be loaded
public class SNSDate: NSDate {}
             ^
reduced.swift:57:14: error: cannot inherit from class 'NSError' because it has overridable members that could not be loaded
public class SNSError: NSError {}
             ^
reduced.swift:67:14: error: cannot inherit from class 'NSKeyedUnarchiver' because it has overridable members that could not be loaded
public class SNSKeyedUnarchiver: NSKeyedUnarchiver {}
             ^
reduced.swift:68:14: error: cannot inherit from class 'NSLocale' because it has overridable members that could not be loaded
public class SNSLocale: NSLocale {}
             ^
reduced.swift:77:14: error: cannot inherit from class 'NSNumber' because it has overridable members that could not be loaded
public class SNSNumber: NSNumber {
             ^
reduced.swift:95:14: error: cannot inherit from class 'NSRegularExpression' because it has overridable members that could not be loaded
public class SNSRegularExpression: NSRegularExpression {}
             ^
reduced.swift:115:14: error: cannot inherit from class 'NSTimeZone' because it has overridable members that could not be loaded
public class SNSTimeZone: NSTimeZone {
             ^
reduced.swift:118:14: error: cannot inherit from class 'NSURL' because it has overridable members that could not be loaded
public class SNSURL: NSURL {}
             ^
reduced.swift:119:14: error: cannot inherit from class 'NSURLComponents' because it has overridable members that could not be loaded
public class SNSURLComponents: NSURLComponents {}
             ^
reduced.swift:126:14: error: cannot inherit from class 'NotificationQueue' because it has overridable members that could not be loaded
public class SNotificationQueue: NotificationQueue {}
             ^
reduced.swift:127:14: error: cannot inherit from class 'NumberFormatter' because it has overridable members that could not be loaded
public class SNumberFormatter: NumberFormatter {}
             ^
reduced.swift:141:14: error: cannot inherit from class 'SocketPort' because it has overridable members that could not be loaded
public class SSocketPort: SocketPort {}
             ^
reduced.swift:143:14: error: cannot inherit from class 'Process' because it has overridable members that could not be loaded
public class SProcess: Process {}
             ^
reduced.swift:148:14: error: cannot inherit from class 'PropertyListSerialization' because it has overridable members that could not be loaded
public class SPropertyListSerialization: PropertyListSerialization {}
             ^
reduced.swift:149:14: error: cannot inherit from class 'RunLoop' because it has overridable members that could not be loaded
public class SRunLoop: RunLoop {}
             ^
reduced.swift:152:14: error: cannot inherit from class 'InputStream' because it has overridable members that could not be loaded
public class SInputStream: InputStream {}
             ^
reduced.swift:153:14: error: cannot inherit from class 'OutputStream' because it has overridable members that could not be loaded
public class SOutputStream: OutputStream {}
             ^
reduced.swift:154:14: error: cannot inherit from class 'Thread' because it has overridable members that could not be loaded
public class SThread: Thread {}
             ^
reduced.swift:155:14: error: cannot inherit from class 'Timer' because it has overridable members that could not be loaded
public class STimer: Timer {}
             ^

[compnerd]

Added a test case that should reproduce the behaviour on any platform. The root of the issue is that we do need the values to be potentially in the vtable (if they are overridden locally). The values therefore are serialised out to the swiftmodule. However, on deserialisation, we need to treat the holes specially as they cannot be overridden.

[compnerd]

swiftlang/swift#40972 helps to a certain extent, there is one last case which isn't solved involving enum storage.