Background

ARKit

A framework provided by Apple which processes and provides sensor data from the IMU for an Augmented Reality experience to work. Such as motion tracking, localizing device, session capture, and image analysis/processing. ARKit uses ARAnchor and it's limited subtypes (ARPlaneAnchor, ARImageAnchor, ARObjectAnchor) to keep track of their points of interest in the ARSession. For more in-depth information refer to the ARKit Documentation.

RealityKit

While ARKit handles the above it does not render any content into the scene. RealityKit is a framework and API above ARKit that follows the Entity-Component Architectural pattern. Entity are 3D models that can have Components or behaviors applied to them. This handles establishing a scene that 3D content and audio can be anchored to from the ARKit anchors. RealityKit has it's own notion of AnchorEntity that have overlapping functionality with ARKit Anchors yet with the purpose of anchoring 3D content. For more in-depth information refer to the RealityKit Documentation.

Reality Composer

Creation of Augmented Reality experiences without a visual understanding of the scene can be difficult. Hardcoding and measuring the xyz locations of content can be tedious. Reality Composer is an app for iOS and Mac with functionalities to compose AR scenes around a chosen anchor type. 3D content can be placed and given conditional actions and audio. A benefit of using this app is that the scene can be previewed in AR and edited in real time using an iOS device.

SwiftUI

3D content design has many challenges. It can be time consuming, expensive, require additional skills, and generates large files. 3D Content is also difficult to make dynamic changes such as animations through interaction and conditions. As a solution, instead of placing 3D content into the scene. Invisible Entities are placed as children relative to the chosen Anchor. Their locations are projected from the world scene onto the screen and SwiftUI Views are rendered at those locations.