A 2D AR object always faces the camera, no matter how you move your phone. That is a dead giveaway you are looking at a flat sprite in 3D space, not a real object. This tutorial fixes that by pairing ARKit with SceneKit to place true 3D models in the real world, models you can walk around and view from any angle, and then goes further by detecting real-world flat surfaces and letting users tap to place objects on them.
Why SceneKit Instead of SpriteKit for 3D?
SpriteKit renders 2D content, which is why a 2D AR object always turns to face the viewer (there is no "behind" to see). SceneKit, by contrast, is a full 3D graphics framework. Pairing SceneKit with ARKit lets you add real 3D models to an AR scene, so as you move around them, you actually see their sides and back, exactly as a physical object would behave.
What You'll Learn
- How Xcode's built-in ARKit + SceneKit demo works
- Where to source free or premium 3D models
- How to convert a 3D model into a SceneKit-compatible format
- How to build an ARKit app from scratch (not just the template)
- How to detect a horizontal plane and visualize it
- How to let users tap to place multiple 3D objects on that plane
Step 1: Inspect the Built-In ARKit + SceneKit Demo
Create a new Xcode project using the Augmented Reality App template with SceneKit as the content technology, and run it without any code changes. Xcode generates a working demo that renders a floating jet aircraft in AR space, viewable from any angle.
Under the hood, the demo connects an ARSCNView outlet (the AR-specific counterpart to SceneKit's standard SCNView) to load and display a scene:
@IBOutlet var sceneView: ARSCNView!
// Create a new scene
let scene = SCNScene(named: "art.scnassets/ship.scn")!
// Set the scene to the view
sceneView.scene = sceneARKit automatically maps this SceneKit scene onto real-world space and renders whatever objects the scene file contains. Opening ship.scn in Xcode's scene editor reveals a scene graph, a hierarchical tree of nodes (here, a ship node containing shipMesh and emitter child nodes). Selecting any node in the graph exposes its position, scale, and other attributes directly in the Node inspector without requiring code to move or resize it.
You can also add new elements this way: dragging a 3D Text object into the scene, setting its text content, and adjusting its Position and Scale values in the inspector renders custom 3D text alongside the aircraft, entirely through the visual scene editor.
Step 2: Source a 3D Model
Once you are ready to move beyond the built-in demo model, you will need your own 3D asset. Two sources for free and premium models:
- SketchFab (sketchfab.com)
- TurboSquid (turbosquid.com)
Models are typically distributed in OBJ, FBX, or DAE format. SceneKit needs a compatible format to load a model. DAE (Collada) and Alembic files work natively, since Xcode automatically converts them to SceneKit's internal compressed scene format while keeping the original file extension. OBJ and FBX files need to be converted first.
Step 3: Convert an OBJ Model to DAE Using Blender
If your model comes as an .obj file (commonly paired with an .mtl materials file describing its textures), you will need to convert it before Xcode can use it. Blender, free open-source 3D software for Mac, Windows, and Linux, handles this conversion in a few steps:
- Open Blender and go to File → Import → Wavefront (.obj), then select your
model.objfile. - Blender automatically adds a bounding cube around the imported model. Right-click it and delete it, since it's not part of the actual model.
- Export the cleaned-up model via File → Export → Collada (Default) (.dae), saving it as something like
robot.dae.
You can preview the resulting .dae file directly in Finder without opening Xcode at all, which is a fast way to confirm the conversion worked before importing it into a project.
Step 4: Build an ARKit App From Scratch
If the built-in template does not give you enough control, here is how to wire up an ARKit + SceneKit app manually.
- Create a new project using the App template (Storyboard interface), and give it a name.
- In the storyboard, remove the default view and drag an ARKit SceneKit View into the view controller from the Object Library.
- Import both frameworks and connect the view to an outlet:
import SceneKit
import ARKit
@IBOutlet var sceneView: ARSCNView!- Create a SceneKit Catalog (right-click the project → New File → SceneKit Catalog), name it
art.scnassets, and drag your converted.daefile into it. - Load and display the scene in
viewDidLoad():
override func viewDidLoad() {
super.viewDidLoad()
sceneView.showsStatistics = true
let scene = SCNScene(named: "art.scnassets/robot.dae")!
sceneView.scene = scene
}- Start and pause the AR session at the appropriate lifecycle points:
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
sceneView.session.run(configuration)
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
sceneView.session.pause()
}- Add camera permissions in Info.plist: a
Privacy - Camera Usage Descriptionentry with a user-facing explanation, plusarkitlisted under Required device capabilities, so the App Store restricts the app to ARKit-supported devices.
Deploy to a real device, and your custom 3D model renders in the real world.
Step 5: Detect Horizontal Planes
Placing a static model is a good start, but real AR apps usually need to understand the environment first, starting with detecting flat surfaces like floors and tables.
Enable plane detection by setting a single property on your existing configuration:
configuration.planeDetection = .horizontal(ARKit also supports vertical plane detection for surfaces like walls, though this tutorial focuses on horizontal.)
Whenever ARKit detects a plane, it calls a delegate method. Adopt ARSCNViewDelegate via an extension:
extension ViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
print("Surface detected!")
}
}Then connect the delegate and optionally enable a visual debug overlay in viewDidLoad():
sceneView.delegate = self
sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints]The feature points debug option renders small yellow dots representing the notable visual features ARKit is tracking, a useful way to see what the system is "looking at" while you test.
Step 6: Visualize the Detected Plane
A console message confirms detection is working, but a translucent overlay on the surface makes the feature visible to users. The didAdd delegate method passes both the new SCNNode and its corresponding ARAnchor. Cast the anchor to ARPlaneAnchor to access plane-specific data like size and position:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let planeAnchor = anchor as? ARPlaneAnchor {
let plane = SCNPlane(
width: CGFloat(planeAnchor.planeExtent.width),
height: CGFloat(planeAnchor.planeExtent.height)
)
plane.materials.first?.diffuse.contents = UIColor(red: 90/255, green: 200/255, blue: 250/255, alpha: 0.50)
let planeNode = SCNNode(geometry: plane)
planeNode.position = SCNVector3(planeAnchor.center.x, 0.0, planeAnchor.center.z)
planeNode.eulerAngles.x = -Float.pi / 2.0
node.addChildNode(planeNode)
}
}Two details worth knowing:
- The plane's size comes from
planeAnchor.planeExtent. ARKit tells you exactly how large the detected surface is, so your overlay matches it. - The rotation (
eulerAngles.x = -Float.pi / 2.0) is needed becauseSCNPlanedefaults to a vertical orientation, while a floor or tabletop is horizontal.
Step 7: Keep the Plane Updated as ARKit Refines It
ARKit doesn't detect a surface once and stop. It continuously refines the plane's size and position as it gathers more visual data, calling a separate didUpdate delegate method each time. Without handling this, you'll end up with multiple overlapping, stale plane overlays instead of one clean, updating surface.
A clean approach is to encapsulate the plane visualization in its own SCNNode subclass:
class PlaneNode: SCNNode {
private var anchor: ARPlaneAnchor!
private var plane: SCNPlane!
init(anchor: ARPlaneAnchor) {
super.init()
self.anchor = anchor
self.plane = SCNPlane(
width: CGFloat(anchor.planeExtent.width),
height: CGFloat(anchor.planeExtent.height)
)
self.plane.materials.first?.diffuse.contents = UIColor(red: 90/255, green: 200/255, blue: 250/255, alpha: 0.50)
self.geometry = plane
self.position = SCNVector3(anchor.center.x, 0.0, anchor.center.z)
self.eulerAngles.x = -Float.pi / 2.0
}
required init?(coder aDecoder: NSCoder) {
super.init(coder: aDecoder)
}
func update(anchor: ARPlaneAnchor) {
self.anchor = anchor
plane.width = CGFloat(anchor.planeExtent.width)
plane.height = CGFloat(anchor.planeExtent.height)
self.position = SCNVector3(anchor.center.x, 0.0, anchor.center.z)
}
}Then track every active plane by its anchor identifier in the view controller:
private var planes: [UUID: PlaneNode] = [:]
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let planeAnchor = anchor as? ARPlaneAnchor {
let planeNode = PlaneNode(anchor: planeAnchor)
planes[anchor.identifier] = planeNode
node.addChildNode(planeNode)
}
}
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor,
let plane = planes[planeAnchor.identifier] else { return }
plane.update(anchor: planeAnchor)
}ARAnchor.identifier gives each detected plane a stable, unique key, so the dictionary always resolves an update to the correct existing overlay instead of creating a duplicate.
Step 8: Let Users Tap to Place 3D Objects on the Plane
The final piece: instead of loading a model automatically at launch, let the user tap anywhere on a detected plane to place one.
First, switch to an empty starting scene and register a tap gesture in viewDidLoad():
let scene = SCNScene()
let tapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(addRobot(recognizer:)))
sceneView.addGestureRecognizer(tapGestureRecognizer)Then handle the tap by converting the 2D touch location into a 3D real-world position using raycasting:
@objc func addRobot(recognizer: UITapGestureRecognizer) {
let tapLocation = recognizer.location(in: sceneView)
guard let query = sceneView.raycastQuery(
from: tapLocation,
allowing: .existingPlaneInfinite,
alignment: .horizontal
) else { return }
let hitResults = sceneView.session.raycast(query)
guard let hitResult = hitResults.first else { return }
guard let scene = SCNScene(named: "art.scnassets/robot.dae") else { return }
let node = SCNNode()
for childNode in scene.rootNode.childNodes {
node.addChildNode(childNode)
}
node.position = SCNVector3(
hitResult.worldTransform.columns.3.x,
hitResult.worldTransform.columns.3.y,
hitResult.worldTransform.columns.3.z
)
node.scale = SCNVector3(0.1, 0.1, 0.1)
sceneView.scene.rootNode.addChildNode(node)
}A few things to note:
raycastQueryreplaces the olderhitTestAPI (deprecated as of iOS 14). It casts a virtual ray outward from the tap location on screen until it intersects a detected plane, letting you translate a 2D screen tap into a precise 3D world position.- The loop over
scene.rootNode.childNodesis needed because many 3D models, this robot included, are made of multiple nodes rather than one single mesh. Adding every child node to a common parent node lets you scale and position the whole model as one unit. - The 0.1 scale factor shrinks the model to 10% of its original size. 3D models are frequently imported at a scale that is far too large for believable AR placement, so this kind of adjustment is common.
Step 9: Fix Orientation Issues
After placing a model, you may notice it partially sinking into the surface or facing away from the camera, usually because the model's origin point isn't centered the way you'd expect. Two targeted fixes handle this for a model whose base sits below its own origin:
node.position = SCNVector3(
hitResult.worldTransform.columns.3.x,
hitResult.worldTransform.columns.3.y + 0.35,
hitResult.worldTransform.columns.3.z
)
node.rotation = SCNVector4(0, 1, 0, Float.pi)The first line lifts the model slightly above the detected surface; the second rotates it 180 degrees around the y-axis so it faces the camera correctly. Exact values will vary depending on how your specific model was authored. Treat these as a starting point and adjust by eye.
Summary
Pairing ARKit with SceneKit gives you genuine 3D content in augmented reality, models with real depth that behave as physical objects do as a user moves around them. The core workflow covered here (sourcing and converting a model, building the AR scene manually, detecting and visualizing planes, keeping those planes updated, and using raycasting to let users place objects interactively) is the same foundation behind most real-world AR apps, from furniture placement tools to interactive games. Animated models (via services like Mixamo) are a natural next step.
Frequently Asked Questions
Why does a 2D AR object always face the camera, but a 3D one does not? A 2D sprite has no depth, so there is nothing to see from behind. It renders facing the viewer by design. A true 3D SceneKit model has geometry on every side, so moving the camera around it reveals its sides and back like a real object.
Do I need to convert every 3D model before using it in ARKit? Only if it is not already in a SceneKit-compatible format. DAE and Alembic files are handled automatically by Xcode. OBJ and FBX files need to be converted first. Blender's Collada export is a free, reliable way to do that.
What replaced hitTest for detecting taps on AR surfaces? raycastQuery combined with session.raycast(query), introduced as the recommended replacement starting in iOS 14. It offers more precise and configurable surface intersection than the older hitTest API.
Why do detected planes keep overlapping or duplicating? ARKit continuously updates a detected plane's size and position, not just its initial detection. If your code only handles the didAdd delegate method and ignores didUpdate, you will accumulate stale overlays instead of one plane that refines itself over time.