Contact

ARKit + SpriteKit Tutorial: Build Your First Augmented Reality App

14 minutes to read

Apps like Pokémon Go and ARMeasure blend virtual objects into the real world through your camera. That is augmented reality, and on iOS, it is built on a framework called ARKit. This tutorial covers ARKit's core building blocks and walks through building a real, tappable AR app using SpriteKit. No prior AR experience is required.

What Is Augmented Reality?

Augmented reality means placing virtual objects into a real-world environment as viewed through a camera. Pokémon Go popularized the concept for a mainstream audience, even though it wasn't built with ARKit. ARMeasure is a more precise example: it turns your iPhone's camera into a virtual measuring tape, letting you measure real objects without a physical ruler. More advanced apps go further still: the open-source ARKit + CoreLocation project combines AR with location data to annotate real buildings and display turn-by-turn walking directions overlaid on the camera feed, the same idea Google Maps later shipped as "Live View."

All of these apps share the same underlying principle: virtual content, anchored convincingly to the real world, viewed through a live camera feed.

What Is ARKit?

ARKit is Apple's framework for building augmented reality apps on iOS, introduced in iOS 11. It ships with Xcode, so no separate installation is needed.

ARKit apps only run on devices with an A9 processor or later: iPhone 6s and newer, iPhone SE, and several iPad Pro, Air, and mini generations. You also cannot test ARKit apps in the iOS Simulator. You will need a physical, compatible device for any real testing.

How ARKit Works

ARKit creates the illusion that virtual objects belong in the real world using a technique called Visual Inertial Odometry (VIO). It continuously analyzes camera sensor data alongside motion data from the device to track its position and orientation in space. From that tracking, ARKit maintains a virtual 3D coordinate system mapped onto the real world. Placing virtual content into that coordinate space and blending it with the live camera image is what produces the AR effect.

The Core ARKit Classes

A handful of classes do the real work in ARKit:

  • ARSKView / ARSCNView: the view that renders your AR experience. Which one you use depends on your content technology: ARSKView for 2D content via SpriteKit, ARSCNView for 3D content via SceneKit. Both automatically render the live camera feed as the scene background.
  • ARSession: manages motion data and camera image processing for the view. Every AR experience requires one; it's the engine synthesizing sensor data into a coherent 3D understanding of the world.
  • ARFrame: represents a single captured video frame plus its position-tracking data. Access the latest one via the session's currentFrame property.
  • ARWorldTrackingConfiguration: the session configuration that tracks a device's position and motion relative to the real world using the rear camera, and supports plane detection and hit testing. On A9+ devices, it tracks movement with six degrees of freedom: three rotation axes (roll, pitch, yaw) plus three translation axes (x, y, z movement). Older devices are limited to the three rotation axes only.
  • ARAnchor: represents the position and orientation of a real or virtual object relative to the camera. To place any virtual object in AR space, you create an anchor and add it to the session.

Understanding worldAlignment

One ARWorldTrackingConfiguration property worth knowing: worldAlignment, which controls how ARKit's coordinate system maps to real-world device motion.

  • .gravity (the default): the y-axis aligns with the direction of gravity. This is the right choice for most AR experiences.
  • .gravityAndHeading: similar to .gravity, but the x and z axes are locked to compass directions (north/south, east/west) rather than the device's initial heading.
  • .camera: all axes lock to match the camera's current orientation. With this setting, virtual objects visibly rotate as you turn the device sideways, since the coordinate frame moves with the camera itself.

Step 1: Create Your First ARKit App (No Code Required)

Open Xcode and create a new project using the Augmented Reality App template.

  • Name the project (e.g., ARKitDemo), using your own organization identifier.
  • Choose Storyboard for the interface.
  • For Content Technology, use the default: SpriteKit.

ARKit is designed to integrate with Apple's existing rendering engines rather than reinvent one. You can pair it with SpriteKit for 2D content, SceneKit for 3D content, or Metal for hardware-accelerated custom rendering. This tutorial uses SpriteKit, the simplest starting point.

Deploy this template to a compatible device without any code changes. Tap anywhere on screen, and an emoji appears, anchored in the real world in front of you. Xcode's template wires up everything needed to demonstrate the core AR loop.

Step 2: Understand How the Demo App Places Objects

Opening the storyboard, you'll find an ARSKView (the SpriteKit-flavored AR view) already added to the view controller, rendering the live camera feed as its background.

In ViewController.swift, the AR session starts when the view is about to appear:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)

    let configuration = ARWorldTrackingConfiguration()

    sceneView.session.run(configuration)
}

The actual object placement happens in a separate SKScene subclass, inside touchesBegan, which fires on every tap:

override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
    guard let sceneView = self.view as? ARSKView else { return }

    if let currentFrame = sceneView.session.currentFrame {

        var translation = matrix_identity_float4x4

        translation.columns.3.z = -0.2

        let transform = simd_mul(currentFrame.camera.transform, translation)

        let anchor = ARAnchor(transform: transform)

        sceneView.session.add(anchor: anchor)

    }
}

The goal here is simple: place the object 0.2 meters directly in front of the camera.

  • matrix_identity_float4x4 is Swift's built-in representation of a 4×4 identity matrix, the neutral starting point for any transformation.
  • Setting translation.columns.3.z = -0.2 turns that identity matrix into a translation matrix, encoding "move 0.2 meters along the z-axis" (negative z is forward, in front of the device).
  • simd_mul(currentFrame.camera.transform, translation) multiplies the camera's current position/orientation by that translation, producing the exact real-world coordinate 0.2 meters ahead of wherever the camera currently is.
  • That resulting transform becomes an ARAnchor, which is added to the session with sceneView.session.add(anchor:).

Providing the actual visual content happens through a separate delegate method. ViewController adopts ARSKViewDelegate, and whenever a new anchor is added, ARKit calls:

func view(_ view: ARSKView, nodeFor anchor: ARAnchor) -> SKNode? {

    let labelNode = SKLabelNode(text: "😊")

    labelNode.horizontalAlignmentMode = .center

    labelNode.verticalAlignmentMode = .center

    return labelNode
}

Any anchor added to the session triggers this method, and whatever SKNode you return is what actually renders at that anchor's position. In the template's case, that is a text label containing an emoji.

Step 3: Make Objects Interactive (Tap to Remove)

A static AR object is a good start, but real apps usually need interaction. To make tapping an existing emoji fade it out and remove it, while tapping empty space still places a new one:

override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
    guard let sceneView = self.view as? ARSKView else { return }

    if let touchLocation = touches.first?.location(in: self) {

        if let node = nodes(at: touchLocation).first as? SKLabelNode {

            let fadeOut = SKAction.fadeOut(withDuration: 1.0)

            node.run(fadeOut) {
                node.removeFromParent()
            }

            return
        }
    }

    if let currentFrame = sceneView.session.currentFrame {

        var translation = matrix_identity_float4x4

        translation.columns.3.z = -0.2

        let transform = simd_mul(currentFrame.camera.transform, translation)

        let anchor = ARAnchor(transform: transform)

        sceneView.session.add(anchor: anchor)
    }
}

The added logic is standard SpriteKit, not anything AR-specific. nodes(at:) checks whether the tap location hits an existing node. If it hits a label node (an existing emoji), it runs a one-second fade-out animation and removes the node afterward, returning early so a new object is not placed on the same tap. If the tap misses every existing node, execution falls through to the original placement logic. Once an object exists in AR space, interacting with it is just standard SpriteKit node handling. The "AR" part was only in getting the object placed in the first place.

Try It Yourself: Beyond Emoji

Two natural extensions to practice with once the base app is working:

Replace the emoji with an image. Swap SKLabelNode for SKSpriteNode loaded from a texture:

let texture = SKTexture(imageNamed: "fat-bird")

let spriteNode = SKSpriteNode(texture: texture)

Resize it as needed, since imported art is often far larger than appropriate for AR placement:

spriteNode.size = CGSize(
    width: spriteNode.size.width * 0.1,
    height: spriteNode.size.height * 0.1
)

Randomize between multiple sprites, and animate them. Instead of a single fixed image, pick randomly from several sprite sheets on each tap, and use SpriteKit's frame animation support to create a flying/walking loop:

let flyingAction = SKAction.repeatForever(
    SKAction.animate(with: birdFrames, timePerFrame: 0.1)
)

Both of these are pure SpriteKit techniques. Once an object is anchored, everything about how it looks and behaves is standard SpriteKit.

Summary

ARKit's core loop works like this: an ARSession continuously tracks the real world using camera and motion data, ARAnchor objects mark where virtual content should exist in that space, and a rendering technology (SpriteKit, in this tutorial) provides the actual visual content for each anchor. Once you understand those three pieces, everything else including interaction, animation, and multiple object types is standard SpriteKit work layered on top. The natural next step is moving from 2D content to true 3D objects using SceneKit, where AR experiences start to feel three-dimensional.

Frequently Asked Questions

Can I test an ARKit app in the iOS Simulator? No. AR features rely on the camera and motion sensors, which the Simulator cannot provide. You need a physical device with an A9 processor or later.

Do I need SceneKit to use ARKit? No. ARKit integrates with multiple rendering technologies: SpriteKit for 2D content, SceneKit for 3D content, or Metal for custom rendering. This tutorial uses SpriteKit because it is the simplest starting point for 2D AR content.

What does ARWorldTrackingConfiguration track? It tracks the device's position and motion relative to the real world using the rear camera, supporting up to six degrees of freedom (three rotation axes plus three translation axes) on supported hardware, and enables features like plane detection and hit testing.

Why does the demo app use matrix math just to place an object? 3D position and orientation are represented as 4x4 transformation matrices in ARKit. Multiplying the camera's current transform by a translation matrix is how you compute a point some distance in front of wherever the camera currently is. The underlying goal in this case is just moving 0.2 meters forward.

All articles