I am a beginner in the field of augmented reality, working on applications that create plans of buildings (floor plan, room plan, etc with accurate measurements) using a smartphone. So I am researching about the best AR SDK which can be used for this. There are not many articles pitting Vuforia against ARCore and ARKit.
Please suggest the best SDK to use, pros and cons of each.
Updated: May 25, 2021.
Google ARCore allows you build apps for Android and iOS. With Apple ARKit you can build apps for iOS; with Apple RealityKit – for iOS and macOS. And a great old PTC Vuforia was designed to create apps for Android, iOS and Universal Windows Platform.
A crucial Vuforia's peculiarity is that it uses ARCore/ARKit technology if the hardware it's running on supports it, otherwise Vuforia uses its own AR technology and engine, known as software solution without dependant hardware.
When developing for Android OEM smartphones, you may encounter with an unpleasant issue: devices from different manufacturers need a sensors’ calibration in order to observe the same AR experience. Luckily, Apple gadgets have no such drawback because all sensors used there were calibrated under identical conditions.
Let me put first things first.
ARCore is based on the three main fundamental concepts : Motion Tracking, Environmental Understanding and Light Estimation. ARCore allows a supported mobile device to track its position and orientation relative to the world in 6 degrees of freedom (6DoF) using special technique called Concurrent Odometry and Mapping. COM helps us detect the size and location of horizontal, vertical and angled tracked surfaces. Motion Tracking works robustly thanks to optical data coming from a RGB camera at 60 fps, combined with inertial data coming from gyroscope and accelerometer at 1000 fps, and data coming from ToF sensor at 60 fps. Surely, ARKit, Vuforia and other AR libraries operate almost the same way.
When you move your phone through the real environment, ARCore tracks a surrounding space to understand where a smartphone is, relative to the world coordinates. At tracking stage ARCore "sows" so called feature points. These feature points are visible through RGB camera, and ARCore uses them to compute phone's location change. The visual data then must be combined with measurements from IMU (Inertial Measurement Unit) to estimate the position and orientation of the ArCamera over time. ARCore looks for clusters of feature points that appear to lie on horizontal, vertical or angled surfaces and makes these surfaces available to your app as planes (we call this technique Plane Detection). After detection process you can use these planes to place 3D objects in your scene. Virtual geometry with assigned shaders will be rendered by ARCore's companion – Sceneform supporting a real-time Physically Based Rendering (a.k.a. PBR) engine – Filament.
Notwithstanding the above, at this moment Sceneform repository has been archived and it no longer actively maintaining by Google. The last released version was Sceneform 1.17.1. It's up to you what 3D engine to use now.
ARCore's environmental understanding lets you place 3D objects with a correct depth occlusion in a way that realistically integrates with the real world. For example, you can place a virtual cup of coffee on the table using Depth hit-testing and ArAnchor.
ARCore can also define lighting parameters of a real environment and provide you with the average intensity and color correction of a given camera image. This data lets you light your virtual scene under the same conditions as the environment around you, considerably increasing the sense of realism.
Current ARCore version has such a significant APIs as Raw Depth API and Full Depth API, Lighting Estimation, Augmented Faces, Augmented Images, Instant Placement, Debugging Tools, 365-days Cloud Anchors, Recording and Playback and Multiplayer support. The main advantage of ARCore in Android Studio over ARKit in Xcode is Android Emulator allowing you run and debug AR apps using virtual device.
This table presents the difference between Raw Depth API and Full Depth API:
|------------|--------------------|--------------------|
| | "Raw Depth API" | "Full Depth API" |
|------------|--------------------|--------------------|
| Accuracy | Awesome | Bad |
|------------|--------------------|--------------------|
| Coverage | Not all pixels | All pixels |
|------------|--------------------|--------------------|
| Distance | 0.5 to 5.0 m | 0 to 8.0 m |
|------------|--------------------|--------------------|
ARCore is older than ARKit. Do you remember Project Tango released in 2014? Roughly speaking, ARCore is just a rewritten Tango SDK. But a wise acquisition of FlyBy and MetaIO helped Apple not only to catch up but also significantly overtake. Suppose it's good for AR industry.
The latest version of ARCore supports OpenGL ES acceleration, and integrates with Unity, Unreal, and Web applications. At the moment the most powerful and energy efficient chipsets for AR experience on Android platform are Snapdragon 888 (5nm), Exynos 2100 (5nm) and Kirin 1000 (5nm) – now Google and Huawei are friends again.
ARCore price: FREE.
|------------------------------|------------------------------|
| "ARCore PROs" | "ARCore CONs" |
|------------------------------|------------------------------|
| iToF and Depth API support | No Body Tracking support |
|------------------------------|------------------------------|
| Quick Plane Detection | Cloud Anchors hosted online |
|------------------------------|------------------------------|
| Long-distance-accuracy | Lack of rendering engines |
|------------------------------|------------------------------|
| ARCore Emulator in AS | Poor developer documentation |
|------------------------------|------------------------------|
| High-quality Lighting API | No external camera support |
|------------------------------|------------------------------|
| A lot of supported devices | Poor Google Glass API |
|------------------------------|------------------------------|
Here's ARCore code's snippet written in Kotlin:
private fun addNodeToScene(fragment: ArFragment,
anchor: Anchor,
renderable: Renderable) {
val anchorNode = AnchorNode(anchor)
anchorNode.setParent(fragment.arSceneView.scene)
val modelNode = TransformableNode(fragment.transformationSystem)
modelNode.setParent(anchorNode)
modelNode.setRenderable(renderable)
modelNode.localPosition = Vector3(0.0f, 0.0f, -3.0f)
fragment.arSceneView.scene.addChild(anchorNode)
modelNode.select()
}
ARKit was released in June 2017 and just two years later it became very popular. Like its competitors, ARKit also uses special technique, called Visual Inertial Odometry, to very accurately track the world around your device. VIO is quite similar to COM found in ARCore. There are also three similar fundamental concepts in ARKit: World Tracking, Scene Understanding (which includes four stages: Plane Detection, Ray-Casting, Light Estimation, Scene Reconstruction), and Rendering with a great help of ARKit companions – SceneKit framework, that’s actually an Apple 3D game engine since 2012, RealityKit framework specially made for AR and written in Swift from scratch (released in 2019), and SpriteKit framework with its 2D engine (since 2013).
VIO fuses RGB sensor data at 60 fps with Core-Motion data (IMU) at 1000 fps. In addition to that, SceneKit can render 3D geometry at 60 fps. It should be noted that due to a very high energy impact (because of an enormous burden on CPU and GPU), your iPhone's battery will be drained pretty quickly.
ARKit has a handful of useful approaches for robust tracking and accurate measurements. Among its arsenal you can find easy-to-use functionality for saving and retrieving ARWorldMaps. World map is an indispensable "portal" for Persistent and Multiuser AR experience that allows you to come back to the same environment filled with the same chosen 3D content just before the moment your app became inactive. Support for simultaneous front and back camera capture and support for collaborative sessions, that allows us to share World Maps, are also great.
There are good news for gamers: up to 6 people are simultaneously able to play the same AR game, thanks to MultipeerConnectivity framework. For 3D geometry you could use a brand-new USDZ file format, developed and supported by Pixar. USDZ is a good choice for sophisticated 3D models with lots of PBR shaders and animations. Also you can use the following 3D formats for ARKit.
ARKit not only can help you track a position and orientation of your device relative to the world in 6 DoF, but also help you perform People and Objects Occlusion technique (based on alpha and depth channels' segmentation), LiDAR Scene Reconstruction, Body Motion Capture tracking, Vertical and Horizontal Planes detection, Image detection, 3D Object detection and 3D Object scanning. With People and Objects Occlusion tool your AR content realistically passes behind and in front of real world entities, making AR experiences even more immersive. Also, Realistic reflections, that use machine learning algorithms and Face-based AR experience that allows to track up to 3 faces at a time are available for you.
Using ARKit and iBeacons, you assist an iBeacon-aware application to know what room it’s in, and show a right 3D content chosen for that room. Working with ARKit you should intensively exploit ARAnchor class and all its subclasses, the same way you’ve been using it in ARCore.
Pay particular attention to RealityKit's satellite – Reality Composer application that's now a part of Xcode. This brand-new app helps you build 3D scenes for AR. Scenes built in Reality Composer can be packed with dynamics, simple animations and PBR shaders. Reality Composer can be installed on iOS and iPadOS as a standalone app.
For creating AR apps built on the latest versions of ARKit 4.0, including brand-new LiDAR scanner support, you need macOS 11 Big Sur, Xcode 12 and device running iOS 14. A sad news is – all ARKit 4.0 top features are restricted to devices powered by Apple A12 chipset and higher. Also ARKit 4.0 is a worthy candidate to marry Metal framework for GPU acceleration. Don’t forget that ARKit tightly integrates with Unity and Unreal. At the moment the most powerful and energy efficient chipset for AR experience is A14 Bionic (5nm).
ARKit price: FREE.
|------------------------------|------------------------------|
| "ARKit PROs" | "ARKit CONs" |
|------------------------------|------------------------------|
| LiDAR and Depth API support | No AR glasses support |
|------------------------------|------------------------------|
| Stable 6 DoF World Tracking | No auto-update for Anchors |
|------------------------------|------------------------------|
| Collaborative Sessions | iOS / Chipsets Restrictions |
|------------------------------|------------------------------|
| WorldMaps, iBeacon-awareness | No ARKit Simulator in Xcode |
|------------------------------|------------------------------|
| 4 rendering technologies | No external camera support |
|------------------------------|------------------------------|
| Rich developer documentation | Quickly drains your battery |
|------------------------------|------------------------------|
Here's ARKit code's snippet written in Swift:
func renderer(_ renderer: SCNSceneRenderer,
didAdd node: SCNNode,
for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
let planeNode = tableTop(planeAnchor)
node.addChildNode(planeNode)
}
func tableTop(_ anchor: ARPlaneAnchor) -> SCNNode {
let x = CGFloat(anchor.extent.x)
let z = CGFloat(anchor.extent.z)
let tableNode = SCNNode()
tableNode.geometry = SCNPlane(width: x, height: z)
tableNode.position = SCNVector3(anchor.center.x, 0, anchor.center.z)
return tableNode
}
You should carefully look at RealityKit that was introduced in WWDC 2019. There’s been a lot of hype around it since then. RealityKit allows you create AR experiences for iOS/iPadOS and VR experiences for mobiles and macOS. This high-level framework works with .usdz assets as well as with .rcproject and .reality file formats which you can import from standalone macOS or iOS app – Reality Composer (RC). Cupertino software engineers built RealityKit from the ground for augmented reality apps that you can create with no repetitive code. It works with Swift from scratch – there’s no Objective-C legacy. And, of course, RealityKit shines not only with SwiftUI and UIKit but with Metal too.
RealityKit framework has several fundamental blocks on which RealityKit's scenes are based: a class ** ModelEntity** and a class AnchorEntity that automatically tracks target (unlike in ARKit). ModelEntity is built on MeshResource and Materials.
RealityKit framework gives you a rich set of building blocks to work with AR and VR: new declarative Swift syntax, 3D primitives (at the moment box, plane, sphere and text), PBR materials with textures, occlusion material and video material, lighting fixtures (directional, spot and point) with realistic ray-traced shadows, spacial audio processing, different anchors types (body, camera, face, image, object, horizontal plane, vertical plane, raycastResult, ARAnchor and world), simplified setup for collaborative sessions, robust animations' and physics' setup, indispensable AI and ML built-in features and many other useful things.
Reality Composer application gives you a simple and intuitive UI for constructing 3D scenes for Augmented Reality experiences. It has a royalty free library with downloadable 3D assets that allow you construct sophisticated 3D scenes with animation, audio, and dynamics which contain a thorough description of how these objects behave. You can also export your composition as a lightweight AR Quick Look experience that lets users place and preview a content. In Reality Composer app you can start your project using one of five anchor types: horizontal, vertical, image, face and object – which corresponds to desired type of tracking.
RealityKit and Reality Composer price: FREE.
|------------------------------|------------------------------|
| "RealityKit PROs" | "RealityKit CONs" |
|------------------------------|------------------------------|
| Can create AR apps w/o ARKit | Intensive usage of CPU/GPU |
|------------------------------|------------------------------|
| Very little boilerplate code | iOS 13+, macOS 10.15+ only |
|------------------------------|------------------------------|
| Suitable for AR/VR projects | Start lagging on old devices |
|------------------------------|------------------------------|
| Robust API for RC scenes | Limited shaders capabilities |
|------------------------------|------------------------------|
| Asynchronous asset loading | Lack of Apple documentation |
|------------------------------|------------------------------|
| Autoupdating tracking target | No AR glasses support |
|------------------------------|------------------------------|
Here's RealityKit code's snippet written in Swift:
override func viewDidLoad() {
super.viewDidLoad()
let textAnchor = try! SomeText.loadTextScene()
let textEntity: Entity = textAnchor.realityComposer!.children[0]
var textMC: ModelComponent = textEntity.children[0].components[ModelComponent]!
var material = SimpleMaterial()
material.baseColor = .color(.yellow)
textMC.materials[0] = material
textMC.mesh = .generateText("Hello, RealityKit")
textAnchor.realityComposer!.children[0].children[0].components.set(textMC)
arView.scene.anchors.append(textAnchor)
}
One more important part of Apple's AR ecosystem is Reality Converter app. Now, instead of using a command line conversion tool, you can use a Reality Converter. The brand-new app makes it easy for you to convert, view and customize .usdz 3D objects on Mac. Simply drag-and-drop common 3D file formats, such as .obj, .gltf or .usd, to view the converted .usdz result, customize material properties with your own textures and edit file metadata. You can even preview your .usdz object under a variety of lighting and environment conditions with built-in Image-Based Lighting (IBL) options.
In October 2015 PTC acquired Vuforia from Qualcomm for $65 million. Take into consideration that Qualcomm launched Vuforia in 2010. So Vuforia is an older sister in AR family. Big sister is watching you, guys! ;)
In November 2016 Unity Technologies and PTC announced a strategic collaboration to simplify AR development. Since then they work together integrating new features of the Vuforia AR platform into the Unity game engine. Vuforia can be used with such development environments as Unity, MS Visual Studio, Apple Xcode and Android Studio. It supports a wide range of smartphones, tablets and AR smart glasses, such as HoloLens, Magic Leap, Vuzix M400, and ODG R7.
Vuforia Engine boasts approximately the same main capabilities that you can find in the latest versions of ARKit but also it has its own features, such as Model Targets with Deep Learning, VISLAM for markerless AR experience and External Camera support for iOS, new experimental APIs for ARCore and ARKit and support for industry latest AR glasses. The main advantage of Vuforia over ARKit and ARCore that it has a wider list of supported devices and it supports the development of Universal Windows Platform apps for Intel-based Windows 10 devices, including Microsoft Surface and HoloLens.
Vuforia has a standalone version and a version baked directly into Unity. It has the following functionality:
Smart Terrain, enables digital content to be placed on floors and tabletop surfaces;Vuforia Fusion is a capability designed to solve the problem of fragmentation in AR enabling technologies such as cameras, sensors, chipsets, and software frameworks like ARKit. With Vuforia Fusion, your app will automatically provide the best experience possible with no extra work required on your end.
Vuforia Engine Area Targets enable developers to use an entire space, be it a factory floor or retail store, as AR target. Using a first supported device, a Matterport Pro2 camera, developers can create a detailed 3D scan of a desired location. Locations are recommended to be indoors, mostly static, and no larger than 1,000 sqm (around 10,000 sqft). Once the scan produces a 3D model it can be converted into an Area Target with the Vuforia Area Target Generator. This target can then be brought into Unity.
Vuforia API allows for a Static or Adaptive mode. When the real-world model remains stationary, like a large industrial machine, implementing the Static API will use significantly less processing power. This enables a longer lasting and higher performance experience for those models. For objects that won’t be stationary the Adaptive API allows for a continued robust experience.
The External Camera feature is a part of the Vuforia Engine Driver Framework. External Camera provides a new perspective on what’s possible with Augmented Reality. It allows Vuforia Engine to access external video sources beyond the camera equipped in phones and tablets. By using an independent camera, developers can create an AR experience that offers a first-person view from toys, robots or industrial tools.
Occlusion Management is one of the key features for building a realistic AR experience. When you're using Occlusion Management, Vuforia Engine detects and tracks targets, even when they’re partially hidden behind everyday barriers, like your hand. Special occlusion handling allows apps to display graphics as if they appear inside physical objects.
Vuforia supports Metal acceleration for iOS devices. Also you can use Vuforia Samples for your projects. For example: the Vuforia Core Samples library includes various scenes using Vuforia features, including a pre-configured Object Recognition scene that you can use as a reference and starting point for Object Recognition application.
Here's AR Foundation code's snippet written in C#:
private void UpdatePlacementPose() {
var screenCenter = Camera.main.ViewportToScreenPoint(new Vector3(0.5f, 0.5f));
var hits = new List<ARRaycastHit>();
arOrigin.Raycast(screenCenter, hits, TrackableType.Planes);
placementPoseIsValid = hits.Count > 0;
if (placementPoseIsValid) {
placementPose = hits[0].pose;
var cameraForward = Camera.current.transform.forward;
var cameraBearing = new Vector3(cameraForward.x,
0,
cameraForward.z).normalized;
placementPose.rotation = Quaternion.LookRotation(cameraBearing);
}
}
Vuforia SDK Pricing Options:
Free license – you just need to register for a free Development License Key
Basic license ($42/month, billed annually) – For Students
Basic + Cloud license ($99/month) – For Small Businesses
Agency Package (personal price) – 5 short-term licenses
Pro license (personal price) – For All Companies Types
Here are Pros and Cons.
|------------------------------|------------------------------|
| "Vuforia PROs" | "Vuforia CONs" |
|------------------------------|------------------------------|
| Supports Android, iOS, UWP | The price is not reasonable |
|------------------------------|------------------------------|
| A lot of supported devices | Poor developer documentation |
|------------------------------|------------------------------|
| External Camera support | SDK has some issues and bugs |
|------------------------------|------------------------------|
| Webcam/Simulator Play Mode | Doesn't support Geo tracking |
|------------------------------|------------------------------|
CONCLUSION :
There are no vital limitations when developing with PTC Vuforia compared to ARCore and ARKit. Vuforia is an old great product and it supports a wider list of Apple and Android devices (even those that are not officially supported) and it supports several latest models of AR glasses.
But in my opinion, ARKit with a Reality Family toolkit (RealityKit, Reality Composer and Reality Converter) have an extra bunch of useful up-to-date features that Vuforia and ARCore just partially have. ARKit personally has a better short-distance measurement accuracy within a room than any ARCore compatible device has, without any need for calibration. This is achieved through the use of Apple LiDAR dToF scanner. ARCore now uses iToF cameras with Raw Depth API. Both iToF and LiDAR allow you create a high-quality virtual mesh with OcclusionMaterial for real-world surfaces at scene understanding stage. This mesh is ready-for-measurement and ready-for-collision. With iToF and dToF sensors, frameworks instantly detect non-planar surfaces and surfaces with no-features-at-all, such as texture-free white walls in a poorly-lit rooms.
If you implement iBeacon tools, ARWorldMaps and support for GPS – it will help you eliminate many tracking errors accumulated over time. And ARKit's tight integration with Vision and CoreML frameworks makes a huge contribution to a robust AR toolset. Integration with Apple Maps allows ARKit put GPS Location Anchors outdoors with a highest possible precision at the moment.
Vuforia's measurement accuracy depends on what platform you're developing for. Some of Vuforia features are built on top of the tracking engine (ARKit or ARCore). Even popular Vuforia Chalk application uses ARKit positional tracker.
Excellent info. However would like to add few points based on the experience in using ARCore and ARkit. With respect to mapping, ARCore has the ability to manage larger maps compared to ARkit. ARcore tracks more feature point compared to ARkit. Another point is ARKit differentiates the horizontal and vertical detection of surfaces better than ARcore.
