Place a digital chair on a well lit wooden floor and a current phone can hold it steady enough to look convincing. Point the camera at a blank white wall and tracking may weaken because there are fewer visual details to follow. Leave the room, move the furniture, and return later. The app may not recognize the place. Current AR systems still depend heavily on the lighting, surroundings, and device.

The industry can prepare a demonstration around good lighting, a compatible device, a known room, and a short session. A public product has to work in dim restaurants, on crowded pavements, and on older phones. It must also cope with moving people, poor reception, prescription lenses, and users who have never been taught how the software expects them to move.

There is no single invention that will perfect AR. The camera has to know where it is. The software has to understand what it sees. The display has to place an image in front of the eye without becoming heavy, hot, dim, or exhausting. People need a reliable way to control it. The system also has to respect everyone who happens to be in view.

Different AR devices solve different problems

A phone uses its rear camera to show the real world on a screen, then draws digital content over the video. It is affordable because the user already owns the screen, battery, processor, cameras, and mobile connection. Most people also know how to use it. The disadvantage is that a person must hold it up, and the view of the world is limited to a small rectangle.

A video headset such as Apple Vision Pro also shows camera video on internal displays. It can block the outside world completely, which gives software far greater control over contrast and lighting. Current models can render detailed spatial images, but the user is still looking at screens through a headset. Camera latency, display quality, weight, and the feeling of being separated from nearby people all matter.

Transparent AR glasses take a different approach. The user sees the real world directly through the lenses while a small projector adds light to the view. This preserves a direct view of the surroundings, but it creates a severe optical problem. The display must be bright enough to compete with daylight and wide enough to cover a useful part of vision. It must also fit inside a frame that people will willingly wear.

Some lightweight glasses avoid that problem by offering a small display for messages, directions, or camera previews rather than trying to fill the room with stable digital objects. Meta Ray-Ban Display, for example, uses a monocular display with roughly 600 by 600 pixels and a field of view near 20 degrees. Meta describes it as a display for brief interactions rather than continuous immersion.1 It serves a different purpose from a pair of glasses that places three dimensional objects throughout a room.

Each category solves a different problem and accepts different limitations. Counting all of them as the same product gives a misleading picture of the industry's progress. A small notification display says little about the readiness of a wide, bright, binocular AR display. A powerful headset does not show that the same experience can run all day on a small battery.

Tracking can fail in ordinary rooms

An AR device must estimate its position many times every second. Current phones usually do this with visual inertial odometry. Motion sensors report how the device is turning and accelerating. The camera follows recognizable points in the scene, such as corners, edges, and patches of texture. The software combines both streams to estimate movement.

The method is good enough to place an object on a floor and keep it there while the user walks around. It is not equally reliable everywhere. Apple tells ARKit developers that tracking can be limited by a blank wall, poor light, excessive motion, or a camera that is covered. The company also warns that a detected plane begins as an estimate and may change as the device learns more about the surface.2

The result is visible to the user. A virtual box may slide slightly as the camera moves. An object placed on a table may sink into it when the estimated plane is corrected. Fast camera movement creates blurred frames, so the system temporarily loses details it had been following. Repeated small errors become drift.

Returning to a saved scene is harder. The device has to match what it sees now with a map created earlier. A room can look different after sunset, after chairs have moved, or after seasonal decorations have been removed. Apple notes that relocalization is less likely to succeed when the lighting or the environment has changed. An app can remain in a relocalizing state without eventually finding the old coordinate system.3

Better cameras and motion sensors will reduce these errors, but hardware alone is not enough. A dependable system needs to combine camera data with depth, satellite positioning, nearby radio signals, and a map of known places when those sources are available. It also needs to tell the application how certain it is. If confidence falls, the app should pause a precise interaction or ask the user to scan the area again. Quietly allowing an object to move is the worst response because the user cannot tell whether the world changed or the software made a mistake.

Outdoor AR needs more than a GPS coordinate

A location on a map and a location in an AR camera are not the same thing. GPS.gov says that a typical smartphone is accurate within a radius of 4.9 metres under open sky. Buildings, bridges, trees, atmospheric conditions, and receiver quality can make the result worse.4

A five metre circle may be adequate for telling someone which park entrance to use. It is not adequate for placing a digital key on one particular brick. The phone also needs a reliable heading. Magnetometers can be disturbed by metal structures and nearby electronics, so an object may appear in approximately the right place but on the wrong side of the user.

Visual positioning narrows the error by comparing the camera view with previously mapped images of the area. Google combines its Visual Positioning System with GPS in the ARCore Geospatial API. Developers can check whether visual positioning is available at the user's current location before offering an experience that depends on it.5 Coverage is not universal, and a place can become difficult to recognize when construction, weather, crowds, or vegetation alter the view.

A precise outdoor experience needs several levels of location. Satellite positioning can bring a person to the general area. Visual positioning can identify the street or building. A local image, marker, Bluetooth beacon, or ultra wideband signal can establish the final position when much greater precision is required. The application should report the level it has reached. A treasure hunt, navigation guide, or repair tool should not claim exact alignment when it has only a rough geographic fix.

A camera can measure a room without understanding it

Tracking tells the device how it moved. Scene understanding tells it what is around the user. That distinction matters when a digital ball is meant to bounce off a sofa or a virtual character is supposed to walk behind a real table.

Without depth information, software may draw the virtual object on top of everything in the camera image. A person walking between the camera and the object can appear behind it. The object may also pass through furniture because the app has no useful model of the furniture's shape.

Current systems can build a depth map, which estimates the distance from the camera to points in the image. Google's ARCore Depth API combines movement between camera frames with a hardware depth sensor when one is available. It works best when the scene is about half a metre to five metres away. The user has to move the phone before useful depth data becomes available, and plain surfaces can still produce imprecise estimates.6

Support also varies by device. Google reported in May 2026 that more than 88 percent of active ARCore devices supported its Depth API. That is broad coverage, but not complete coverage, and some phones have dedicated depth hardware while others infer depth from camera movement.7 A developer either has to exclude unsupported phones or design a less convincing fallback.

A mesh of the room is only an approximate shape. Software still has to recognize which surface is a floor, a seat, a window, or something else. It must update that understanding when a person moves a chair. Reflective glass, mirrors, thin objects, repetitive patterns, darkness, and rapidly moving people remain difficult.

ARKit can estimate reflections from its camera view, but it cannot see every direction and does not update the resulting environment texture in real time. Apple advises against using that estimate for an accurate mirror.8 The device can make a shiny virtual object look plausible, but it does not have a complete and current model of the light in the room.

A more complete spatial model would combine geometry, object recognition, movement, material properties, and lighting. It would have to update quickly without sending a constant video record of the room to a remote server. Producing that model requires substantial computing power within the heat and battery limits of a mobile device.

Glasses still force difficult physical compromises

Apple Vision Pro with the M5 chip weighs between 750 and 800 grams with its Light Seal and Dual Knit Band. Its separate battery weighs another 353 grams and provides up to two and a half hours of general use. The current Canadian price begins at $4,999.9 It provides high resolution displays and substantial computing power, but its weight and price put it in a different category from ordinary eyewear.

Snap's fifth generation Spectacles reduced the device to 226 grams and offered a 46 degree diagonal field of view. The published continuous battery life was 45 minutes. Snap released that model to developers rather than the general public.10

Meta's Orion prototype reached an approximately 70 degree field of view in a glasses shaped device, but it moved part of the computing into a separate wireless puck and used a wristband for input. Meta explicitly presented Orion as a product prototype rather than something consumers could buy.11

The companies made different choices because each device has a limited allowance for display size, brightness, processing, battery life, heat, weight, and price. Improving one of those qualities can make another worse.

Transparent displays add another constraint. The optical system has to send an image into the eye while allowing the person to see through the lens. A wider field of view requires more demanding optics. A larger eyebox makes the image easier to see when the glasses shift on the face, but enlarging it can reduce brightness. Reviews of current display research describe persistent tradeoffs among field of view, resolution, brightness, colour, eyebox size, eye relief, and a frame people can wear comfortably.12

There is also a problem with black. A transparent display can add coloured light, but it cannot create a dark object simply by emitting no light. The real scene remains visible through that part of the lens. Strong contrast in daylight may require an additional layer that selectively dims the outside view, which adds more optical complexity and consumes power.

Several components must improve together before full AR can fit into ordinary glasses. More efficient projectors and waveguides can reduce the required battery. Better chips can perform spatial mapping with less heat. Batteries need greater energy density, and frames need to spread weight across the head without thick arms or pressure points. Some products may need to place part of the computing in a phone or a separate module until those components improve.

The image can look sharp and still feel wrong to the eye

Human eyes focus and converge together in the real world. Look at a nearby hand and the lenses inside the eyes focus at that distance while both eyes turn inward toward it. Look at a distant building and both responses change.

Many head mounted displays create several apparent depths through binocular images while the light from the screen still comes from one fixed focal plane. The eyes may turn toward a virtual object that seems close while continuing to focus at the display's optical distance. This is called the vergence accommodation conflict. Research has found that the conflict can increase the time required to focus and can contribute to visual discomfort.13

The problem becomes more noticeable when an interface asks the user to move attention repeatedly between nearby and distant virtual objects. A display may have enough pixels to make text readable but still demand an unnatural response from the eyes.

Possible solutions include varifocal displays that change focus, multifocal displays with several focal planes, and light field or holographic systems that recreate more of the light entering the eye. Each adds hardware, calibration, or computing cost. Eye tracking can help by telling the system where the user is looking, but it introduces another sensor that must work for different eye shapes, glasses, contact lenses, and gaze patterns.

Better specifications will help buyers understand this issue. Pixel count and field of view are not enough. Manufacturers should also publish the focal behaviour of the display, the range of supported eye positions, adjustment limits, and results from comfort testing over realistic sessions.

Every AR control method has limits

A touchscreen works because the user can see and feel its boundaries. Spatial interfaces remove that certainty. The system has to infer whether a glance, hand movement, spoken word, head movement, or button press was intentional.

Hand tracking feels direct when it works. It can fail when a hand leaves the cameras' view, one hand covers the other, or the lighting is poor. A 2025 study of optical hand tracking documented these failures across low light, self occlusion, and positions outside the tracking area.14 Holding the hands up for long periods also causes fatigue.

Eye tracking is fast for aiming, but looking is not the same as selecting. A system usually needs a second action to confirm the choice. Voice works when the user's hands are occupied, but it is unreliable in noise and awkward when a command is private. Controllers provide buttons and precise movement, though carrying them weakens the case for glasses that are supposed to be available throughout the day. Meta's Orion prototype adds a wristband that reads electrical signals produced by muscle activity, which may allow smaller gestures but requires another charged and fitted device.

A reliable AR product needs several control methods that can substitute for one another. A repair worker wearing gloves may prefer voice and gaze. A passenger in a quiet train may need a physical control. A person with limited arm movement must be able to reach the same functions without performing a prescribed gesture. Apple advises visionOS developers to provide menu commands for custom gestures and alternatives that work with keyboards or assistive devices.15

Every control method also needs clear feedback. The user should know what the system believes they selected before an important action occurs. Undo must be easy. If tracking confidence is poor, the interface should not interpret a failed gesture as a command.

Comfort and safety must be measured, not assumed

Weight is only one part of comfort. Balance matters because a lighter device can still press painfully on the nose or pull forward from the forehead. Face shape, head size, distance between the eyes, hairstyle, prescription needs, and sensitivity to pressure all affect the fit.

Video headsets add another source of discomfort. The camera records head movement, the processor renders a new view, and the displays show it. Any delay or mismatch between physical motion and the displayed result can make a person feel unwell. In a 2024 study of 40 participants, video passthrough headset use produced simulator sickness and altered how participants judged distance and social presence. The sample was small, so it does not establish a population wide rate, but it demonstrates that a clear image alone does not guarantee a comfortable or natural view.16

Apple advises new Vision Pro users to begin with sessions of 20 to 30 minutes, take regular breaks, and stop if they experience eye strain, headache, nausea, or blurred or double vision. Its safety guidance also says to use the device in a controlled, well lit area and not near roads or moving vehicles.17 Those precautions are sensible. They also show why a headset that is safe for a seated session is not automatically ready to guide someone through a busy street.

AR safety testing should match the intended activity. A navigation product must examine attention, peripheral awareness, crossing behaviour, and display failure outdoors. A workplace product must be tested with the actual protective equipment, noise, lighting, and hazards of that workplace. A game should establish a safe search area and stop the experience when tracking is unreliable. One general warning at setup cannot replace testing in the conditions where people will use the product.

The privacy problem includes everyone in view

An AR device does not observe only its owner. Outward facing cameras can record people, homes, computer screens, licence plates, and private conversations. Eye tracking can reveal what attracts the user's attention. A spatial map can expose the size and layout of a room. Location history can show where someone lives, works, and spends time.

The owner can accept a privacy policy. A person sitting nearby has no comparable chance to consent. This is why an indicator light is useful but incomplete. It can show that a camera is recording, yet it does not explain whether the device is mapping the room, identifying an object, recognizing a face, storing audio, or sending sensor data to a server.

The Office of the Privacy Commissioner of Canada has warned that wearable devices can collect images, audio, video, location, and activity in ways that create a detailed picture of an individual. Its current guidance on biometrics calls for data minimization, limits on use, operational testing, and safeguards appropriate to the sensitivity of the information.18

AR platforms should process raw camera, microphone, eye, and room data on the device whenever possible. Applications should receive only the result they need. A furniture app may need a plane and its dimensions, not a copy of the user's room video. A game may need to know that a player reached a zone, not retain a permanent location history.

Recording indicators should be obvious, difficult to disable, and consistent across brands. A physical camera shutter can provide stronger assurance when recording is not needed. Users need plain controls for deleting maps and seeing which applications accessed sensitive sensors. Face recognition of bystanders should be disabled unless a narrow, lawful purpose and meaningful consent exist. These protections will reduce some convenient features, but public acceptance depends on people being able to tell what a device is doing around them.

Software built for one AR platform may not work on another

A developer can build a strong AR experience for one platform and still face substantial work to bring it to another. Apple ARKit and RealityKit, Google ARCore, headset runtimes, and web browsers expose different combinations of anchors, depth, room meshes, hand tracking, permissions, and shared sessions. The same feature name does not guarantee the same data quality or behaviour.

Standards have improved the situation. OpenXR 1.1 provides a common interface between many immersive applications and device runtimes. Khronos moved widely used extensions into the core specification specifically to reduce fragmentation.19 The glTF format gives developers a common way to transmit three dimensional models and scenes.

Those standards do not solve every problem. A model can load correctly while anchors, lighting, occlusion, input, and permissions still behave differently. Apple's guidance for bringing an ARKit application to visionOS says developers should not simply reuse code that depends on unsupported technologies.20 Even movement within one company's products can require a redesign.

The web could lower the cost of reaching different devices because a person can open an experience through a link. WebXR defines browser interfaces for virtual and augmented reality, but as of June 2026 it remained a W3C Candidate Recommendation Draft rather than a completed Recommendation.21 Browser support and access to advanced sensors remain uneven.

The next useful standards need to cover more than rendering. Developers need consistent ways to request spatial permissions, save an anchor, describe a surface, receive hand input, share a map, and understand the quality of each result. Platforms can still compete on hardware and interface design. They should not require every small studio to rebuild the foundations of the same spatial experience.

AR should be judged by whether it improves the task

The industry often presents success as a pair of normal looking glasses that can place perfect digital objects anywhere for an entire day. That is one goal, but it is not a useful test for every current product.

Phone AR can already be useful when a short camera session answers a practical question, starts a game, explains an object, or confirms a location. A headset can justify its size in training, design, medicine, or remote work when the spatial view is worth the effort of putting it on. Lightweight glasses can provide directions or captions without attempting to reconstruct the whole world.

The relevant question is whether AR is the best way to complete the task. A floating menu that takes longer to select than a phone button is not an improvement. Directions that hide a real hazard are worse than a map. A virtual object that loses its location is not useful, regardless of how convincing it looked at first.

Products should therefore publish measures tied to the job they promise to do. A navigation system should report where accurate visual positioning is available. A shared game should show how well two devices agree about an object's location. Glasses should report battery life under continuous display use, not only a mixed pattern that leaves the display off much of the time. Workplace systems should publish error recovery and safety results from the workplace itself.

What dependable AR would require

There is no useful definition of perfect AR. A practical standard is predictable operation. People should understand what a system can do, trust it to perform that job, and know immediately when it cannot.

For phone AR, that means stable tracking in ordinary light, clear instructions when the camera needs more information, and graceful operation on devices without the newest depth hardware. Saved objects should return to the same place, or the app should say plainly that the place could not be confirmed.

For outdoor AR, a map coordinate must be treated as the beginning of alignment rather than the end. The system should combine broad positioning with a local visual or radio reference. It should expose its current accuracy and refuse interactions that demand more precision than it has.

For glasses, the industry needs displays that remain readable outdoors without turning the lenses into dark goggles. A useful field of view, a forgiving eyebox, support for prescription users, low heat, balanced weight, and enough battery for the promised task must exist in the same product. A breakthrough in only one of those measurements is not enough.

Every form of AR needs better scene understanding. Digital objects should know which real surfaces can support them, which objects are moving, and what should pass in front. That model has to update quickly and remain private. Input must offer alternatives rather than requiring every person to use the same eyes, voice, hands, or range of movement.

AR companies also need clear and enforceable rules for sensor data. People should know when a device is recording and what data leaves it. Developers should receive the minimum information required for a feature. Users should be able to erase spatial maps, revoke access, and understand the result without reading a long legal document.

These problems will not be solved at the same time. Tracking, optics, batteries, chips, standards, interface design, and public rules will improve at different rates. AR will become dependable when those improvements are combined in products that work outside a prepared demonstration.

The basic idea of AR has been proved. The remaining work is to make it function reliably in ordinary places, on devices people can afford and tolerate using. If the product does not continue to solve a useful problem after the novelty fades, the quality of its graphics will not matter.

Notes and sources

This article uses current manufacturer documentation, standards, public guidance, and research. Product examples explain specific engineering choices. They are not endorsements.

  1. Meta Ray-Ban Display specifications and its intended use for brief visual interactions: Meta product guidance and the September 2025 announcement. ↩
  2. How ARKit combines camera and motion data, and the conditions that limit tracking: Apple Developer Documentation. ↩
  3. Relocalization and changes in lighting or the physical environment: Apple Developer Documentation. ↩
  4. Typical smartphone GPS accuracy under open sky and the factors that reduce it: GPS.gov. ↩
  5. How the ARCore Geospatial API combines satellite and visual positioning, and how applications can check local visual positioning availability: Google for Developers. ↩
  6. How ARCore calculates depth, the useful distance range, and the need for camera movement and visual detail: Google for Developers. ↩
  7. Current device certification, feature differences, and reported Depth API support as of May 2026: ARCore supported devices. ↩
  8. Limits of estimated environment textures for reflections: Apple Developer Documentation. ↩
  9. Current weight, battery duration, display specifications, and Canadian price for Apple Vision Pro with M5: Apple technical specifications and Apple Newsroom. ↩
  10. Weight, field of view, battery duration, latency, and developer release of the 2024 Spectacles: Snap Newsroom. ↩
  11. Field of view, display technology, input methods, separate compute puck, and prototype status of Orion: Meta Orion. ↩
  12. The linked tradeoffs among field of view, eyebox, brightness, resolution, eye relief, and frame size in transparent displays: Kress and Chatterjee, Frontiers in Virtual Reality, and Xiong and colleagues, Light: Science and Applications. ↩
  13. Measured effects of vergence accommodation conflict on the time required to focus: Konrad and colleagues, Journal of the Society for Information Display. ↩
  14. Failure conditions for optical hand tracking, including low light, self occlusion, and positions outside the camera view: Shi and colleagues, 2025 study in Sensors. ↩
  15. Alternative input and accessibility guidance for visionOS applications: Apple Developer Documentation. ↩
  16. Simulator sickness, distance judgement, and social presence during video passthrough use in a study of 40 participants: Cyberpsychology, Behavior, and Social Networking. ↩
  17. Fit, breaks, symptoms, controlled spaces, and safety advice for Vision Pro: Apple Support. ↩
  18. Privacy risks from wearable cameras, microphones, location, and activity data: Office of the Privacy Commissioner of Canada. Current requirements for the responsible use of biometric information: Canadian biometrics guidance. ↩
  19. OpenXR 1.1 and the consolidation of common extensions into a cross platform interface: Khronos Group. ↩
  20. Platform changes required when bringing an ARKit application to visionOS: Apple Developer Documentation. ↩
  21. Current status and scope of the WebXR Device API: World Wide Web Consortium, Candidate Recommendation Draft dated June 9, 2026. ↩

Related HuntAR guides