The New iPad Pro's LIDAR Sensor Is An AR Hardware Solution In Search of Software (theverge.com) 45
One of the biggest new additions to Apple's new iPad Pro is a new "Light Detection and Ranging" (LIDAR) system on the rear camera, which Apple argued was the missing piece for revolutionary augmented reality applications. "It claims that by combining the depth information from the LIDAR scanner with camera data, motion sensors, and computer vision algorithms, the new iPad Pro will be faster and better at placing AR objects and tracking the location of people," reports The Verge. "But it doesn't change the fact that, right now, there still aren't a lot of compelling reasons to actually use augmented reality apps on a mobile device beyond the cool, tech-demo-y purposes that already exist." From the report: R apps on iOS today are a thing you try out once, marvel at how novel of an idea it is, and move on -- they're not essential parts of how we use our phones. And nearly three years into Apple's push for AR, there's still no killer app that makes the case for why customers -- or developers -- should care. Maybe the LIDAR sensor really is the missing piece of the puzzle. Apple certainly has a few impressive tech demos showing off applications of the LIDAR sensor, like its Apple Arcade Hot Lava game, which can use the data to more quickly and accurately model a living room to generate the gameplay surface. There's a CAD app that can scan and make a 3D model of the room to see how additions will look. Another demo promises accurate determinations of the range of motion of your arm.
The fact that Apple is debuting the iPad for AR doesn't help the case, either. While Apple has been rumored to be working on a proper augmented reality headset or glasses for years -- a kind of product that could make augmented digital overlays a seamless part of your day-to-day life -- the iPad (in 11-inch and 12.9-inch sizes) is effectively the opposite of that idea. It's the same awkwardness of the man who holds up an iPad to film an entire concert; holding a hardcover book-sized display in front of your face for the entire time you're using it just isn't a very natural use case.
It's possible that Apple is just laying the groundwork here, and more portable LIDAR-equipped AR devices (like a new iPhone or even a head-mounted display) are on their way in the future. Maybe the LIDAR sensor is the key to making more immersive, faster, and better augmented apps. Apple might be right, and the next wave of AR apps really will turn the gimmicks into a critical part of day-to-day life. But right now, it's hard not to look at Apple's LIDAR-based AR push as another hardware feature looking for the software to justify it.
The fact that Apple is debuting the iPad for AR doesn't help the case, either. While Apple has been rumored to be working on a proper augmented reality headset or glasses for years -- a kind of product that could make augmented digital overlays a seamless part of your day-to-day life -- the iPad (in 11-inch and 12.9-inch sizes) is effectively the opposite of that idea. It's the same awkwardness of the man who holds up an iPad to film an entire concert; holding a hardcover book-sized display in front of your face for the entire time you're using it just isn't a very natural use case.
It's possible that Apple is just laying the groundwork here, and more portable LIDAR-equipped AR devices (like a new iPhone or even a head-mounted display) are on their way in the future. Maybe the LIDAR sensor is the key to making more immersive, faster, and better augmented apps. Apple might be right, and the next wave of AR apps really will turn the gimmicks into a critical part of day-to-day life. But right now, it's hard not to look at Apple's LIDAR-based AR push as another hardware feature looking for the software to justify it.
Need porn applications (Score:1)
Re: (Score:2)
Yeah, Apple's patent application suggests the Lidar is to create a high resolution scan of your cock for VR porn simulators.
I see nothing in the patent about high-gain magnification
Also (Score:3, Interesting)
Re: (Score:3)
if you're looking at a screen, that ain't augmented reality, thats 2005 webcam technology.
OK bud, let us know when the brain implant option is available then. There is no rule for AR that says how the overlay is presented. A thermal imaging camera is an example of AR, and it is still AR if it is projected on a transparent surface, or a head mounted display, or a monitor.
Using your logic, over the ear headphones could never be used for AR sound because they aren't transparent. That's bull and you know it.
Also, I don't remember 2005 webcams with depth sensors.
Re: Also (Score:1)
Re: (Score:2)
OK bud, let us know when the brain implant option is available then. There is no rule for AR that says how the overlay is presented. A thermal imaging camera is an example of AR, and it is still AR if it is projected on a transparent surface, or a head mounted display, or a monitor.
Its literally in the name - augmented reality , despite what you may think your tablet screen is not reality. On the other hand, Microsoft's Hololense and Magic Leaps Goggles are overlaying graphics over reality.
Also, I don't remember 2005 webcams with depth sensors.
Old webcams had the ability to track faces on screen and overlay graphics on top of them.
Real Work (Score:4, Insightful)
Re: (Score:2)
To be fair - that's not something that would be a killer app for the general market.
Re:Real Work (Score:5, Interesting)
Re: Real Work (Score:1)
Re: (Score:2)
Something like this already exists https://www.kaarta.com/ [kaarta.com]
Re: (Score:2)
Re: Real Work (Score:2)
Especially since the stated primary design was measurement precision.
Re: (Score:2)
LiDAR can be cool, if it has a purpose (Score:3)
I was curious how this LiDAR compares to a professional terrestrial LiDAR scanner... it's a little bit hard to tell, but my assumption is "not favorably." Here's Apple's description of it:
That's about the most thorough and technical description I can find.
So I don't know anything about the resolution of the sensor, but if it were good enough, it may be suitable for purposes like 3D scanning, using physical objects to create 3D models? The applications they list are pretty mundane, things like making the measurement app work better.
Re: (Score:2, Interesting)
If it fits in the iPad it must be tiny. It's not going to have much, if anything, in the way of movable optics either. So the amount of scanning it does will be extremely limited.
I imagine they intend it to be used with existing AR technology. I don't know about iOS but on Android it's been around for a few years. You can measure up a room just using the camera on your phone (well, it uses the accelerometer too). Lidar would help automate the process of measuring things.
How long ago did they shut down PrimeSense (Score:2)
LoB
Re: (Score:2)
I think it needs to be so cheap and light that even if someone doesn't use it, having it on board doesn't really compromise the design of the device. Maybe it took some extra miniaturization for that time to arrive.
muscle fatigue (Score:2)
The article makes a good point. If you have to hold a device out in front of you to use it, you won't be comfortable doing that for very long. Whether or not AR has a good use case, this particular form factor doesn't seem well thought out.
Re: (Score:1)
For measuring it's fine. I use AR for measuring rooms when I don't have a tape measure handy or just need a quick estimate.
Re: (Score:2)
For measuring it's fine. I use AR for measuring rooms when I don't have a tape measure handy or just need a quick estimate.
I can see how that would be useful, especially if I found myself measuring rooms often enough to justify the price tag...
Re: (Score:1)
If I was doing it often or had money to burn I'd probably get one of those laser measures with Bluetooth.
It will help them get dominance in extra niches (Score:4, Interesting)
I can see kitchen/bathroom installers and interior designers eventually using this to get 3D scans of rooms and fill them in. Which might not be a huge market, but complete dominance in niche markets will help you even outside those markets.
Maybe with the right software you could just take some pics in underwear and do simulated dress up and get accurate warnings about tight fits for online shopping of clothes.
Real time AR I think would be by far the least relevant use.
Re: (Score:1)
Please make me some jeans that fit correctly (Score:2)
Re: (Score:2)
Future headline: Apple's 3d scanning database breached, millions of asses dumped onto the internet.
FlightRadar24 is the only AR app I use on my phone (Score:1)
https://apps.apple.com/us/app/... [apple.com]
The main app I'd want in AR on a tablet would be fixit instructions.
Comment removed (Score:3)
Re: Wait for WWDC. (Score:2)
Absolutely this.
This is no doubt a somewhat pricey peripheral, at least in consumer electronics terms.
Even if the âoehidden agendaâ is to get tons of real-world applications and frameworks started for eventual use for their upcoming glasses, Apple no doubt has a bigger plan in mind than simply getting more accurate placement of virtual furniture in the Ikea App.
Re: (Score:2)
Apple doesn't put features into hardware without a purpose. In a couple of months, we'll know what it's really for.
-jcr
So you can put Animoji on everyone, clearly.
Lidar vs Stereo Vision (Score:2)
Why not just have two cameras some distance apart. It is fairly easy to analyze the two images to calculate distance of each pixel. Cheaper and works in bright lights. Heck, use your wide angle and telephoto to produce stereo in the region covered by both. No additional hardware needed provided the cameras a separated by an inch or two.
Re: (Score:2)
I believe they also have two cameras on there in addition t the Lidar.
Re: (Score:2)
Re: Practical and useful applications that aren't (Score:1)
Re: (Score:2)
3D scanning (Score:1)
I'd be more interested to see if it's useful for 3D object scanning applications.
Current solutions like Qlone and Trnio with their cloud-based processing of a photo series are "balls." Very poor results, when you can get results anyway - half the time their service is offline, timing out, or scans just disappear to /dev/null.
R apps (Score:2)
Hey, it's nice Apple are embracing the R programming language [r-project.org] for mobile as well!
Gold digging? (Score:2)
Sounds like a great replacement for metal detectors or for archeological use on digs before actually digging.
Apple's Kinect (Score:3)
This is just like Microsoft adding the Kinect to their Xbox. It was supposed to revolutionize gaming. It turned out to be a flop.