A Demonstration and Walk-Through of the New Live Recognition Features in iOS 27

In this episode, Levi Gobin demonstrates the new Live Recognition capabilities in iOS 27.

Transcript

Disclaimer: This transcript was generated using Google Gemini, an artificial intelligence (AI) tool. It was lightly edited and formatted, and it may not accurately capture the speakers’ names, voices, or content.

Announcer: You're listening to another AppleVis podcast. Levi Gobin: Hello, everyone, and in this podcast, I'm going to be demonstrating the new features for Live Recognition and Magnifier in iOS 27. So the first big new feature is the fact that you can now ask Magnifier a question, and it will use Apple Intelligence to describe the scene. And you can access this from either the rotor or a VoiceOver gesture, or from the Magnifier app itself. I will be discussing all of these today. So I'm going to go ahead and unlock my phone. VoiceOver: 12:34, 98% charged. Levi's iPhone 17 Pro Max. Levi Gobin: Okay, it's reading my battery widget. And I'm going to now navigate to the the Live Recognition rotor and swipe down to Ask. VoiceOver: Characters. Actions, activate. Image Recognition, item. Text Navigation. L, Live Recognition, Ask, off. Levi Gobin: And I'm already on it. Um and now I'm going to double tap, and it will bring up the Ask flow like this. VoiceOver: Close. Question, text field, is editing, Ask, insertion point at start. Levi Gobin: Okay, and so now it puts me in a text field where I can type, but first I'm going to explore the interface and show you what else is here. VoiceOver: Close Transcript, button. Levi Gobin: So, a button to close the transcript, which will take you back to the Live Recognition area. VoiceOver: Ask, heading. Close Live Recognition, button. Question, text field, is editing, Ask, insertion point at start. Ask, dimmed, button. Hi, button. Levi Gobin: And uh that's all you have. So I can then type in a question. VoiceOver: Speech off. Speech on. Levi Gobin: Oops. Um, like VoiceOver: w h Do you see on the dot 15 Translated, box. Levi Gobin: on the box. I didn't type the a for some reason in the word what, but it will be able to understand it. And now I'm going to hit the Ask button. VoiceOver: Question, text Close Live Question, Ask, button. Levi Gobin: If you're using braille screen input, you can also swipe up with three fingers. Um but I have this box right here, and I'm going to hit the Ask button on it. VoiceOver: Just a moment. This is a box for an Instamic microphone recorder, which is labeled as a wearable, wireless, and waterproof device featuring 32-bit float recording. Levi Gobin: And say I wanted to um ask it VoiceOver: Space. Does it say how long you can dot 1 2 3 Record record Record on the box. Translated, box. Levi Gobin: And then I'm going to VoiceOver: Question, text Ask, button. Levi Gobin: hit the Ask button. VoiceOver: Just a moment. No, the visible text does not state the recording duration. Levi Gobin: Now I'm going to flip the box over. I'm not sure if it does say the actual duration on it, but that's why we have AI. I flipped it over, and I will say, VoiceOver: D How about Translated, now. Levi Gobin: And I will then hit the Ask button again. VoiceOver: F O P w h t Do you see on Question, text Ask, button. How Just a moment. No, the text on this side of the box does not state the recording duration. Levi Gobin: Okay, so the recording duration is not on the box. Um, so you can ask it directly from there, and you can also get to this flow by assigning a VoiceOver gesture to it, which I'll show you. So I'm going to go ahead and close Live Recognition. VoiceOver: Close Live Recognition, button. Close Live Recognition, heading. Batteries, Levi's iPhone 17 Pro Levi's iPhone 17 Levi Gobin: Okay, and there's my battery widget. Now, I have a VoiceOver gesture assigned to take me directly to the Ask flow, and I will now do that. VoiceOver: Question, text field, is editing, Ask, insertion point at start. Levi Gobin: And now I'm ready to ask another question. I'm going to close it again. VoiceOver: Close Live Recognition, button. Close Live Recognition, heading. Batteries, Levi's AirPods Pro 3 Case. Levi Gobin: Okay. And the next way you can get to the Ask flow is through the Magnifier app itself directly. I used to use Be My Eyes and never really use the Magnifier app, so I had the Camera Control button open Be My Eyes. But I've disabled that particular automation, which whenever Magnifier opens, we'll open Be My Eyes for this demonstration. I'm now going to double-click the Camera Control button, and it will open Magnifier. VoiceOver: Magnifier, viewfinder, image. A remote control on a wooden surface. Possible text: Stams, Di2, High-Quality Recording and Streaming, USB Type-C, Waterproof, CE FE, and Ro Levi Gobin: And so now I'm going to go to the Detection Mode button. VoiceOver: Hard controller, selected, zoom, Zoom, 0, Detection Mode, off, Ask, Detection Mode, off, button. Levi Gobin: So you can go to Detection Mode here, and you can also VoiceOver: Ask, button. Levi Gobin: ask, VoiceOver: Freeze Frame, Ask, button. Levi Gobin: which I will now select. VoiceOver: Just a moment. Ask. A white, rectangular Instamic Pro Plus C product box resting on a wooden surface. Along the bottom edge, Instamic registered Pro Plus C, the smartest mic on earth, ready to record. In the center, arranged around a line drawing of the device, High-Quality Recording and Streaming, microphone icon, Waterproof, water droplet icon, USB Type-C, lightning bolt icon, Capture Instants, two overlapping squares icon. In the upper right corner, Bluetooth logo, barcode, CE FC, recycling bin icon, RoHS compliant, designed by Instamic in California, USA, made in China, additional small, illegible text lines. Levi Gobin: Oh yeah, so it does that if you want to dictate something afterwards. Um but I have the microphone disabled in the Magnifier app because I don't use that. So it would have it would have made the noise like you're starting to record something, then you would have asked a question, and then it would have responded to you. So now, um you can't seem to right now in 27.0, like you can't seem to type a question, VoiceOver: Freeze Frame, button. Ask, button. Levi Gobin: because when you ask, VoiceOver: Zoom, 0%, Detection Mode, off, button. Ask, button. Just a moment. Ask. A white, rectangular Instamic Pro Plus C product box resting on a wooden surface. Along the bottom edge, Instamic registered Pro Plus C, the smartest mic on earth, Levi Gobin: So it's going to read this again, VoiceOver: ready to record. Levi Gobin: and if I swipe away, VoiceOver: Detection Mode, off, button. Levi Gobin: yes, it's going to try to want me to dictate, but I have its microphone disabled because I don't think the Magnifier app needs my microphone, but that's just me. You can have it enabled, and then you can dictate a question. And the other new thing for Magnifier is scene descriptions. They are now powered by Apple Intelligence. Uh so, for instance, VoiceOver: Ask, but Detection Mode, off, button. Levi Gobin: I'm going to go to Detection Mode, and it's probably going to start reading a scene description because scenes get turned on by default when no other mode is on. So I will now double tap. VoiceOver: Detection Mode, on. Detection Mode, on, dimmed. Scenes, on. A white product box resting on a polished wooden surface next to a dark gray electronic device. The box is oriented upside down relative to the viewer, with various symbols and text printed in black along its top edge. Along the top edge, CE, RoHS compliant, contains Bluetooth module IC, 2753G-T008CZ, designed by Insta360 in California, vertical Levi Gobin: And now I'm going to point the microphone, or not the microphone, but the phone away at the wall, for instance. And VoiceOver: Zoom, 0%. Just a sheer white curtain hanging over a window on the left side of a room, with bright sunlight filtering through the fabric. The curtain extends from the top of the frame down to the floor, pooling slightly on the dark, polished hardwood planks. To the right of the window, a smooth, off-white wall stretches across the remainder of the scene. A dark wooden baseboard runs along the bottom of the wall, meeting the flooring. A small, metallic object, possibly a doorstop, protrudes from the baseboard near the lower right corner. Levi Gobin: I'm now going to point it at the door. And it should Oh, so when you do two-finger double tap or two-finger single tap, it will stop speaking. VoiceOver: Selected, Zoom, 0%. A dark, reddish-brown wooden door standing closed within a matching wooden frame, resting on a polished hardwood floor. The door features a traditional panel design, with a large, vertically oriented rectangular panel visible in the lower section and a horizontal panel just above it. The wood grain is highly visible, displaying swirling patterns across the smooth, varnished surface. To the left, a thick, fluted wooden door casing borders the door, meeting a plain, off-white wall. A matching wooden baseboard runs along the bottom of the wall, where a small, black, ribbed rubber doorstop is mounted near the floor. The floorboards run diagonally away from the door, showcasing a rich, dark stain that complements the warm tones of the door and trim. Levi Gobin: So it gives you descriptions of the different things in your environment, um and it does this automatically just by you having the Magnifier app in Detection Mode in scenes mode, and it can do this anytime you want. You also have the same um you also have this as well in the Live Recognition rotor. So if I go to the home screen, VoiceOver: Speech on. Image Recognition. Text Navigation. Live Recognition, Ask, Navigat- Live Recogni- People, off. Doors, off. Scenes, off. Levi Gobin: and go to Scenes, VoiceOver: On. Live Recognition, alert, Magnifier is using the camera. Would you still like to start Live Recognition? Start Live Cancel, button. Start Live Recognition, button. Levi Gobin: and we'll go ahead and click Start um Recognition. VoiceOver: Levi's iPhone 17 A smooth, off-white wall rising above a polished, dark stained hardwood floor. The floorboards run vertically from the bottom edge of the frame Levi Gobin: And it goes right back into the scenes mode. VoiceOver: 100%. Levi Gobin: And the beauty about this is you can do it in any app, and you don't have to be in the Magnifier app, and then you can just touch the top middle of the screen, VoiceOver: 100%. 88% 100 88 Levi's iPhone 17 Pro. Camera in use. Purple Reign, 3: সাই 12:45. Batteries, Levi's AirPod Image Recognition, I Actions, a Image Recogni- Text Navig- Live Recognition, Scenes, on. Levi Gobin: Okay, and then you can go to the Live Recognition rotor and turn it off, VoiceOver: Off. Levi Gobin: and it disables. VoiceOver: Large. Minrow. Mail, six unread emails. Levi Gobin: So that's a quick look at some of the new features in the Magnifier in iOS 27 for Detection Mode and the asking questions feature in Magnifier. This also exists on macOS. You can go to the Magnifier app in the Utilities folder, and you can ask a question there, and it will show you the response. In 27.0, as of now, it does not speak the response back to you. You have to navigate through the list and view the response. Um, but otherwise, you can do the same exact thing on Mac. Anyways, I hope you've enjoyed this recording, and I will catch you in the next one. Peace. Announcer: This podcast was brought to you by the community at AppleVis.com. AppleVis: empowering people who are blind, deafblind, or who have low vision to get the most from Apple products and related technologies. A proud member of the Be My Eyes family. For more information, visit our website at www.AppleVis.com.