VisionOS
2000 apps for vision pro. 1.5 mil compatible apps from iPad OS
VisionOS 2
Photos - Automatically creating depth images from 2D images using machine learning.
Mac content display going to get higher resolution etc.
Adding train support to transit (along with airplanes which was previously supported)
Canon is launching a new lens to shoot spatial videos.
Can be edited in FinalCutPro and viewed in Vimeo
Partnered with Blackmagic Design to support Apple Immersive Design
Expanding to China, Singapore and Japan in the next couple of weeks.
iOS 18
Logos - Logos will change color in dark mode. Or can change color of all logos at the same time. Way too much personalisation for my preference.
Control center - Apple is making control center more powerful (and more confusing).
Developers can add their own items on the control center. (this might be useful, depending on what app developers do with it).
Can swap out torch and camera controls in lock screen with this.
Privacy -
Lock an app - Sensitive apps will be protected by Facetime verification.
Hide an app - Sensitive apps can be hidden.
iMessages -
Allow tapbacks with any emoji.
Scheduled messages / send later.
Text formatting (strike text, underline etc).
Text effects
iMessages via Satellite
Mail -
On-device categorisation
Maps -
Offline save your hikes
Wallet -
Tap to cash.
Pay from apple wallet.
Apple pay online
Event tickets get a redesigned. More details of venues.
Journal -
A new insights view.
Better search for past entries.
Gaming -
Game mode is coming to iPhone.
Just like on Mac, minimises background activity.
Improves responsiveness for wireless audio
Photos -
“Biggest redesign ever to photos”
Adding more themes on the home page (called collections) instead of just getting to the photos. (mostly not useful for me personally)
Filter option seems very useful!
Instead of people search, you also now have “Groups of people”. Say best friends together.
Audio and Home
Airpods
Nod head as an action instead of just the tap actions. This is perfect for indians.
Bringing voice isolation to airpods pro.
Personalized spatial audio API
TV OS
Audio
InSight - Swipe down on TV. See actors, music etc in real time. Also available on iPhone app.
Enhance dialogue! Very useful.
Subtitles will autoprompt at more
Video
21:9 support
Much better screensavers
WatchOS 11
Activity
Training load - Will calculate “effort” automatically. Can adjust it manually. This will calculate Training load.
Goals can be adjusted based on day of the week etc.
Health
New Vitals app to see body vital measurements and see trends.
Cycle tracking will now help during pregnancy.
Connected
Automatically add new widgets to the smart stack.
Live activities coming to apple watch.
Check in also coming to apple watch.
Customisation
WatchOS11 will help automatically identify and frame best photos for apple watch.
iPadOS 18
New ways to personalise home screen (same as iPhone)
New photos app (same as iPhone)
Introducing a Tab bar to help ensure apps are full screen. (i love this, but might not be very easy for most people to get used to this). This is replacing/augmenting the side bar.
Shareplay introduces annotations and remote control.
Freeform adds scenes.
Bringing calculator to iPad!
Math notes is cool! (especially that the output handwriting is same as input)
This is also coming to notes.
SmartScript in Apple notes -
improve your handwriting as you go! you can also paste text.
Spell check works great.
Can move/adjust text.
To erase, scratch text out.
MacOS Sequoia
Math notes in Mac!
Can plan a hike on Mac.
Continuity
iPhone mirroring on a mac
iPhone notifications on Mac
iPhone’s audio also comes through on the Mac.
Your phone stays locked.
Can drag/drop files from computer to the mirrored iPhone.
MacOS does tiling automatically! No need to use rectangle.app or something similar. Finally!
Screen sharing - Background replacement coming natively to Mac.
New Passwords app - Available on all devices, including windows.
Safari
Highlights automatically pulled in from a page.
Summaries included for apps in the reader mode.
Introducing “Viewer” for “focussed mode for videos”
Gaming
Game porting toolkit 2
Easier to bring mac games to iPad, iPhone etc. As an example, Assassin’s creed shadow will come to iPhone, iPad and Mac.
Generative AI (Apple Intelligence)
Apple’s AI design Principals
Powerful
Intuitive
Integrated
Personal
Private
Capabilities
Create language and images.
Take actions
Understands you
Specific things -
iPhones can prioritise notifications.
Brand new writing tools system wide!! Mail, notes, safari, third party tools will all support it.
Autogenerated images when writing text (sketch, illustration and animation).
This is built into all apps too!
Action -
“Pull up the files that Joz showed last week”
“Play the podcast that my wife sent me last week”
Personal context -
Can refer to content on your screen.
Search your calendar, past messages etc.
Architecture
On-device privacy.
A17 pro and M family of chips can all support it! (which means all current macs will get it, along with iPhone 15?)
Semantic index.
Apple is creating a private cloud. (verifiable by third party). This is scary, so apple is spending a LOT of time on it.
Experiences
Siri
Different visual look.
You can correct what you are speaking. Mistakes are okay.
Carries context from previous messages.
Double tap on bottom of the screen - Write to siri.
More info about features and settings.
Next year -
Siri will have on-screen awareness.
Ability to take actions inside apps (and more importantly, across apps).
Will be made possible by “App Intents”
Ability to extract drivers license numbers etc from historical pictures.
All Siri updates on iPad, Mac, iPhone.
Some more updates -
Writing tools
Rewrite coming natively to Mac.
Proofread.
Smart reply in mail.
Mail - Instead of seeing email in preview in the mail app, it will show summaries. interesting!
Reduce interruptions focus mode. AI decides how important a notification is.
Communicating with Emoji upgraded - Genmoji. Create a new emoji every single time! (man, this is why the researchers did text to 3D generation?). You can add people in Genmoji.
Image Playground
you can add a bunch of objects and automatically generate previews. hence no prompt engineering!
happens on device. so no cost.
pick from animation, sketch or illustration
integrated in most apps, but also as a separate app.
Rough sketch to a polished app in apple notes. Circle to call image playground.
Photos
Photo-editing - Native object removal built on photos.
Can search images and videos using NLP. (in videos, it will take you to the right spot automatically)
Create a memory movie - Just write what you would like to create the video to be about, it will automatically pull in the right content
Notes app can record audio.
Phone app can record a call and take notes!
Apple intelligence will be free.
Integrating ChatGPT, GPT4o.
Siri can tap into GPT4o.
Include photos into message sent to ChatGPT.
Integrated ChatGPT into writing tools for “compose”
Compose will also tap into ChatGPT’s image capabilities.
ChatGPT available for free. For people who have paid accounts, they will get native features in iOS.
Coming “later this year”
Will add support other vendors (outside of ChatGPT). No idea when.