iPhone 6 and Scaled Resolutions
It seems quite clear at this point that there will be new phones at 4.7 and 5.5 inches. Some people are looking for larger text, some are looking for more space, while others want both.
Fortunately, the Retina MacBook Pros have already solved this problem with scaled resolutions. Bringing this feature to iOS would be an obvious evolution of Dynamic Type and fit well with Apple’s recent habit of sharing features between platforms.
The MacBooks default to native 2x size and then offer both smaller and larger scaled sizes for those who want either (in Apple’s terms) “Larger Text” or “More Space”. I see scaled resolutions on iOS working similarly.
On the Larger Text extreme, I envision our current 4 inch interfaces simply scaling up to 4.7 inches and 5.5 inches. This would allow every app in the store to run on the new devices, providing a smooth transition for users.
On the More Space extreme, I envision on-screen elements will remain the same size. It is already quite established that 44 points (at 163 points per inch) is the minimum size for a touch target. Maintaining that would maximize the space—Almost 40% more on the 4.7 and almost 90% more on the 5.5.
If a 4.7 inch screen must display both a 4 inch interface and and 4.7 inch interface, then it would make sense that the default resolution would be right in the middle, at 4.35 inches.
Below is a 4.7 inch device running 3 proposed resolutions:
Similarly if a 5.5 inch screen displays a 4 inch interface and a 5.5 inch interface, the default resolution should also be in the middle, and 4.7 inches is very close. The 5.5 inch screen could also support the 4.35 inch interface, as well as a 5.1 inch interface, making a smooth transition between 4 and 5.5 inches.
Below is a 5.5 inch device running at all 5 proposed resolutions:
iOS notifications are a bit of a mess. It’s easy to say that notifications and their read status should be synced across your devices. But solving our notification problem isn’t quite that simple.
A big win could be made by simplifying the notification settings screen. Sounds, badges, banners, alerts, show on lock screen, show in notification center, etc. It’s an unusual level of control and complexity for Apple and iOS. What I’d expect is something much simpler: High priority, low priority, and off.
• High priority: can wake the device and make a sound (or vibrate). These are prioritized at the top of Notification Center. Banners come down when using the device, and app icons are badged. These would be best for messaging, email, reminders and calendar events.
• Low priority: these notifications won’t wake the device, but do show on the lock screen and in lower in notification center. Banners are also shown. Would do well for news updates, social media mentions, and other casual apps.
• Off: notifications from this app will never be shown.
Making it easier for users to manage their preferences on how and when they receive notifications would alleviate many of the common issues: missed notifications, lost notifications, notification overload, etc. it’s currently possible to get your notifications under control, but it’s a lot of work to get it there.
I’d also like to see a way to sync notification settings across devices, and for devices to be aware of each other’s notifications. The goal being to separate the notification from the device. We’re not trying to send notifications to devices, we’re trying to send notifications to people.
One suggestion I’ve seen is for devices to track a user’s “attention” in an effort to predict which device should display the notification. If you’re actively using your laptop and your phone is plugged in and asleep across the room, the laptop should display the notification and the phone should stay quiet. If the first device finds that it has lost the users attention, then other devices could then display the notification.
To take this concept further, connected speakers (Apple TV, Airport, etc) throughout the house could provide a notification system removed from all devices. When one is at home (determined geofence, wifi connection, etc) this could be the first notification destination, or perhaps only when it is deemed that no device has the user’s attention (for example they are all upstairs charging). Just like on the device, priority preferences could be set for each application. Banners could be replaced with Siri dictation, and lower priority notifications could simply provide their various beeps and boops. I see this as particularly useful for timers, incoming phone calls, reminders, calendar events, etc.
iOS 8 Wishlist
This is not a prediction, just a personal list of what I’d like to see added to iOS to enable users and developers to do more. Of course a challenge with a system this widely used is to empower the more advanced user without complicating the experience for the novice.
Some Simple Updates
- Remove the stupid textures from notes and reminders and the bubbles from Game Center.
- Spotlight: Access from within apps, perhaps from a gesture, multitasking mode, control center etc. I’d also like it to better search content in third party apps.
- Reminders has the opportunity to be one of the most useful apps on a device that’s always with you. But it’s kind of terrible. It could use a few new features (Geofencing an entire list, pasting several items at once, sharing, more repeat options) but most importantly it needs a much simpler user interface.
- iCloud Clipboard: When enabled, you simply have a single clipboard between all devices. Copy here, paste there.
- Airdrop between iOS and Macs.
- In-app access to autofill passwords: Currently it’s only available in Safari. I’d like to see it not just in competing browsers, but in all apps.
- Favor FaceTime audio over voice calls (just like iMessage is favored over SMS).
- Multiple user accounts. It’s absurd to think a family can’t easily share an iPad.
- Battery-saver Mode: A one-step battery saver that reduces brightness, network activity, push notifications, background processes etc.
- More Do Not Disturb: A new setting where notifications can be hidden even while using the device. Perhaps activated automatically when using specified apps (for example, while reading a book, or watching a movie).
- Night Mode: Dark interfaces for everything, a developer API, and a setting to let the ambient light sensor activate it automatically.
- Live App Icons. Restrictions such as updating only once per hour, for apps that were used recently, etc. would keep battery life in check. Obviously there’s not a ton you can show in such a small area, but I think simple things could make a big difference: see clock and calendar. I think this could likely replace the red badge count (which many developers have hacked into something else) for more interesting solutions.
- Hide apps, just like on Apple TV. Tons of apps I want to keep but don’t need to look at every day. Including system apps.
- Set default apps, for common functions: Browser, Mail, RSS, Twitter, Messaging, Notes, Maps, Music and Reminders would be a good start. Ideally it’s a manual setup so every app won’t be bugging me to make it the default.
- More advanced home screen editing. Moving an app over several screens, moving entire screens, moving several apps from one folder to another. Just a few examples of things that are much harder than they could be.
- Genius App Layout: Moves, groups, and hides your apps based on what you use, and what other people use.
- Customizable Status Bar: A settings panel allowing users to show, hide and rearrange the status bar icons. A few new options could include upcoming calendar events, today’s date, unread iMessages, VIP emails, more specific location alerts, twitter mentions, etc. I wouldn’t mind third party support here, as long as it’s off by default.
- Notifications: I’d like to see some actions made available in the notification like “reply” or “archive” to emails for example. Android is handling it quite well. I’d like apps that use Auto-layout to shrink vertically when a notification shows up, so that nothing is obstructed by the notification.
- Better inter-app communication: To be able to easily move documents, files, text, images, audio etc between apps. This is a limiting factor for taking iOS productivity to the next level. The important part is for an app to say “I’m sending out X filetype, which apps can receive it? Windows and android handle this well.
- Siri For Everyone: third party developers could make Siri so much more useful. The biggest challenge with this is competing apps. If she doesn’t know which app to use, she could simply ask, and after a while she’ll learn how you want things done.
- Today Tab Widgets: in notification center for third party widgets with a corresponding section for them on the store.
- Official Safari Extensions: bookmarklets are obviously a hack. We’ll also need a place in the app store for downloading them. I’m sure for security reasons there would be plenty of limitations, which is fine. We probably need an updated interface for accessing them, but can also serve to organize Safari’s more advanced features like iCloud Tabs, reading list, etc.
- Improved keyboard shortcuts for hardware keyboards. I’d like shortcuts for Home, back, app switching, type to select (like on Apple TV), navigational shortcuts etc. I’d like to see a global shortcut (command-?) to display shortcuts for the current app.
- Dual-app Multitasking: a way to see and use two apps at once, likely only on the bigger iPad (and definitely not on a phone). Pages and Safari, Reminders and Calendar, FaceTime and Notes etc. Others have tried to pull this off with varying levels of success. I think Apple should give it a shot.
Is it the physical form factor or the simplicity of the operating system that has made the iPad successful? Tablets running desktop software have never done well, so I suggest that It may be more about the software. If that is the case, I think there is room for an exciting new product category from Apple: the iPad Notebook. (A better name escapes me.)
What is it
I envision this product as a hybrid between MacBook and iPad. At first glance it may resemble a MacBook, but internally it’s much more an iPad. The most noticeable feature is a permanent hardware keyboard, so it must be big enough to accommodate one of full size. So let’s say 12-inch multi-touch retina display. As I’ve mentioned before, it could run iPad apps scaled up, and also allow for a new class of apps that take advantage of the extra screen space.
Many users are already enjoying an iPad combined with an external keyboard or keyboard case. iPad Notebook would easily provide a better experience for users who prefer that setup. Build quality would be unquestionably better. The device would be better balanced with more weight under the keys. And the device would likely have room for a bigger battery than an iPad.
As I see it, this is Apple’s ultra-portable touch screen notebook. Most importantly it runs an OS designed for touch. Apple strives to create thoughtful and simple products with clear intended uses. iPad Notebook is for customers who prefer a notebook form-factor, but don’t need the complexity of a desktop operating system.
iOS does not and should never support a mouse/cursor. If developers come to expect precision pointing, the touch experience could be ruined. Therefore, a traditional trackpad should not be present on this device. However reaching up to the screen for almost every interaction is not the best experience either.
So, in the trackpad’s place I imagine something similar: a multi-touch gesture area. I see it spanning the entire width of the device and being effectively invisible. The goal would be to allow the most common actions—such as scrolling, swiping, and basic navigation—to be performed without requiring users to reach up to touch the screen. APIs would let developers easily build these gestures into their own apps.
System defaults are already defined by current iPads: single-finger swipe to scroll, bezel-swipes to access notification center and control center, and four-finger pinch and swipe for Home and Multitasking.
This leaves two and three finger swipes, double taps, multi-finger taps, and anything else open to developers to create shortcuts for their apps. I can imagine many creative uses: musical instruments, steering wheels, page-turning, map-zooming, text manipulation, etc.
No Portrait or Tablet Mode
I doubt even Apple’s Industrial Design team could create a twisting hinge that isn’t stupid. A portrait-only app could run column-boxed on a landscape screen without being too small, but that’s terribly inelegant. What I prefer: landscape only. How many Apps require portrait orientation? I can only think of a few games. Similarly the keyboard should not flip back behind the screen or otherwise “transform” in an effort to feel more like an iPad. Doing so would detract from the simplicity and purpose of the device.
Keys and Buttons
I envision the keyboard being almost the same as a MacBook. I’d like to see a few changes to the Function keys to include Home, Sleep, Multitasking and a way to show the on-screen keyboard.
My concern here is that putting the Home button off to the side may not be prominent enough, as it is so central to the iOS experience. My thought was that replacing escape would make sense as you “escape” from an app. An alternate solution could put the Home button below the display. Fn and Control keys are probably unnecessary.
iPad Notebook will not be for everyone. If you are a happy iPad or Mac User, you will likely find little use for this product. But I do think users who work mostly at a desk or on their lap, and do a lot of typing would find such a device quite useful. Diversification of hardware will help iOS and it’s ecosystem enter and thrive in new markets.
Launchpad as a Desktop Replacement
I’d like to have the desktop removed from my Mac. I’ve known the Macintosh Desktop for 20 years now, and I’m afraid it might be time to move on. I can’t think of any feature of the Desktop that can’t be accomplished as well or better through a Finder window. I’d still like to keep a wallpaper behind my windows, but no icons. The last few months I’ve spent ignoring the Desktop: I won’t put a single thing on it, and life has only been better. I’ve been more organized because I’ve filed everything away immediately instead of hoping to organize it later (i.e. never).
And that got me thinking about Launchpad, as a sort of Desktop replacement.
Everyone seems to hate Launchpad, and I think I’ve realized why: few take the time to organize it. You scrutinize over the arrangement of your iPhone screen, yet neglect your poor Mac. (And before you even say Spotlight, my well organized and memorized Launchpad screen puts any App two clicks away—which is faster than moving my hands to the keyboard.) I challenge you Launchpad haters to organize the apps that you use and put anything you don’t
Launchpad is an obvious desktop replacement, but it’s not quite that simple. You can’t just drop it in. The primary goal of disabling the desktop was to reduce clutter, so the last thing I want is more icons. What I need is a smarter and more accessible launchpad.
Launchpad is far from perfect. Here’s what I’d do to fix it:
- Animations and other UX choices could better position Launchpad as a default bottom layer, a true “home screen” rather than a secondary layer laid on top of everything else.
- Kill the background blur.
- Rather than Finder, Launchpad could be made the new default app. It could appear automatically when all apps have been quit, hidden, or had their windows closed. You would now be able to quit Finder.
- Menubar apps and other non-dock apps shouldn’t be shown in Launchpad, at least by default. These often don’t need to be launched and only add clutter.
- The search bar should simply be Spotlight, not just an app search.
- Easier to access. I suggest a hardware Home Button on laptops and keyboards.
- Add a software Home button in the top left corner next to the Apple Menu. Fitz Law makes it incredibly easy to click, just throw your mouse into the corner.
- Apps in the dock should not also appear in the Launchpad grid.
- iOS icon shapes. They look better on a grid, and it helps with the app ecosystem brand image.
- Customizable grid spacing and icon size, this would coincide with the dock size.
The name iPad Air all but implies there is some sort of (as I’ll call it) iPad Pro in the works. How would Apple design an iPad for Professionals?
Of course it’s got a faster CPU, more storage and all that, but that’s expected and boring.
It’s bigger. I’d guess 12.1 inches diagonal at 2560 x 1920 (similar pixel density as the iPad Air). The existing library of iPad apps should be able to scale up to fit without making the interface unbearably large. However I think the App Store should expand to hold a new class of “Pro” apps designed to really take advantage of the new size. AutoLayout and other techniques will help minimize development effort for building these new apps, and the Universal app model will still apply. The beauty of this system is that a 10 inch iPad Air could also run these new pro-sized apps at a scaled down size resulting in a virtual density similar to what we’re used to with the iPad mini.
At these scaled sizes (from iPad up to iPad Pro and iPad Pro down to iPad Air) pixels would not map 1:1, but I would expect similar results as a retina Mac running at a scaled resolution: practically perfect.
According to my calculations, a 12 inch iPad Pro has plenty of screen real estate to simultaneously show two iPad-mini-sized screens side by side. Notes and Browser, Code and Browser, Messages and Twitter, etc. There are sure to be some interaction issues (keyboard, focus, entering this mode) to work out in an iOS update but it’s a seemingly simple solution compared to what Microsoft is doing.
A single Thunderbolt 2 port. Give developers and hardware manufacturers whatever they need to build really interesting Thunderbolt accessories for iPad. Storage, audio, video, cameras, networking, anything. And then there’s external displays. A lot like existing HDMI output, but higher resolution. The iPad would act as a controller for content on the external display. It’s a new interaction model that would need some work to get right, but I don’t think it’s out of the question. Games, photo and video editing, and general content consumption come to mind as good uses.
iPad Keyboards and Shortcuts
This was in my drafts folder for the last month until I learned today that Apple has added (bluetooth) keyboard shortcut support for developers in iOS 7, in what seems to be exactly how I hoped. I figured I’d post this anyway:
The iPad wants to replace the laptop. And for many types of users it already has. There are still many advanced users who need more precision. I think there is a small change that would be quite easy to implement. It would bring much added benefit to those who want it and would bring no added complexity for those who don’t.
The iPad has always supported an external keyboard, and there are a few shortcuts supported by the system, mostly cursor movement and basic text formatting. But beyond that it’s mostly a bummer. I’d love to see some some APIs and other tools open up for developers to build in complex navigation, controls, and anything else we can think of into our apps.
When using the iPad propped up (with a Smart Cover, dock, stand, etc.) continuously reaching up to touch the screen is far from the perfect interaction. The laptop design solves this problem by putting keyboards and trackpads flush with the surface of the table so your hands are always at rest. The Microsoft Surface’s keyboard cover was a nice try, but I think including a trackpad (an mouse support) only encourages the development of apps with small touch targets, designed for a pointer rather than a finger. It just gets awkward.
Designing an interface to be controlled with a keyboard does not alter the visual interface of an application the way designing for a mouse pointer would. For example converting a pointer-optimized interface to a touch-optimized one, a button must grow significantly larger to be easier to tap with a finger. Keyboard shortcuts on the other hand are invisible to the average user.
One counter argument is exactly that: the shortcuts will not be discoverable. But I have a solution. A global keyboard shortcut for “Keyboard Help” supported by the system and therefore available in every app. I propose Command-? the Mac equivalent for help. Keyboard users must commit only this single shortcut to memory and they can easily reference all available shortcuts in any app.
A few system shortcuts would be a good start. I’d like to see additions for pressing the home button, navigating pages of the home screen, type the name of an item to jump to it (just like on Apple TV), control center, notification center, etc. The built in apps could follow suit, and my suggestion makes it easy: copy from the Mac.
I understand that we are introducing a layer of abstraction here. Typically the iPad benefits from direct manipulation of onscreen elements. If you want a thing to happen, you simply touch it, a concept so simple a toddler can understand. And yet, this proposition takes nothing away from that. Only the power users who’ve paired up their bluetooth keyboard will ever see this complexity. And they’ll be more productive because of it.
Missing Mac Apps
Mac OS X Mavericks introduced two new native Mac apps: iBooks and Maps. Despite being basic clones of their iOS counterparts, they do bring useful functionality to the Mac. However there are still a few more apps that Apple has yet to bring back to the Mac. (Listed in order of importance)
Siri isn’t really an app, but I could see her living happily in the dock. And with a full keyboard, I wouldn’t mind typing to her either: less awkward and more accurate. I have no idea why this hasn’t happened yet.
Newsstand. If we’re going to be reading books, why not magazines and newspapers too? They may need to be displayed at iPad size, until publishers begin supporting fluid or multiple layouts.
Find my Friends, Find my iPhone. The map is already there, dropping some pins on it seems simple enough.
Weather. Simple enough, with room for even more data. Full screen mode could be gorgeous too. While we’re at it, let’s do the same for iPad.
Clock. Alarm, stopwatch, timer, world clock. No reason a Mac shouldn’t have these functions right out of the box.
The Mac may not be the ideal device for many of these apps. But the best device for the job is the one you have on you. No reason to make me get off the couch, or worse, open a web browser, just to check the weather.
The case for a Google Glass competitor from Apple
If Apple wanted to compete with Google’s Glass (and surely they do), I think an interesting and appropriate strategy would be an updated Siri designed specifically for use via headphones plugged into an iPhone. Just as glass users get constant notifications and interaction in the corner of their vision, Siri could periodically deliver similar services to the ears. She could also be called upon with a likely optional, Glass-like “okay Siri” command.
Interaction would be would be as simple as carrying on a conversation, which could even be initiated by Siri herself. For example:
Siri: A new email just arrived from your sister
User: About what?
S: Vacation Plans. Would you like me to read it?
S: The email reads: ‘Hey, I’d like to go over some of our plans for the vacation. What days do you have available in October?’ Would you like to reply now?
U: No, but remind me when I get home
S: OK, I’ll remind you to email your sister when you get home.
In this example, Siri is set to announce the arrival of any email from your VIP list. She understands that your question “about what?” could be answered by reading the subject of the email. She then offers you additional assistance: to read the contents of the email and to reply to it. She also understands that “remind me when I get home” refers to the email you were just talking about, so she sets up a geofence and creates the reminder.
Siri is already quite close to doing many of these things. She just needs to have a better grasp on the idea of a conversation, the ability to initiate one herself, and to recognize that your headphones are plugged in. (A mode could exist for her to converse over the speaker, but in many situations that might be too awkward or reveal private information, so I think headphones are best.) And of course she would certainly benefit from being faster.
Here are a few ways Siri for headphones could be better than Google Glass:
- The hardware, specifically headphones, are common, cheap, included with iPhones, comfortable, and socially acceptable
- Headphones do not need to be charged
- Users are already very comfortable and accustomed to wearing headphones in public
- Bluetooth headsets would also be compatible
- Spoken notifications are potentially less distracting than visual notifications
- Headphone-Siri could be made available to millions of existing iPhone users at no additional cost
Of course, there is no camera, but some might see that as a benefit. There’s no display for video, photos or menus. There’s also no touch-control, but could a rudimentary interface be made from the playback remote? Click once to accept, click twice to decline… And the phone itself is always close by for any advanced functionality.
I think we’d all love an API for developers, but that’s a different story.
In general I am still skeptical about conversation and voice as a user interface. I don’t think users are quite ready to talk to their devices for everything. But in the right situation (Driving, walking, or otherwise preoccupied) it could be a reasonable way to get something done.
Wipe the windows
The purpose of a windowed user interface, simply put, is for multitasking. For seeing several things at once. For dragging or pasting data between documents and applications. A browser window and a page of notes. Email and a calendar. Perhaps a music player or a twitter stream in the background.
Yet managing these windows is not perfect. They are flexible, but they are really quite inelegant. And much of the management must be tediously done by the user. Windows can get pushed offscreen, overlap and obscure content, or be lost entirely. And everything covers the desktop. We’ve come up with clever ways to deal with these issues: Mission Control, the “window” menu, “Expose”, “hide others”, “bring all to front” and so on. But these are just proof that dealing with windows is far from perfect.
I propose a new system that divides the screen into sections. For example you might have a section on the left half for a browser window, and two sections stacked on the right half for twitter and a to-do list. Making the browser section wider would simultaneously shrink the other two sections. The content in each section would adjust to best fit its new size, not unlike current windows, but now you’re essentially resizing everything at once, without any overlap or unused space.
Developers might be encouraged to support more sizes and create more responsive layouts, but for the most part, we’re not far off from supporting this today. Tabs will be more useful than ever, allowing users to keep several windows in one section.
Mac OS has lately been following the trends set by iOS. One such trend has been the full screen experience. And with smaller screened devices this is often an appropriate use of space. However, the more screen real-estate you have, the better you can take advantage of seeing more at once. On 15, 21, 27 inch screens, using an application in full screen is often a terribly inefficient use of space. With my proposed system, the full screen experience need not be limited to a single task.
A mission-control-like system could be used for moving, resizing, adding, and removing sections. Spaces would work well to group various tasks together (and I’d welcome return of the 2-dimensional grid). Common window groupings and sizes could be maintained across sessions.
Menus. If several windows are now spanning across the screen, where do you put the menu bar? This is about to get a bit blasphemous. Mac users (myself included) have long argued that a reliable menu bar at the top of the screen is easier to find, easier to point to, and just generally better. However, I foresee a change of times. More and more the menus hold only advanced features for advanced users—users who might often be using keyboard shortcuts anyway. My thought is to add a button to every app that produces a menu of menus where it can stay out of the way until you need it.
No window. A mac app can run with no windows open. It’s a nice feature, when you want a download to finish or a song to keep playing. And to get something going again you go to the menu bar or click on the dock. How does this work with sections? I’d say clicking an icon in the dock would still open a new window, and a click-hold on the dock icon could give access to the menus as well.
Active window. If two windows are guaranteed not to overlap, must there still be an active window? I would love to say no, but what to do with conflicting keyboard shortcuts? I think there must be an active window for that reason, but click-through rules can be greatly simplified.
New windows. Oftentimes an app will create a new window, clicking compose in Mail, for example. A few options come to mind: A new section is created for the new window, the app presents the windows as a tab, the new window is presented as a modal above the first window. Ideally developers would be able to decide which actions produce which results.
The Desktop. Perhaps it’s time to lose that too. A full screen finder window would be a fine replacement. Or perhaps a new app called desktop? LaunchPad might finally feel like it fits. And the dock can certainly stay as a quick way to jump to open apps.