10th January 2016
The iPad Pro and the Apple Pencil are amazing products but they aren’t any good without good software that supports their hardware features. Naturally the Notes app on the iPad itself has been optimised for the Pencil, and it shows with excellent pressure sensitivity and tilt support with some of the cleanest lines you’d like to see.
It has one big drawback though: there’s no OCR and with it no searchability. Maybe I’m asking too much but this isn’t the days of the Palm Pilot where I learned to write in "Graffiti" one letter at a time. OCR is becoming quite common and honestly with support now in OneNote it’s becoming very mainstream as Microsoft are leveraging their decades of experience with pen support.
Apple don’t have a native solution for it on iOS and whilst I recognise that typing will always be significantly faster than handwriting, in engineering I spend a significant amount of my time making hand-written notes and drawing diagrams with a reasonable amount of text. Being able to search that handwritten text would be a dream come true.
OneNote to the rescue - kinda…
The problem is that in my office environment access to the servers is firewalled and I have to tether to get external access to my OneDrive for OneNote sync and Microsoft do OCR on the notes on the server side, feeding the results back to the document when they’re done. They suggest 5 minutes but I’ve waited hours. The advantage of that method is it’s genuine image OCR and it should therefore work for any image of sufficient quality in the OneNote document (more on why that matters later) but it requires the server side and that’s a problem for me in my use case.
I need something that works locally on the device. Maybe it’s coming someday but for now, OneNote can’t help me.
Good Notes 4
I’d spent a week or two of work with my iPad Pro just prior to Christmas and used Notes and it was fine, except it began to mirror my real-world notes problems: they were an unsearchable, poorly indexed mess but at least I could look at them wherever I was without having to drag my engineering notebook everywhere with me.
I’d researched a lot of software and after watching several how-to videos forked out the $7.99USD ($12.99AUD) for the iOS and shortly thereafter another $7.99 for the Mac version because it blew me away so completely.
GoodNotes 4 allows you to create Categories for your notes starting you off with "Uncategorised" and "Trash" but I simply added Work and Home and went with those. When you create a Notebook you can choose several different kinds of Covers with options ranging from Bright, Calm, Dark, Enclosed, Plain and Simple: each with a subset of interesting patterns. Being me, I stuck with the default cover: Simple Blue.
The first page (and each subsequent can be added prior or after any page you’re on quite easily) gives you options for different paper sizes with pre-ruled and graph-style papers. My two favourites I use are Standard (A4) because that’s the standard in Australia, and I always choose between Narrow Ruled (if I’m writing a lot of words), Narrow and Quad Ruled (aka graph paper for sketching diagrams) or just Plain (just for freehand sketching).
Swiping between pages is easy with a two-finger slide left or right, with pinch to zoom works as expected plus there’s a handy zoom in close function for fine work that all worked nicely and were quite polished compared to the standard Apple Notes app.
A single tap along the top selects your drawing implement be it a stylus as a pen, a highlighter, an eraser or to enable the handy Lasso tool for selecting areas of your notes and cut/copy/paste/resize them as required. A second tap for those drawing implements with colour selectivity brings up a palette with line thickness control as well. There’s shape recognition which I found to be passable but Grafio does a far better job as does OneNote.
There are options to sync your content via iCloud which works fine between my iPhone 6S+, iPad Pro and Macbook Pro, but also includes Import/Export options for Google Drive, OneDrive Box and DropBox as well as using DropBox for Automatic backup beyond iCloud if you so wish. You can also import content from other iCloud Drive sources, Photos or the Camera directly.
Ultimately though that’s not what hooked me.
Searching Handwritten Notes
Open a workbook and select the ellipsis in the top right corner then "Search" and type in the text you want to find. GN4 then does a real-time handwriting recognition search through the workbook and highlights in yellow all instances it found of the text. It works on the device in isolation, which I tested by killing the WiFi, writing something then searching for the text I’d just written.
Selecting the search entry takes you straight to that occurrence as per any standard typed-text search. My handwriting cursive is terrible so I switched to printing when I was in Grade 7 to make my handwriting legible. It’s not as fast as cursive but it works and you can usually understand what I’ve written. Unlike the days of the Palm Pilot and Grafitti, GN4 had no problem recognising my handwritten notes provided I didn’t overlap too many letters.
I’ve only used the English (US) handwriting recogition so for other languages I can’t make comment. I did try to trip-up the software by underlining through y’s and g’s but it didn’t skip a beat.
As you can see my cursive (bottom) is terrible, and my slow printing is quite readable, with my normal writing speed for notes is, relatively okay. That said, GN4 found the search text "speed" in all three sentances.
GN4 also found "friend" in the cursive section. Not that I’m intending to write in cursive though.
The Not So Good
Like all things, it’s not perfect and it would be remiss of me not to mention the issues I’ve encountered.
- Palm rejection is somewhat twitchy when I’m taking notes in Landscape the most common annoyance is the iPad bringing up the app-switcher randomly and drawing a line across the document in the process that I see when I return to the app. There is a hand-angle adjustment I’m playing with to try and improve that but it throws me out about once every 20 minutes or so of writing. The Notes app never does it, so I’m assuming that will improve with time.
- Broken lines appear from time to time when I’m writing quickly and it might be based on pressure sensitivity though it’s hard to be certain. Notes does a better job but the broken lines don’t seem to affect the handwriting recognition as it’s only a 2 or 3 pixels and it’s really only a visual annoyance if you zoom in closely.
- Handwriting search being restricted to the open workbook only is honestly my biggest gripe. When searching workbooks at the top level screen it only text-searches the workbook titles. To search for handwritten text you need to open the workbook and then search within it. If I’m trying to organise my workbooks for different purposes but need to find something I wrote I need to open them all in turn and search individually.
- Handwriting search is not OCR in a traditional parlance and at first that doesn’t seem to be a show stopper until you try and import notes taken in another app, like Notes. To get handwriting recognition to work on my old notes I’d taken, I saved them as images to the Camera roll, imported them into GN4 then I traced them over the top and deleted the original image. That’s fine if you have 20 pages of notes like I did, but if you have more it’s a big deal.
- Desktop app is an iCloud viewer for the moment at least as it’s only been out for four months. It’s what I mainly need (handwriting search works here too) but I’m hoping in future they add synchronising to other sync-services and some editing features but honestly I’m using it as a reference on my desktop/laptop. I enter/write all of my notes on the iPad Pro anyway.
It’s a rare issue but broken lines aren’t too bad at normal zoom.
Zooming in the broken lines are quite pronounced. It’s not a common problem but I never experienced this with Notes app.
How I’m Using It
To overcome the restrictions on search I’ve created two Categories, one for work and one for home each with a small number of workbooks in them. For Work in particular where I’m going to need as much searchability as possible I’ll be keeping a single workbook for every 3 or 6 months (let’s see how unwieldy it gets after 3 months) before starting a new one. This is analogous to how I operate in the real world and in future hopefully they will add searching across multiple Workbooks or I could just copy and paste each into a master "searchable" file.
I’m still hoping for an open-pen format that’s widely supported for pen-captured data that would allow handwritten notes to be copied and pasted between documents. We have that for text but OCR can’t be the answer for every such problem. That said, searching handwritten text allows me for the first time ever to take notes the way I always have with handwriting but not require paper or a pencil, and then have all of my notes available on all of my devices; not stuck in a set of physical notebooks that turn ragged and fall apart and requires storage for years.
We are so close to going paperless. I can feel it.
10th January 2016
A great deal of digital ink has been spilled (pixels have been randomly arranged?) over whether the iPad is actually a useful productivity tool and much of it has been written by artists, technology journalists and bloggers, but I have yet to see much of a comprehensive examination of the iPad from an engineering perspective.
I know several other engineers that aren’t interested in the Apple cult, and prefer the configurability of Microsoft products of the past and they latched on to the Surface when it came out however reports of its sluggishness and heft as a tablet made it more of a laptop with a detachable keyboard than an actual tablet, and the Surface Pen was not held in very high regard either. I heard of issues with palm rejection and accuracy as well as lag, but observation is the best tell and I’ve observed those people using their Surface almost exclusively as a laptop, and seldom if ever using the Pen.
As I am more personally invested in the Apple ecosystem, I’ve owned and throughly used an iPad 1, iPad 2, iPad Air, iPad Mini, iPad Mini 2 and now an iPad Pro. In that time I’ve used a Griffin 2-in-1 Pen/Capacitive Stylus, an Adonit Jot Pro and now an Apple Pencil. So with those qualifications out of the way…
What’s Special About Engineering?
Engineering involves conveying a lot of information diagrammatically and it’s not the only profession that does. In that sense marking up drawings, flow charts, red lines and whiteboards are our tools for this job, but what frustrates me more than the old tongue in cheek comment "the age of the paperless office" which apparently happened in the 80s but the world must have missed it somehow, as I intuitively know that moment is coming. What device/service or combination thereof will finally deliver on that promise-turned-running-gag?
To be effective as an engineering tool for me personally it needs to solve several problems I have:
- An engineering notebook everywhere that stores sketches, handwritten as well as typed notes
- Accurate markups of PDFs without loss of resolution
- Creating flow charts and diagrams quickly and easily
- Portable enough to take everywhere with me
- Able to physically sign documents, in Word and PDFs accurately
The goals are to eliminate three issues I encounter every day at work:
- I need to carry three pens everywhere (Blue, Black and Red) and they leak, they get lost, they run out of ink, they suck
- I currently have to print a document, write my markups on it, then scan it in again which erodes the resolution every time and makes the original document unsearchable even with the best OCR software currently available.
- I make notes in my engineering notebook (a legal requirement) and need to take it everywhere with me but its size is limited so I move from book to book with transitional periods between books requiring that I carry multiple at once.
iPad for Engineering: Take 1
The iPad Pro for engineering is therefore first and foremost about the stylus, but the ability to use the device with a touch screen keyboard would be a huge plus as well if it could pull it off. That said I’d tried styluses before without much luck and touch screen keyboards as well without any success, so call me skeptical from the outset.
I drew several Enginerd comic strips using my iPad Mini 2 and the Griffin and Adonit mentioned previously but to be honest it was difficult and frustrating. I tried to use older iPads and styluses at work and the two big issues that kept arising were:
- Stylus accuracy and speed made drawing markups, signatures and notes effectively impossible to do
- I found myself regularly erasing and trying strokes and signatures again and again and eventually giving up (signatures bared no resemblance to my signature with a traditional pen and paper)
- Typing was slow because the key sizes on the Mini were too cramped and on the original iPad the typing lag was beyond horrible much of the time
Of course the typing problems of an iPad can be overcome in some ways by using an external keyboard. To that end I used an original iPad keyboard dock with all of the models from time to time, even using a 30-pin dock to lightning adaptor with a box of old business cards to support the weight of the iPad (a trick that worked fine until the iPad Pro came about). It was excellent, however only useful on my desktop at work and for portability to meetings and on the train it was useless.
I also tried the Clamcase, but it just wasn’t stable enough even on the train with the keyboard component regularly disconnecting and being horribly sluggish at the best of times. I wrote about it, I podcasted about it and honestly I gave up on the iPad as an engineering productivity tool.
That was, until the iPad Pro and the Apple Pencil were released.
iPad for Engineering: Take 2
Ever willing to give technology another chance, I obtained an iPad Pro, Apple Pencil and Smart Cover and took it to work for several weeks in the lead up to Christmas determined that if it didn’t work out, I would return them. If they couldn’t earn their place then they had no place. The TL;DR: I still have them all.
Double Touch Typing
Yes I did a year of typing classes at high school and I’m a touch typist as a result of those hard yards. Keyboard key spacing is critical when I’m going from device to device. I’ve tried many times to use touch screen keyboards for touch typing, but there have always been two issues: No locators for your index fingers (see those notches on your F and J keys on your physical keyboard? Yeah those things) and the key-sizing and inter-key spacing a were always just off/small enough such that when typing on the screen like it was a physical keyboard, many keys didn’t align. Typing on older/smaller iPads was slower, very inaccurate and ultimately frustrating typing and in the end I just gave up.
When I’m talking about key-sizing and inter-key spacing, what I’m referring to is that the physical dimensions of the screens on iPads make it physically impossible to create a virtual keyboard that matches a physical equivalent. The same is true of the iPad Pro, actually but it’s damned close. The key sizes and spacing on the iPads 1, 2, Mini, Pro, Smart Keyboard and a Apple Wireless keyboard for comparative purposes as shown below:
|Device / Keyboard||Width (mm)||Height (mm)||Key Spacing (mm)|
|iPad Mini Landscape||12||11.5||14.5|
|iPad Pro Portrait||12||11.5||13.5|
|iPad 1 & 2 Landscape||15||14||17|
|iPad Pro Landscape||16||15||18|
|iPad Pro Smart Keyboard||15||14.5||19|
|Standard MBP and 1st Generation Aluminium Physical Keyboards||16||15||19|
A real world physical keyboard is 273mm wide from the left hand edge of the Caps Lock to the right hand edge of the Return key. The full width of the iPad Pro screen is only 263mm, and that missing 10mm has to come from somewhere. As you can see from the table above, Apple shaved it off the inter-key spacing, which is the obvious choice and honestly the one I would have made as well.
Touch Typing: How Fast is Fast?
As a way to test just how different the typing was between the touch screen and a real physical keyboard, I took a good old-fashioned typing test, using a 1st Generation Aluminium Apple physical keyboard as well as the new Smart Keyboard vs the iPad Pro touch screen keyboard in landscape mode. I used the app "TapTyping" and each test was performed three times in each configuration and the best time taken from each.
Just for good measure I threw in a test on the same hardware and software by using the iPad Pro in portrait mode whose keyboard dimensions closely approximates my previous failed attempts to get screen touch typing utility from the iPad Mini keyboard. Hence, same software, same hardware iPad with only the keyboard data entry as the variable. The results:
|Device/Keyboard||Speed (wpm)||Accuracy (%)|
|iPad Pro soft keyboard in Portrait Mode||39||94|
|iPad Pro Smart Keyboard||63||96|
|iPad Pro soft keyboard in Landscape Mode||67||97|
|iPad Pro with Apple Bluetooth Keyboard||85||98|
It’s important to note that the typing test accuracy does not rely on auto corrections and letters must be corrected prior to proceeding. That said, the occasional need to glance down at my virtual keys really hurt my speed and the slightly different key spacings also hurt my accuracy, but it’s the same old story: physical keyboards will always be faster for raw speed.
Without a physical edge to feel for, once your fingers are visually aligned on the F&J keys, the maximum error on the iPad Pro is now only 1mm for the standard keyboard keys. For me at least, that turns out to be the threshold of usability for the first time for touch typing on a touch screen. (I henceforth refer to that as double touch typing)
I’m still slowed down initially when I glance down at the glass to position my fingers for the first letters but after that it doesn’t require any further thinking or retraining and I’m happily typing away. My fingers sometimes need to realign on the stretch keys that can drag my hands away from their home position over the F&J keys which then slows me down.
The verdict though: the iPad Pro is by the far the best and perhaps only iPad out there that I can double touch type on.
The Smart Keyboard is designed to work only with the iPad Pro and uses the three small and well disguised pins located on the left hand side of the tablet. It has several folding configurations depending upon whether you want to use the keyboard or not, or in carrying mode. All of these took a fair amount of practice in the store to get your head around.
I tried typing on the Smart Keyboard in an Apple Store several times for up to 30 minutes at a time trying to decide what I thought about the key mechanism but I ultimately came away with the impression that it felt rubbery, sluggish and annoying.
Interestingly though, despite being a physical keyboard, I wasn’t able to type faster than the touch keyboard, but as the results show it’s not by much of a margin. The marginally smaller keys and texture of those keys made it harder to feel the edges than the standard Bluetooth keyboard but those locating notches were nevertheless helpful.
The truth is however, if I wanted to lug around a physical keyboard it needs to have more normal physical keys on it. The keys on the Smart Keyboard feel like a material compromise too far, most likely as a weight reduction measure, thinness and possibly also for moisture resistance, but either way it doesn’t feel that great, and for that kind of money it needs to.
I wanted to try the touch typing experience on the touch screen before I spent money on the smart keyboard and with the results of the touch keyboard being promising, for the moment at least I think I’ll pass on the smart keyboard. If I didn’t have a laptop and the iPad Pro was my only device then I could understand buying a physical keyboard as well.
Such a big screen needs some sort of protection however so if you’re serious about protecting your investment then the ideal typing experience could include a Smart Cover and an external keyboard of some kind. To that end I looked at two Apple-based options.
|Device||Cost (AUD)||Cost (USD)||Weight (g)|
|Apple Wireless Keyboard||$165||$99||231|
|(AWK + SC Combination)||-$15||-$11||+56|
The best typing experience would be to buy the Bluetooth keyboard and the Smart Cover and only take the keyboard when lots of typing was anticipated. I dare say for the vast majority of tablet owners the Smart Keyboard will get little use if they have another computing device with a genuine keyboard. I think, to borrow Tim Cooks expression, the keyboard makes the iPad a more "confusing product" than it needs to be. Especially when you consider just how good the double touch typing experience now is on the iPad Pro screen.
Also to prove the point that the touch screen is plenty good, this entire article was typed in double touch.
I’ve always dreamed of having both a responsive and an accurate stylus but until I tried this one, I hadn’t found it. Yes I did the slow motion thing and I know there is still lag, and yes some apps haven’t been updated to fully utilise the Apple Pencil specific APIs. Honestly though, those apps that have been updated are a dream to use and the drawing lag is barely perceptible unless you’re trying to find it.
Before we get to that though, as a drawing device I had no option but to compare them with the competition:
|Writing Implement||Weight (g)|
|Papermate InkJoy boring old normal ball-point pen||7|
Of course it’s not a fair fight since the Apple Pencil is an active device with a battery and such, and the metal case of the Adonit makes it quite a bit heavier, but the thing I noticed the most was that the Apple Pencil feels like the heaviest pen to hold in your hand when you’re writing like due to its weight distribution. I sometimes remove the lid from pencils to make them lighter and reduce their overall length when using them for longer periods. (Reduces hand fatigue) This also has the effect of reducing the pressure I write with when I do so. The plug/cap on the end of the Apple Pencil doesn’t make much of a difference to either the weight or the length of the device and it still feels just as heavy.
Okay that’s a bit of a nit pick for sure but I personally don’t like a hefty pen when I’m writing with it for a long period of time. It feels unnecessary and adds no real value since it appears as though they’ve added steel to weight it to prevent rolling and also to be attracted to the magnets mounted in the iPad. Directly comparing it then, I’ve found using it for longer periods tires my hand more so than a traditional pen would, which is understandable given the additional weight.
I can get used to that of course, since I used to be able to write six pages of text notes during a two hour lecture 20 years ago without my hand cramping. Those were the days…(insert old man comment here)
Carrying the iPad Pro between meetings is no different to bringing my traditional notebook, and the battery life easily lasts the day with practically constant use. I do get annoyed about the short battery life of the Apple Pencil with a full day of use in my job requiring an emergency charge mid-afternoon on two separate days. That said, the charging was quick even though inserting it into the Lightning port looks ridiculous and is particularly prone to accidental damage if you or someone close by is careless.
I started out using the built in Notes app for taking my notes but moved to GoodNotes shortly thereafter and even transcribed my notes into that app for its amazing handwriting recognition features. I’ve been using Grafio for charts and diagrams and PDF Pen for marking up PDFs. Sync services via iCloud and Dropbox for those apps work really well but refer to my individual reviews of those apps for more about the software component of the iPad Pro equation.
Having owned and extensively used all of the iPad form factors now, I can safely say that the size extremes have clear use cases with the mid-size model being a bit of both.
- Mini is for single handed use and great for reading novels
- Pro is for two handed use and great for newspapers and comics
- Air is a bit of both and balances portability with the above two
There is something magical about being able to touch the screen and see things respond directly to your touch. When I used my first light pen in the late 80s I was impressed but it was nothing like what we have now. I worry that people get a bit blasé about it with the near ubiquity of smartphones these days, but the fact remains the immersiveness you feel interacting with a touch device only draws you in more, the larger the screen is.
I use my laptop on the train a few hours each work day but primarily at my desk. I use my iPad on the couch or in a comfy chair. For reading Twitter, the newspaper, Instapaper, my RSS feeds, and just surfing the Internet the iPad Pro has been the best iPad yet for those tasks.
I thought that I would miss the one handed operation of the Mini but ultimately I just prop the Pro up on my leg or knee bent at a comfortable angle and don’t really hold it at all and I’ve been fortunate to spend a few hours in that position over the Christmas holidays and it hasn’t been an issue.
Video and audio playback is simply amazing with excellent bass reproduction and the volume can easily fill a small room. Much to be happy about there.
I’ve been avoiding using my iPad in bed for a few months now since I’ve trying to avoid artificial light before bed so that hasn’t been issue either. In short: one-handed operation of an iPad turns out to not be a big issue for me. Your mileage may vary.
There is no doubt that achieving a truly paperless office is a challenging task. Asking yourself the question: ‘do I really need that printout?’ certainly helps, but truly collaborative software tools are only just now becoming available that allow the sorts of digital collaboration we need to bring ourselves over the line technologically speaking.
So long as organizations and legislators rely on wet ink-signatures, we’ll be stuck with paper. So long as employees are given laptops and not tablets and styluses for their jobs, we’ll be stuck with paper. Where we have a choice, or decide to draw a line and stump up our own funds to try and escape the shackles of the paper world and all of its flaws, I think going paperless is absolutely possible.
The iPad Pro and the Apple Pencil, in conjunction with GoodNotes has allowed me to completely ditch my written notebooks. I can search those notes with good accuracy, and I can double-touch type on the tablet with no external keyboard or special (heavier) cover required, at quite a respectable speed for the first time ever.
For me at least, I’ll be keeping this device and using it for my job every day, and for many days to come.
10th October 2015
I’ve been working on a project for a few months now with the aim of providing a platform for other engineers that want to create a podcast and get themselves out there. It’s a podcast network and it’s called "The Engineered Network" and it launches today with three shows: Pragmatic (returning from the ashes), Causality and Neutrium.
Rather than reiterate what I’ve already said on TEN, check it out directly.
What that means for TechDistortion should be clear: I’m focussing on TEN for the forseeable future and TD will only be updated occasionally as I have something to say that isn’t network-related.
Thank you in advance for your support everyone.