Take a little break and watch the performance of 'The Great Patriotic War' here. Link to large size video, which I recommend. 8:30 long.

There are three exceptional things about this video. One — it's great art — enthralling performance, emotional themes, beautiful imagery. Secondly, the performance itself is technically amazing, yet, apparently the artist has only been doing this for one year. Finally, all of this is being achieved with some plain sand on a flat (backlit) surface. The tools for art don't get much simpler.

And yet…this is exactly the type of real-time, subtle, organic, sensual and fast art I always imagine computers could be capable of. Unlike many swooshy multitouch demos, this is not art for art's sake, instead the animation covers very human topics; one of every four people in the region died in WWII's Eastern Front. And she's using every last creative aspect of sand, from brushing, to finger and palm painting, throwing sand and scraping with the edge of her palm.

Two Hands are Better Than One

So this is how great it can be with some sand. How about some silicon? Matt Gemmell wrote a great piece on iPad application design I enjoyed. On the topic of the iPad's large, multitouch area, he writes…

The important point is that there are other, more obvious ways to accomplish these things; the two-handed input features are conveniences and power-user features. They’re useful and time-saving and possibly discoverable, but they’re not the only way to accomplish those tasks. We’re only just beginning to come to terms with the possibilities of dual-handed input; essential functionality shouldn’t require it yet.

You can see in the video that Kseniya rarely uses two hands. My stopwatch recorded only 1:15 minutes of two-handed use in the eight-and-a-half minute performance. That is, she only uses two hands simultaneously in this performance — 15% of the time. When she does, it's to do something quickly like clear an area. She also seems to use two hands when she's wants to draw symetrically, like the hair at 3:43.

 The matter is not that simple though. Many times she switches hands in the performance because she wants to draw on the far left (she appears right-handed) or because she wants a particular shape, or needs to approach from a particular side. 

Sometimes she switches for speed, and artistic effect; alternating left and right throws.

Just the Tip(s) of the Iceberg

I love this video because of the richness in the interaction. It's an encyclopaedia of gestures, from a single finger-painting, to multi-finger dabbing, parallel lines with thumbs and middle-inger. French-curve arcs with a palm, broading erasing strokes with the whole hand and intricate air-brush effects with sand released from above. I agree with Matt: we are at the beginning of this whole wonderful adventure. I'm going to keep Kseniya performance in mind as something to strive for. This is a great interface. 

iPad’s revolutionary Multi-Touch™ interface makes surfing the web an entirely new experience…

 

That's Multi-Touch (TM). I'm amazed it's possible to trademark that.

People who are really serious about software mockups should make their own hardware mockups.

The backstory is as follows: ReadWriteWeb had published a piece about Facebook. Through the magic of PageRank, this page became the top Google listing when you searched for "facebook login". Here's the unexpected bit: comments started pouring in declaring this new Facebook redesign to be terrible from people who thought the ReadWriteWeb page WAS Facebook. Some viewers of this saga took this to be a display of the utter stupidy of the majority of Facebook users. Others interpreted it as a demonstration that technology was poorly designed. To my understanding, Jono's piece breaks the problem down into a dichotomy between a simple web for simple users, VS a web with educated users. The solution he proposes would involve educating users about URLs, to result in the latter. I don't quite agree.

I like education. It's a wonderful, empowering, liberating thing. I also like stories. Let's start with one of them.

Twinsong

The year was 700AD. The Angles and the Saxons had been living in Britain for a while, having migrated over from Germany. Apparently everyone forgets about the Jutes. They were there too. But we'll forget about them. Anyway, this bunch spoke their own language: Anglo-Saxon. Though it laid the foundations for modern English, some fundamentals differed;

[From a truly excellent course guide on the history of Anglo-Saxon by Professor Michael D.C. Drout. I highly recommend listening to the Audible version for it's out-loud Anglo Saxon]

In Anglo-Saxon, you could say “dog cat ate,” “ate cat dog,” or “dog ate cat” and still not tell your reader who did the eating and who got eaten. Instead of relying on word order, you would put a tag on who got eaten, so “dog ate cat-ne” and “cat-ne dog ate” and “cat-ne ate dog” would all mean the same thing and would be different from “dog-ne ate cat” or “cat dog-ne ate” (whoever gets the –ne is the one who gets eaten).

Got it? OK. A hundred years later, the Vikings invaded, and stayed. Things got confusing for a while with the Danish and Norwegian native languages intermingling with Anglo-Saxon. Later the French rocked up. Bah. What a mess. Jump in your DeLorean, punch in 2010, and —sssSPOW!— we find that English now uses word order, not tags, to define whom ate who. Dog ate cat. Not cat-ne ate dog. Point being that unless you know the rules, it can be very confusing.

Let's continue the story here in the modern era. I remember Tech journalist Leo Laporte* telling the following story…

Leo wanted to share with his wife his newly created streaming tech-news site. I'm sure he must have been bursting with pride of his accomplishments of duplicating the functionality of a multi-million dollar broadcast station in a single room in his cottage. He instructed his wife to visit live.twit.tv to enjoy the fruits of his labour. But, she accidentally typed into her browser something not quite right — twit.live.tv. After seeing the result she demanded an explanation. I can tell you the site she found is Not Safe For Work and doesn't contain in-depth comparisons of iPad vs the Kindle book reader.

Same problem, different millennium. 

The URL bar seems to me to be the last bastion of CLI that the average joe is forced to deal with. All these slashes and dots. Get one letter or dash wrong, order some words around and it all goes to hell. So, users give up and just type into Google. 

So what the heck does the URL actually do, from a UX perspective?

1. Lets you go to a particular place on the web by typing something in
2. Tells you where you are, for example if you've been clicking links and find yourself on a new page, or site
3. Provides a secure 'where am I' information. If you're browsing in the Western World, facebook.com usually means you're at facebook.com

Now I don't know much about networking. I haven't been following browser/ HTML5/DNS debates. BUT what I do know is that URLs are a weak point in the user experience. In the words of MC Hammer, breakitdown… 

1. Lets you go to a particular place on the web by typing something in

Bit of a fail here, as we've already discovered above. One of the problems is that the semantics are not understood by average punters.

How about we have the browser help?

 The important stuff is big and bold. Much more could be done. Props go to the Mozilla team for the Awesome bar, which is forging ahead in usability. 

2. Tells you where you are, for example if you've been clicking links and find yourself on a new page, or site

Fail, because people aren't watching/noticing. I really like the little icons that appear next to the URLs — favicons. I'd love a bigger version. How about we colour the chrome of the browser with it? Facebook uses 'you're leaving Facebook.com' dialogue boxes to let people know they're going out in the Wild West. Effective, but clunky.

  3. Provides a secure 'where am I' information

Smells like a Pareto Paradigm to me: 

• Many people, not just a few (hence the newsworthiness of the story) are accidentally visiting the wrong site
• There's very few popular browsers that many use: Firefox, Chrome, Safari, IE, Opera.
• Of all the web's sites, there's a few that many non-savvy surfers use: Facebook, Myspace, Hotmail, GMail, Yahoo, Ebay, Amazon

It follows that the big browsers could have a whitelist for sites they know non-savvy users would be visiting and show some warning for sites that look like they're trying to pretend.

As for Verified sites: Firefox is doing the best job of the browsers I use, showing that a site is verified. FF displays a large green button for 'verified'. But that only means something if you're expecting it to appear. Classic UX design problem: If you don't know, (or forgot) that it's supposed to be there, you won't notice when it's not.

How about a standard symbol that would appear in a browser that a page could reference "you should be seeing this symbol"? Or some better specification for integration between the browser and the secure site itself. 

You May Argue

"It's not that complex to learn how a URL works!"

Ever seen someone double-click on a link to 'open' it? They learnt to double-click on desktop icons in order to open them, and are now applying that rule to the web. And here's the problem: there's a lot of arbitrary technicalities in computing that the users are being asked to learn about. I'm all for the education that Jono suggests. But general, reusable knowledge. Not the inner gears and ratchets of a mechanism conceptually born two decades ago that we call the World Wide Web. 

"If we hide the guts of the internet away from the average user, won't they become docile clueless consumers?" "One day they'll find themselves locked into some DRM encrypted, Apple/Microsoft/Google-only internet."

I like the way Google Reader handles this. The average user may not know anything about RSS, or where to find the appropriate RSS link. Instead, they just plonk the site into Google Reader and it works out the appropriate RSS URL for itself. The RSS standard is not compromised.

"If the user isn't savvy enough to see the huge ReadWriteWeb banner, how are they to notice anything more subtle? AKA "what hope is there for these losers?"

The perceived complexity of the URL bar is a self-fufilling prophecy to failure. People don't understand it, so they don't use it, so they don't notice when it displays something different to what they should expect. 

* Interestingly, http://www.leolaporte.com/ is hijacked. If the 'president of the internet' doesn't have a URL, what faith can we have in URLs at all?

 ** Bonus points to anyone who gets the Twinsong reference to the stories

OMT in San Francisco #3: 'Let it beep' from One More Thing on Vimeo.

Video interview with Jim Reekes creator of that fat C chord boooooonnng.

NOTE: Introduction is in Dutch, interview in English. 

via @jasperhauser

"It's something that we used to point at objects on a computer screen"

"Just one thing at a time?"

"Yes honey."

"Wow! But how did you do this?"

[She resizes a square with two fingers and then touches the others to propagate the change]

"Well…in the past it was different. First you need to select all the objects you were interested in, by clicking in a space nearby, then dragging an imaginary rubber band around them all. If they weren't next to one another, then you needed to hold down Command on the keyboard while you clicked on each one. Then you would adjust the size of them with a separate control panel at the side of the screen. Or you might size one how you want, then press Command+C to copy, then Command+V to paste the squares…… are you listening?"

"No, sorry Daddy, that's all too technical for me. I don't know how you remembered all that in the old days!"

In the future, our children will all use rich multi-touch devices. They will look at the mouse & keyboard combination in the same way we today look at the Command Line Interface.

Tomorrow…

Apple is rumoured to be announcing a new Tablet device. You probably know this. Rumours of it being shiny and thin (which it probably will be.) How it will be always connected to the internet, and show you books and newspapers and movies on-demand (which it probably can.) How it will have some magical new jaw-dropping interface (which it probably will have.)

But what excites me most is a possible feature that no one seems to have thought of. It's not sexy, and it's something we use everyday on our desktop machines. In fact you probably can't remember computing without it. And yet, I feel it's the key to the future of computing, and without it, the Tablet will not be able to spawn the New Age of Computing. So what's this amazing technology? I'll tell you: mouseOver. You know, the feature whereby links on a page change when you mouse over them, buttons darken and tooltips appear. The subtle interaction that lets you learn more about an interface without committing to anything as serious as a mouse click.

Of course, the Tablet is all about Multitouch -insert choirs of angels- so there's no mouse to be seen. Just a finger or three. So let's call it 'touchOver'. Imagine icons that darken, lighten and pop-out as you waver your finger over them like a tantalising box of fancy chocolates. 

So, why bother to include an interaction feature from the past?

First, let's look at the existing benefits of mouseOver in desktop and web applications:

1. Users feel more comfortable with unfamiliar interfaces, exploring without the commitment of clicking
2. The user has feedback which helps them "aim" their cursor

Both of these are valuable. But in multi-touch interface history I see rare mention of support for the touchscreen equivalent of mouseOver. I don't know why—maybe it has been technically difficult to cleanly detect fingertip position as they hover over a touch-surface. Maybe the interaction design was never solved. Maybe I've been looking in the wrong places. Maybe it wasn't deemed necessary.

But. Fast forward to now — see a recently awarded patent to Apple…

[0095]Another potential technique for the determination between "hovering" and "touching" is to temporally model the "shadow" region (e.g., light impeded region of the display). In one embodiment, when the user is typically touching the display then the end of the shadow will typically remain stationary for a period of time, which may be used as a basis, at least in part, of "touching". In another embodiment, the shadow will typically enlarge as the pointing device approaches the display and shrinks as the pointing device recedes from the display, where the general time between enlarging and receding may be used as a basis, at least in part, of "touching"…

…where it seems that Apple now has the technology, the art and the desire to achieve touchOver. Their patent in essence describes an artificially drawn 'shadow' of each fingertip as it hovers over the interface. Here's a very quick mockup I made of how this may look, as applied to iPhone.

touchOver mockup from Keith Lang on Vimeo. 

So why does touchOver matter so much?

First, I think this will make the touchscreen user experience even better. Less mis-tapped buttons because you have a greater sense of where the device 'thinks' your finger is. More accurate detection of taps because the device knows about your finger position even before you tap. 

Secondly, and more importantly, it serves as a stepping stone to a multitouch proxy device. 

What do I mean by 'proxy device'? Take the mouse for example. You can see a physical 'mirror' of the mouse on the screen at all times — the cursor —that lets you interact without looking at the physical device.

For a multitouch tablet to replace, or at least augment the mechanical keyboard and mouse, there should be a way to let you keep your eyes on the screen at all times. I know of at least one device that works in this way, the Tactapad by Tactiva (never released commercially).

You can watch a movie of the Tactapad in action here. The tactapad uses a video camera looking down on the users hands to generate an artificial silhouette. A sufficiently advanced multitouch trackpad could generate an even more minimalist/clean version. Note: I'm not saying Apple would mimic the tool workflow as per Tactapad, simply that they'd share the idea of proxy manipulation.

The end result is the same.

A device that brings all the benefits of a dynamic multitouch interface to the desktop computing experience.

Caveats

"But touchscreens are so finicky!"

Lay your palm down on many touchscreens and it will register that incorrectly as a touch event. Other Apple patents describe logic to rule this out. In addition they boast the ability to switch between 'typing' mode (all fingers down) 'pointing mode' (one finger down) and drawing mode (three fingers down, like holding an imaginary pencil.) It may be a solvable problem.

"I'd get tired holding my fingers up all day"

Yes, you wouldn't want to hold your fingers 1cm above the desk all day long. I'm sure there is some solution. See above.

"But what about haptics/force feedback?"

Yes, haptics/force feedback may help you 'feel' your way around an interface without looking. I've been lucky enough to play with some lab-quality (read: $$$) haptic interfaces and agree that it's completely possibly to emulate the feel of pressing a phsyical button or pushing around a lump of clay. But those devices were not cheap, not light nor low-power. I'm looking forward to sophisticated haptics in out everyday devices as much as you, but in some years' time.

"I'd never give up programming on my trusty IBM mechanical clunkity-clunk keyboard."

Maybe writers and programmers will stick to using mechanical keyboards forever. Maybe we'll always keep a mechanical keyboard handy. But it will get harder to resist the appeal of a device where everything is under your fingertips… imagine, for example a Swype-like input interface that dynamically changes it's dictionary depending on what application, or even what part of a line of code, you're currently typing in. A truly context-aware device, done in an subtle and sensible way.

"Why hasn't someone done it before?"

Hehe. They said that to the Wright brothers too. Actually, I'd love to mock this up using something like Keymote for iPhone, but it's very difficult without touchOver-like functionality

And yes, Apple predictions are folly. But from my perspective it's simply a question of 'when' and 'by who'. And from my perspective, the answers are 'soon' and 'Apple'. 

Past. Present. Future.

Here's the bit where I'd love your help: Have you seen any examples of touchscreen interfaces working with touchOver like capacity? How did they work? What other problems do you envision?

Is touchOver essential to a rich desktop multitouch experience? I love the fluidity of interfaces like this multi-touch puppetry (via by Bill Buxton) and think touchOver will be essential to move rich interaction like this to mainstream computing. Let me know. :)

I recently read No Impact Man, a story of one guy who decides to trial a year of living without impact. And, as you may know 

Electricity generation using carbon based fuels is responsible for a large fraction of carbon dioxide (CO2) emissions worldwide; and for 41% of U.S. man-made carbon dioxide emissions. Source: wikipedia

so I was very pleased to note that the host of UI&us, Squarespace, has an extensive Green statement. Overall a very environmentally responsible company, based on this. I'd love to know what you guys think of their claims.

I didn't see a single thing. More on change blindness.

From thread comments in Boing Boing