The EchoUser Experience

This email last week totally threw me:

Now, forgive me for wondering what AT&T was doing in Angola, especially because the African country is one of the most unstable places on the planet.

Oh wait, they meant Angola, Indiana?! My bad, forgive me for thinking globally instead of locally. Except that Angola, Indiana isn’t anywhere near local - in fact, it’s more than 2200 miles away. So remind me again why I got this email?

Maybe that’s why I keep getting dropped calls, AT&T thinks I’m in Angola.

#fail

I got this notice from JetBlue today telling me about new TSA regulations concerning “vetting” of passenger information. Now, I tend to read carefully and try to comprehend things before coming to a conclusion, yet I couldn’t help but feel put off by the message. In the end it seems pretty innocuous (they just want to make sure your JetBlue info on file matches your official government-issued ID info in order to allow online check in), but I still think they failed with their delivery of the message. Stay tuned for my take on the whole thing.

Courtesy of Wikipedia

When I heard that iOS4 was going to enable some kind of multitasking, my first reaction was a resounding “meh”. You see for me, even though I use my iPhone to do business related tasks and the like, I simply didn’t see the need to be able to switch between apps and keep things running. I had no trouble booting in and out of email (since it’s native, it would save my state for later, really useful when writing an email), checking my calendar, or being on a call and looking up Google Maps. For me, and I’ll wager most iPhone users, this was multitasking enough.

But then something strange happened - namely, I started using my iPhone to listen to music, specifically Pandora and Stitcher (disclaimer below). I have a pretty nice sound system at home, and since I recently re-purposed my AirPort for an extra WIFI network, I took to plugging in my iPhone instead. Everything was working wonderfully until “Ding Ding!”. I received a text message.

Now, it’s not the text message alert that was annoying so much that every time I checked the message I’d have to boot out of whatever music app I was using - and interrupt the music. The first few times this happened I dismissed it with a “Tsk tsk. That’s kind of annoying”, and thought nothing of it. But a few weeks in and I was ready to throw my iPhone across the room. Not to mention that I work from home, and the incessant stop-and-start of the music was driving me batty. I eventually stopped using Pandora, Stitcher and a ream of other apps on my iPhone because I couldn’t take all the interruptions (I did switch to Pandora One on my computer, though, so Pandora should be happy).

Enter iOS4. With more open multitasking I could all of a sudden start Stitcher, then check a text message, write a quick email, or check Things without the jarring audio cut-off. What was once a thoroughly painful exercise became, in an instant, seamless. And, you guessed it, my use of Pandora and Stitcher has gone through the roof (same with my roomies).

I’d wager that iOS4 practically saved some music apps, and will rapidly increase user adoption from here on out - because let me tell you: there’s nothing worse than interrupted music, and iOS4 just saved the day.

Thoughts?

Disclaimer: EchoUser has done some work with the fabulous team over at Stitcher, so we’re definitely biased

During a recent project, a client asked us to evaluate the initial user experience for setting up a telephony product. They were not interested in doing a more traditional usability test as they felt that this “out of the box” experience could not be easily captured or replicated by forcing a user to perform a set of pre-defined tasks. They wanted feedback that was more contextual and ethnographic in nature, but they still wanted the quantitative data such as times, errors, assists, and usability ratings that you get from standard usability testing. The more we thought about this, the more we came to realize that there is a gap in methods from where lab testing ends and contextual/ethnographic studies begin.

Bridging the gap - Event Style Testing

Usability testing is often used for getting feedback or benchmarking the usability of key task flows of a product or service. These key tasks are discrete and isolated to make it easier to get more quantitative data, but this comes at the cost of losing what the natural workflow may be since a structure or order is enforced. The natural workflow is something that is researched through more contextual methods. What we wanted for this project was to be able to allow a natural workflow that would allow users to use multiple tools and do things that were not expected by the product team while being able to get metrics such as time, errors, assists, and perceived usability. With this in mind, we created a method that we’ve dubbed internally as Event Style Usability Testing to help bridge this gap between usability testing and contextual/ethnographic studies.

Components of Event Style Usability Testing

There are a couple of basic components to the use of Event Style Usability Testing

• A set of pre-defined “events” are identified that users are asked to perform.
• Users are allowed to use a system (or software) to do the events in any way and any order that they deem appropriate. There is no forced order.
• The test facilitator records the events as they take place creating a workflow record as well as keeping metrics such as time that events start and stop, number of errors and assists for events, and perceived usability of events.
• New events that were not pre-defined or expected are added ad hoc as they are observed during sessions.

To better understand this, what do I mean by an “event”? Events are very similar to tasks that are used in more standard usability testing with a couple of distinctions. Like tasks, events are created that users are asked to perform or experience. For example, an event would be to send an online payment to a person from an online bank account. If this were simply a task in a usability study, then users may be told specifics about who to send the payment to, for how much, and when it should arrive by. For an event, things are turned into a scenario as much as possible.  They are told what they are trying to accomplish, not given specifics.  In the case of the event for sending an online payment, users would choose who to send it to, for how much, from what account, etc.  Allowing them to shape the event using their own data or specific to their own interests allows interesting insights and observations about what users may do in real situations.

Once the events have been pre-defined, an event list is created for participants to use for the session. Prior to beginning the session, users are asked to read the entire list of events that they will be asked to do. They are then instructed to complete the list of events in whatever order they would like and makes the most sense to them. Having them read through the entire list of events this way allows them simulate more closely what they would do out in the real world. It’s assumed in most systems and software that users generally know what they want to do (or else why are they using the system or software in the first place) but may not know specifically how to do it. Event style testing aims to more closely replicated this.

The rest of the process now relies on the careful observation of the testing facilitator. Since there is no specified order that participants are using to complete events, the facilitator needs to monitor what events the participant is attempting or in the process of doing. This includes capturing time stamps of when an event is started, completed, or stopped and not completed, or resumed after being stopped previously. While observing each event, other usability metrics can be captured as well such as any observed errors, assists needed, and other usability scores.

The last piece to the Event Style Process is being flexible to add new events to record and capture in the middle of a session. When trying to replicate real world workflows, it can be nearly impossible to predict and plan for all paths and avenues that users may try to use to work through events. Also, unforeseen or unanticipated events may occur in context that would not occur in a highly structured usability test. The event style testing allows these newly observed events to be captured mid test so that they can be reported on later.

Benefits of Event Style Testing

The event style testing allows for a number of benefits and deliverables that cannot normally be obtained in a usability test. These include:

Workflow Analysis

Since the order that events take place is recorded during the study, a full task flow can be created and analyzed. This is particularly useful for systems or software where very little is known about how users setup or experience them, especially in complex systems where multiple tools or software may be needed. The resulting task flows can show where various tools and software are frequently used, in what order, and for what purpose. In the case of the telephony product we tested, the task flows highlighted at what points different software products were used for configurations and at what points users stopped to verify correct setups and troubleshoot problems.

Event Style Workflow

Quantitative data collection

While the natural workflow analysis can be observed, quantitative data such as errors, assists, and usability ratings can still be collected and compared across users. This creates a more holistic view of the experience being evaluated from a qualitative and quantitative standpoint, the best of both worlds.

Event Style Usability Testing has become a powerful method in our user experience toolbox to bridge the usability test and ethnographic research divide. We are always looking for new takes on old methods and hope others may find this useful as well!

Good beginnings.

I’ve been following WalMart’s sustainability plans for awhile now, and I have to admit I was initially impressed. One of the world’s biggest companies actually making strides in sustainability and forcing its suppliers to come to heel?  The ripples of the Sustainable Product Index, I was sure, would spread far.

That was until I read more about their plan. It’s pretty simple really, and involves three steps:

1. Ask suppliers 15 simple questions about their supply chain and production practices.

2. Create a Sustainability consortium of universities, companies, NGOs and government to “develop a global database of information on the lifecycle of products…from raw materials to disposal.”

Ok, so far so good.  And finally:

3. To (and here I quote) “translate the information stored in the database into a simple tool that informs consumers about the sustainability of products.”

Right. Because consumers are inherently dumb, and the complicated supply chain information needs to be “translated” before we get to have access to it.

While I’m sure that the information is complicated, and that most people buying Kraft mac ‘n cheese, jelly donuts and frozen dinners don’t give a flying crap about supply chains, leaving the end user out of the equation until the very end is a huge mistake. WalMart may have made strides on its way to taking over American industry, but it committed the cardinal sin of standards creation: only including the end user, you know, the actual people who will use the standard on a daily basis, at the end of the process.

It’s not about the labeling, stupid.

I imagine WalMart’s thinking has gone something like this:

“We have about a million moving parts to coordinate with the Index, so we’ll tackle our suppliers first because they’re more complicated. Then we’ll make things seem more legitimate with a warm-and-fuzzy Consortium (I actually like this move, FYI). And finally, we’ll slap some pretty labeling system on everything so Gina from Shattunket, Georgia can choose the toilet paper that speaks to her conscience just as well as it does her bottom.”

But this is all backwards. Creating an index to be used by regular people isn’t just about making it legible or usable (although that’s definitely important and should be thought about up front anyway), it’s about making it relevant.

And the only way to do that is to include the end user - you, me, everyone - right smack at the beginning the whole process. Word.

Fatal error message in a terminal window

While I understand that command line interface programmers don’t feel like they need to adhere to the same rules as UI designers, I think it would be best if they reconsidered.

Under no circumstances is it necessary for an error message to tell me something was “fatal”. It evokes feelings of panic and makes me feel as though I’ve done something irreversible. In the example above I simply called “git pull” (grab the latest code from the repository) on the wrong folder.  Because that folder wasn’t tracked by git, it doesn’t do anything… it just exits.

Consider using another response, such as “Not a git repository, nothing has been pulled. Are you in the right directory?”. I don’t need to hear about what process was killed or how it died, I just want to know what happened and what I should do about it.

Thanks

Happy (belated) New Year, one and all - I do hope 2010 turns out to be a smashing success for everyone.

I figured I’d start things off with a video made by my friend and colleague Etan (@Zaqintosh).  It’s a concept for what an iPhone app designed to measure experiences (any experiences, from surfing a website to hopping on BART) could look like, and even though it’s crude, I think it gets the potential across.

Can you imagine if we were able to rate experiences on the fly, all day every day? I, for one, would love to track which ones get me up, and which bring me down.  The trick, I think, will be to get people used to rating their life experiences; Yelp’s easy because it’s tied to businesses, but it might not be obvious to people that rating a bad jog, conversation, taxi ride (due to traffic, not the cabbie) or plane ride is just as valuable as rating your local Starbucks.

In any case, without further ado, voila:

iPhone experience measurement concept - EchoUser

Dropbox and the Human Touch

A few months back I got an email from the crew at Dropbox reminding me to use their service.  If you haven’t tried it yet, Dropbox is a neat file syncing service that lets you back stuff up online as well.

Anyway, usually I hate reminders to use a service - I find them intrusive, and the repeat offenders get put on my spam list even if the service/product is actually pretty cool.

But Dropbox’s email actually made me smile:

It’s got three key attributes that make it much less annoying than other reminder emails:

1. It’s visual.

I get it, you want me to use your service. But please don’t show me a long bullet-point list of why you think you rock.  Dropbox gets this, and replaces long text with an image instead.

2. It’s funny.

The picture did actually make me chuckle.  It won’t win any best joke awards on BET and Carlin certainly wouldn’t be impressed, but in my book any laughter is better than none (and certainly better than a groan as I hit Spam). Further, it shows they have balls, which is always a good thing.

3. It’s human.

This is the most important part: the hand drawn image really lets me connect with the company.  Dropbox isn’t just a bunch of faceless programmers shoving their product down my throat - somewhere somebody actually sat down to draw this picture, and that’s a powerful thing.  So powerful that I actually pictured him or her doodling away with their tongue stuck out. Pretty cool.

Even though in the end the email didn’t get me to keep using Dropbox (I honestly don’t have a need for it), it did get me to take the time to write about it, which in the end is almost as good.

When San Francisco banned plastic bags 2 years ago, my whoops of joy could be heard across the Golden Gate in Marin.  Having grown up in the developing world I grew accustomed - sadly - to seeing plastic bags everywhere they shouldn’t be: the streets, clogging drains, polluting rivers, the ocean, even stuck on phone lines. Everywhere except in garbage cans. You can be sure that I for one wasn’t going to miss their absence at all.

So imagine my surprise when last week my local Delano’s bagger put my groceries in one of these:

That’s funny, I could have sworn that plastic bags are illegal in San Francisco. So unless I’m missing something, this shouldn’t be allowed.

On second look, the makers of the bag have done their utmost to convince everyone that this is anything but a plastic bag with the liberal use of clever marketing copy.  Phrases like “no trees were harmed in the making of this bag”, and a cute little “nutrition” box highlighting exactly how it hasn’t hurt trees are nice tries - but belie the fact that eventually this bag will no doubt end up in a dump, where it most certainly will hurt a tree.

I get what the folks behind the bag are getting at: we have so many bags in production already that it certainly makes sense to train people to reuse them, thereby preventing more bags from ending up in landfills. Indeed, these particular bags do seem more durable than their crappy white plastic counterparts, so I could definitely imagine using them for more than one grocery run - but let’s face it: I can count on one hand the number of people I know who bring eco bags to the store every time they go.  Heck, I have 3 such bags in the trunk of my car, and I still manage to forget them each and every time.

As luck would have it, these bags have an answer for our laziness, too - a message nudging us to take them to “participating stores” for recycling. Good idea in principle, until my roommate tried it: turns out our local Delano’s isn’t one of those stores.

Go figure.

A last minute glitch and I could not get international roaming activated on my cell phone. This was the first time I was going to be without a convenient access to a phone while on an international trip. I wasn’t happy…

First stop Hong Kong airport, and I was pleasantly surprised to find the complete airport WiFi enabled. I was able to use Skype on my iPhone to talk back home, check my emails and send updates to my Facebook and Twitter accounts. So far, so good…

My entire stay in San Francisco, I must admit I never missed not having cellular connection…it seemed like the entire city was WiFi enabled even though it was only in parts. Any restaurant, shop, pub I found myself in had WiFi so staying connected was a breeze. In all my years in the US and my trips there over the last 3 years, I never saw WiFi so entrenched in the ecosystem until now.

It goes without saying that WiFi enabled Internet access on mobile phones is the way of the future (Virgin America has even started offering WiFi on their domestic flights).

Suddenly, with broadband data access, 3G seemed a bit too slow, cumbersome and a hindrance to the overall user experience.

In almost all aspects of consumer consumption behaviors in India, there has been a leapfrog like trend where the Indian consumer has bypassed some of the technical/behavioral aspects of adoption due to the late entry of some technology or product. Jumping from having no phones to the cell phone, getting introduced to the Internet directly on the mobile are some classic examples.

Apart from the government run service providers, 3G in India is still some distance away. Can India leapfrog in this aspect of adoption as well? Can we jump directly from Edge to WiFi? One can argue that carriers like TTSL, Reliance and Airtel are well positioned with their already existing home based broadband service to provide blanket (to some extent) WiFi coverage in metro cities. This could earn them enhanced revenue from their broadband service and also help bypass 3G. They could focus on mid/high tier WiFi enabled mobile phones, and generate greater ARPU through a much enhanced user experience of their VAS services.

It will be worth investigating a two-tier strategy - A limited 3G rollout to cover the rural geography in India for enhanced voice/data services and WiFi rollout in metro cities for data services.