But the opportunity here is not in simply delivering the crack of the day to our users, it is in developers to easily create user interface experiences that our users can provide feedback on what works and what doesnt. Imagine if I could write up a small Python script that uses Glade and PyGTK that demonstrates a user interface idea. I want to share that idea with interested users and see what other ideas could be inspired from my concept. I also want to make the solicitation of feedback as no-brainer as possible. No subscribing to mailing lists. No account creation. I want it to be as simple as walking up to someone and saying “so, what did you think?“
So speaketh the Baconio in his build up to bigging up gritty by ryan lortie – which if I understand correctly is a neat hack to get code from a scm repo to a random desktop in a couple (one?) of clicks. All very good and worthwhile stuff, but lets just backup a second – gritty is going to be used to get feedback? From Users?
I would expect that user feedback will fall into two camps; either love it or hate it. Actually 3 camps, love it, hate it, or it’s OK but it would be great if you added my crack idea here. Also if what we’re talking about here are UI mockups, then the mythical ‘average user’ is going to have to be pretty inspired to even try a mockup and pretty self aware to provide useful balanced feedback. Jono continues;
There is nothing new in what I am suggesting, at least outside of IT. In the car industry, manufacturers produce concept cars. In the fashion world there are a range of convoluted and frankly ridiculous creations gliding down the catwalks. These innovations are never intended for the mass market, they are intended to show what is possible when the creative minds behind them can create with no restrictions.
We can always stretch analogies too far, but I would point out that as a very ordinary user of cars and clothes, no one has ever asked me for feedback on a concept car or catwalk design or particularly made any effort to get the designs in front of me in the first place, and if they did I would likely just say ‘That’s brilliant!” or ‘Frankly ridiculous’. Now if I was car enthusiast or a fashion follower then I probably be well aware of the latest stuff, and have the vocabulary to give more sophisticated feedback.
Which is my point – this sounds great as a way for us, the enthusiasts, to give each other feedback. Nothing wrong with that – anything that improves the ‘Flow of Ideas’ within a project is good in my book. But to get clues as to whether UI mockup is any good, or where it could be improved, then we need to test it by observing people playing with it, for example like the Meetup user testing described by Clay Shirky
Glusman and van Schaardenburg have also made it possible to take Jacob Nielsen’s user-testing advice — “Test with five users” — and add “…every week.” Obstacles to getting real feedback are now mainly cultural, not technological; any business that isn’t learning from their users doesn’t want to learn from their users.
That’s a pretty sobering thought – I’d always thought that doing distributed user testing, for a distributed developer base, was a pretty hard technical challenge, but maybe mashing up Istanbul and Cheese, to record the desktop session and the users expressions and mix them into a side by side video, is the easy part. Working out how we would use Istaneese, and interpreting and accepting the results would be hard part?
Instaneese… sounds compelling. Is anyone working on that? I’m the package maintainer for istanbul in Fedora I wouldn’t mind helping get that sort of mashup in front of users if it were helpful.
I can imagine having an instaneese daily task service that gnome runs. Users of istaneese sign up with the service submitting a user profile,get the daily notifications, and then run istaneese while performing that task. At the end the task video is sent to a gnome email drop.
But you are right…the hard part will be intepreting the results.
All I can say is try to pre-define the methodology you are going to use before you look at any istaneese videos.
Select a random subset of your available feedback and apply the methodology. Debate the results and adjust the methodology if there are problems. And here’s the key part..do not use the adjusted methodology on the same data subset you already looked at. Take a new random sampling from your larger set of instaneese videos and apply the updated methodlogy.
It’s okay to tweak methodology, its okay having competing methodologies. Just don’t get caught in a process where you adjust how you look at the data concurrently with when you look at the data. You do that and you can get any result you want.
-jef
— Jef Spaleta Dec 17, 06:25 pm #
Hi,
wouldn’t be possible to add a layer to GTK+ which collects all the activity in a window? With this information you could extract which is the regular use of your application, usual long paths to make an action, how people doesn’t use that feature that you though is so awesome… etc.
Probably is easier to add it to pyGTK but the idea applies to all GTK+.
Eclipse recently added something in this way http://www.eclipse.org/epp/usagedata/
— Diego López León Dec 18, 10:41 am #