Image from IWM Family in Wartime Gallery
We all know that developing ways to define, evaluate and ultimately measure the success of digital activities is an issue faced by all parts of the cultural sector. It’s a difficult task, particularly as definitions are still fluid, and what is measured is ultimately down to the requirements of funding rather than visitor engagement.
So as part of the Social Interpretation Project we have done a lot of thinking about how to evaluate success on all three digital outputs. For online success we have taken a lot of ques from Culture 24 lets Get Real Report.
But for Digital interactives on the gallery floor and QR codes we’ve had to be a bit more inventive. By inventive, I mean ask other people what they have done.
We’re going to try and tackle QR code success criteria first. This is where the twitters came in. I asked
“ Does anyone have any benchmarks for success on QR code usage that they would be willing to share with us for #socialInterp?”
And I had a great response.
First up looking at some of the scan rates of QR code use in museums.
QRpedia use at Fundació Miró
Here QR codes were placed alongside 18 of the most prominent artworks of the exhibition. These codes linked to Wikipedia articles. The sample is from October 1, 2011 – March 30, 2012 (data taken from here)
The scan rate is pretty impressive!
| month | scans |
| OCT | 2124 |
| NOV | 2293 |
| DEC | 1990 |
| JAN | 1966 |
| FEB | 2014 |
| MAR | 1994 |
| Total scans | 12381 |
Next is
QRpedia use at The Children’s Museum of Indianapolis
The Children’s Museum has four QR codes against four specific labels. Again the codes linked to Wikipedia articles. The sample data is from June – November 2011. (data taken from here)
| QR code label | Scans | Average scans/day |
| Broad Ripple Park Carousel | 1300 | 8.5 |
| Captain Kidd’s Cannon | 797 | 8 |
| Reuben Wells (locomotive) | 378 | 3 |
| The Children’s Museum of Indianapolis(The Museum itself!) | 69 | 1 |
| Total scans | 2,544 | |
These QRpedia examples, with really impressive scan numbers give QR code use in Museums hope. However when not linked with Wikipedia, QR code usage leaves a lot to be desired.
Brooklyn Museum have done various trails with QR codes, and admitted to multiple fails, including a five fold drops in traffic.
So should we all be linking QR codes to Wikimedia? Shelley asks “Could Wikipedia get visitors over QR code hump of technical hurdles and poor user experience?”
But then comes another benchmark case study
QR code implementation in the Love Lace Exhibition, Powerhouse Museum
Data below is from a 4 weeks period. (data taken from here)
What’s nice here is that the Powerhouse built a bespoke app, with a QR code scanner built in, just as we are doing with IWM. So it was really interesting to see the number of app downloads.
| iOS | Androids | |
| Downloads | 572 | 165 |
| Sessions | 3,126 | 502 |
When it comes to actual scans – has had 844 scans including 45 failed scans and 17 non-exhibition codes. Despite many objects not being scanned at all, 844 scans in a 4 week period is massive!
During the course of the Social Interp project we’ve had a few mental fisty cuffs about the use of QR codes. In an post on the Social Interp Blog I asked several questions: Are the useful? Are they just a transient technology? Are they even a technology? How do they help visitor experience? Where do they lead the visitor once they have scanned it? And ultimately who actually scans QR codes? Is it just us?
We haven’t really reached any conclusion on this, but in reality whether you love them or hate them (yes QR codes are the marmite of the digital tech museum world) the fact remains that QR codes are an incredibly cheap, easy and compelling way to provide information, get visitors interacting, and illicit responses from them.
Now I have a couple more questions to add to the QR conundrum list:
- Should all QR code content take the QRpedia approach, and utilising an already existing, well used platform?
- Will a bespoke Museum specific application solve the barriers to access when using QR codes?
Any answers or other benchmarks for QR code usage in museums would be gratefully received!
So this hasn’t really given us some clear cut success measurements, other than number of scans and number of app downloads. You can break that down into time spent on page, shares etc. But if we can get anywhere near the QRpedia case studies, I’ll be more than happy.
Claire Bailey-Ross 
Luke at Powerhouse has much more up to date data on the Love Lace experience which he presented at MW. (Hopefully it will be online soon!)
One key thing that is missing from the data above is the number of visitors to the exhibitions in question. Much like audio tour take up is measured as a percentage of total visitors (and 5-10% take up is good), we need to figure out what percentage of visitors are using QRs – not just how many QRs are scanned.
I look forward to Luke’s paper, I missed the talk in San Diego unfortunately!
Of course all scans need to be in context, but I didn’t have the data for that. Measurement is only as good as the data you have. At IWM we are also facing the issue of how to you count visitors to a free, un-ticketed, un-maned exhibition in order to get those visitor numbers for a comparison. Success criteria work in theory but in practice can be very tricky.
In The 1968 Exhibit, which we opened in October 2011 at the Minnesota History Center, we used QR codes to connect gallery visitors to an interactive timeline with text, graphics and video content to supplement what was in the exhibit gallery. QR codes were located at each one of the sections tracing the year month by month. You can look at the identical non-mobile web version here:
http://www.the1968exhibit.org/1968-timeline/timeline
We did a summative evaluation which revealed these interesting patterns:
11% of visitors reported using QR codes in the exhibit. Over half of the visitors who used the QR codes thought the content was “informative or interesting” (56%). Young adults, though most likely to use QR codes (28% usage rates between the ages of 18-39), are also LEAST likely to report use of QR codes as “informative or interesting,” but by wide margins are more interested in QR codes in future exhibits (80% in 18-29 group, 75% in 30-39 group).
The main takeaways here are that younger people are definitely more receptive to using QR codes in the gallery, basically the the younger, the more receptive. The quality of content/user experience, however, should not be underestimated since users are inclined to be critical, especially those most inclined to use QR codes–young people. Also, with uptake rates as low as 11%, the resource commitment to the development is a very real question going forward. Is it worth the investment?
I agree with Seb’s comment about percentage of visitors using QR codes. If a reasonable % of the audience use QR codes then fine and they are providing good value for money, but in some (many?) instances I doubt that’s the case.
In terms of your free exhibition – if it’s not in a space where you can use an automated people counter, then sample on several different days and calculate the sample as a % of overall visitors on each day, average out (may need different averages for weekdays, weekends, school holidays) then multiply the average figure(s) by overall visitors for the duration of the exhibition.
Beyond that there are harder-to-measure figures like how much time visitors spend engaging with the content (possibly there might be some way to investigate this through Analytics but not something I’ve ever looked into) and more qualitative data about what the content accessed through the QR code added to their experience, which could then be compared to other more traditional enhancements to exhibitions such as a reading area (takeup probably also quite low, though even harder to measure).
hi Clairey, I was already intrigued by your talk at MuseumNext on QRcodes, since we are often dealing with the same QR issues at the MAS (Museum aan de Stroom) in Antwerp. We use QRcodes extensively, initially to provide our non-Dutch speakers with translations of all exhibitions texts: from general thematic texts to object labels.
We are now working on a website, which will feature the same textual information but also pictures of the objects (both the website as the QRcodes fall back on the same database). This website will also give the possibility to link an object to your Facebook page or Twitter profile. In addition, you can also bookmark particular objects and make your own tour, private or public. When you come to the MAS, you can lend out an iPod, sign in and take up the tour you designed at home. Or, approaching the topic of QRcodes, you can do it the other way around: you can scan the QRcodes in the museum, read some more information on the objects or relating stories in others parts of the museum or the city, link the objects to your Facebook or Twitter on the spot, or design your own tour. On an iPod of the museum, or with the app you can download on your own mobile phone. When returning home, you’ll find an email waiting for you, which points you to the same website where your bookmarks and/or tours are saved.
We’re hoping to start all this up from the end of June, so we will also get some stats on the use of QRcodes. I’ll keep you posted. (and if I should forget, please remind me. Or come to Antwerp!).
Pingback: What does success look like for museum QR code usage? « QR Code Fun
Pingback: Retour d’expérience sur l’utilisation des QRcodes dans les musées « Mon Canet en Roussillon
Pingback: BUZZEUM » Réflexions sur les QR code dans les musées
Reblogged this on Things I grab, motley collection .
Pingback: business and finance apps for iphone and ipad, best business apps, best finance apps » Scan
Pingback: L’echec des QR-Codes « Des Bibliothèques 2.0
Our QR-codes link to a fragment of our audiotour on our website instead of wikipedia-texts. It is an alternative for listening to the audiotour on our ipods.