Hannah Clipson – Audience Research and Advocacy, January 2012
atmosphere … exploring climate science
Rizk – Online Game
Summative Evaluation Report
HANNAH CLIPSON
AUDIENCE RESEARCH AND ADVOCACY GROUP
LEARNING AND AUDIENCE DEVELOPMENT UNIT
SCIENCE MUSEUM
Key Findings
· There have been over 2 million visits to Rizk during its first year.
· Seeding the game on external sites is a successful way of reaching a wider and greater audience.
· Dwell times for the game are high with averages of 14.23 minutes for the 20 levels on the Science Museum site and 9.05 minutes for the 5 levels on the seeded sites.
· To date, Rizk has been ripped onto 900 sites.
· It was apparent from the testing that there were two approaches to playing Rizk, the strategic approach and the spontaneous approach, largely influenced by the individual’s experience and confidence in gaming.
· Players found the game engaging – the intriguing, visually appealing design hooked them in and the sophisticated concept and the challenges ensured attention was held.
· Providing a set of optional instructions was a good way of catering for all players with different levels of gaming confidence and experience.
· The tutorial nature of level one did not appear to promote correct gameplay in level two.
· However, developing a game where players learn how to play through using a trial and error approach appeared to be successful at enabling users, regardless of their gaming confidence and experience, to play the game correctly.
· Although this approach did not increase the confidence of the less ‘game able’ – despite playing correctly they were unsure of their actions and questioned their understanding of the game.
· Players knew the game was about dealing with risk and they were able to translate this understanding into action – choices made by the players showed they were weighing up the probability and impact of risk.
· Players could see how the game was a representation of climate science issues.
· Though most could not apply the skills and knowledge they had used during the game to understand real life issues of climate change and risk management.
Recommendations
For future projects:
· Seed whole games onto other websites.
o Seeding games is a good way of accessing the target audience. The 5 levels of Rizk which were seeded on external sites received more hits than the Museum website version but did not promote traffic to the Science Museum site suggesting the whole game should be seeded externally.
· Clearly define whether the target audience for a web game is gamers or non-gamers.
o Gamers and non-gamers have different needs, wants and expectations so it would be difficult to cater for both in one game. The success of Rizk with the target audience shows that the Science Museum can develop successful games for a more experienced gaming audience.
· Provide more of a challenge in the level one tutorial stage – demand more attention and action from the player.
o Making the level one tutorial stage more challenging can cater for both the confident gamer and the less confident gamer.
o Demanding more attention and action from the less able player will help them to absorb the instructions if they need to apply them to overcome an obstacle.
o Demanding more attention and action form the confident player will help to combat feelings of frustration at the pace and ease of play.
· Continue to provide optional instructions so players who want these can access them but those that do not are not forced to.
Introduction
A background to the atmosphere: exploring climate science Gallery.
The atmosphere: exploring climate science gallery opened to the public on 4th December 2010. It was developed for the Science Museum as part of the upgrade of the Wellcome Wing – the Museum’s contemporary science space.
What is Rizk?
As part of the web presence for atmosphere, the online game Rizk was created in collaboration with PlayerThree.
What is the aim of the game?
Rizk is a tower defence game about risk management. It involves the user growing a plant through collecting energy. Collecting energy results in spores being released which attack the plant. The user must manage the risk to the plant through deciding where to collect energy and buying and positioning defenders to protect the plant.
How is the game linked to climate change?
The gameplay described above has been designed to represent an analogy of climate change. For example, decision makers need to balance the risks posed by climate change with the costs of managing its impacts or reducing its probability (for example by spending money on reducing Greenhouse Gas emissions or building defences to cope with impacts).[1]
How do you play it?
Players are presented with the home page (pictured below) and have the option to go straight into gameplay, to read the instructions or to view high scores.
Figure 1: Rizk Homepage
· Play Game:
o When the player opts to play game, the screen shown below will appear. On the Science Museum website, there are twenty levels of increasing difficulty and each level will only become unlocked and available to play once the preceding level is completed.
o On the seeded sites, only 5 levels are available to play but the same principle applies to progress as the Science Museum website version – each level must be completed to advance.
Figure 2: Level Select Page
· Learning to Play the Game:
There are 4 ways in which players can learn to play the game.
Applicable to all players:
1. Trial and error approach during Level 2 onwards.
2. Level 1 Tutorial – instructions are built into the gameplay to help players make the correct moves (see image below).
Applicable to self-selecting players:
3. Instructions accessed before and/or during gameplay
4. Instructions tutorial accessed before and/or during gameplay
Figure 3: Level 1 Tutorial Stage
· Instructions:
o Players have the choice to read the instructions before and /or during gameplay.
o The instructions guide the player through the aim of the game and the various components which need to be understood for successful gameplay.
Figure 4: Instructions Page
o In addition, players can also view a tutorial which enhances and further explains the basic instructions (see below).
Figure 5: Tutorial available to view on instructions page.
· Results
o On completing a level players are presented with a score page showing a breakdown of their results. In addition, players also receive a ‘tip’ which links to the climate change and risk management message (see image below, right hand box). This is one of the only places throughout the game when players are explicitly exposed to the game content messages.
Figure 6: Score Page
· High Scores:
o When the game was developed, players were also able to share their scores on Facebook (see below).[2]
Figure 7: Connect with Facebook option.
Who is the game for?
Rizk is targeted at independent adults 17-35 with low-medium levels of engagement with climate change but medium-high engagement with computer games.
Where can Rizk be found?
Players can access Rizk in 3 ways, through:
Ø The dedicated Science Museum site.
Ø The 20 sites seeded by the Science Museum. For example, Miniclip and E4 Games.
Ø Sites it has been unofficially published on through other people “ripping” the content.
Evaluation
Aim
· How effective is the web game “Rizk” in engaging its target audience in the subject of risk managing climate change?
Objectives
· What is the profile of people playing the game?
· What is their behaviour on the site?
· How do participants respond to the game?
· Are the learning outcomes / content messages for the game met?
· What are the barriers to motivation, usability and comprehension?
1
Hannah Clipson – Audience Research and Advocacy, January 2012
Methodology
· Analysis of Google Analytics data to identify number of users, distribution and dwell times.
· Accompanied surfs and interviews with the target audience (see table below) to test the engagement, usability and comprehension of the game play.
Target Audience / Audience Tested With17-35 year olds / Yes
Low – moderate engagement with climate change.
Are bored or disconnected with the topic.
and/or
Are unconvinced climate change is happening. / · Most are concerned about the issue of climate change.
· Only 2 were bored with the subject of climate change.
· Overwhelming majority agreed that climate change is happening.
· 8 agreed that they were convinced that humans were the cause of climate change. 1 neither agreed nor disagreed and 1 disagreed.
Visitors unsure about the science behind climate change. / · Most felt confident they understood both the causes and impacts of climate change.
People who play online games, though not daily.
Visitors who are looking for engaging online experiences. / Yes
Structure of the Accompanied Surf:
· Participants were shown the home page and asked to use the game as they would at home.
· Participants were encouraged to play as many levels as they think they would if using for personal use and also to stop playing when disengaged.
· During gameplay the players were asked to explain certain actions to the Researcher to establish their understanding of the game.
· The players were interviewed after they decided to end their game.
Main Findings
Player Numbers and Profile (Google Analytics Data)Through embedding hooks for Google Analytics into the game on the dedicated Science Museum site and the seeded sites[3], data could be gathered showing the number of users, distribution and dwell times.
Analysis of the data collected via Google Analytics during the first year of Rizk’s web presence (Dec 4th 2010 to Dec 3rd 2011) is presented here:
· There have been over 2 million visits[4] to Rizk during its first year: 2,340,818.
Figure 8: Pattern of Visits to Rizk on Science Museum Site
· The pattern of visits to the site is broadly similar between the Science Museum and seeded sites.
· Number of visits peaked in February 2011 seemingly in conjunction with the Science Museum’s PR campaign. The highest number of visits in one day is recorded as 87,176 on 22nd February 2011 and coincides with Rizk being rated as a “hot game” on www.kongregate.com.
· Seeding the game on external sites is a successful way of reaching a greater and wider audience. Although it does not appear to encourage traffic to the Science Museum website.
o There were more visits and more return visits to the game on seeded sites than on the dedicated Science Museum site (see table over page).
Figure 9: Visitor Numbers
Seeded / Science MuseumNo. of Visits / 2,010,140 / 330,678
% of Total Visits to Rizk / 85.87% / 14.13%
% of Return Visitors / 61.71% / 43.34%
o Seeding the game also increased the game’s geographic distribution. When comparing the top 10 countries playing Rizk on the Science Museum site to the seeded sites, although most countries appear on both lists, there are some differences. Poland and Japan appear on the list of top 10 countries playing Risk on the Science Museum site but not the seeded sites, while the Czech Republic and Brazil appear on the list for seeded sites but not the Science Museum site (see Figures 8 and 9).
Figure 10: Top 10 Countries visiting Rizk on the Science Museum website
Figure 11: Top 10 Countries visiting Rizk on Seeded Sites
· As the visitor numbers for the seeded sites is so much larger than for the Museum site, it suggests seeding 5 levels of the game did not promote traffic to the game on the Science Museum site.
· Although there were more visits to the game on seeded sites, visitors looked at fewer pages and spent less time on the game than on the Science Museum website. This is likely to be because only 5 out of 20 levels were seeded on external sites.
Figure 12: Dwell Times
Seeded / Science MuseumAverage No. of Page Views / 5.91 / 6.83
Average Dwell Time / 9.05 / 14.23
Ø This suggests that to capitalise on the greater audience reached through seeded sites, whole games should be seeded.
Player TypesIt was apparent from the testing that there were two approaches to playing Rizk, largely influenced by the individual’s experience and confidence in gaming.
1. Strategic Approach
2. Spontaneous Approach
Figure 13: A Table to Show Typical Behaviours according to Game Approach (Observations)
Approach / Number of Participants taking this Approach / Typical Behaviours(based on observations made during accompanied surfs)
Strategic / 4 / Happy to consider choices, make a decision and patiently watch each level unfold.
Spontaneous / 6 / Happy to make quick, spontaneous decisions then deal with the consequences later, impatient if feel game is unfolding too slowly.
The model below highlights the relationship between a player’s confidence and experience in online gaming with the approach they take to playing Rizk. For example:
1. A person low in gaming confidence and experience will take fewer risks and approach the game more strategically; hoping to avoid any negative effects they feel they are not confident enough to overcome.
2. The person with a medium amount of gaming confidence and experience will play more spontaneously as they feel comfortable with dealing with the potential adverse consequences later.
3. The strategic approach will then be reverted to by those gamers with the highest amount of confidence and experience. They want to avoid having to deal with negative consequences, through making spontaneous unconsidered decisions, by establishing a strategy and putting measures in place to prevent the adverse obstacles from occurring.