Operation: Sleeper Cell Post-Mortem

Preparation and Design

The original game design brief, written in late 2007, was required as part of the Let's Change the Game (LCTG) competition. In the first round, only a very short game description was required, while in the second round, a much more detailed brief was needed. The choice of Cancer Research UK as the charity benefitting from LCTG was made earlier in 2007, and was partly based on their size, goals, and reach.

The original game design brief was good, if overambitious. Although there's nothing wrong with being ambitious as long as you realise that fact at the time, and then adjust the game accordingly, we should have done this earlier than we did.
The brief lacked detail in the game design, story and planning sections. Resolving these details earlier, with a smaller team, would have saved valuable time later in development.
We were not asked to address project management in the original brief, but should have done so at an early stage after winning the competition. We should also have done this not just in terms of a plan with milestones, due dates and deliverables, but also how the team would communicate, the roles required, responsibilities, etc.
Cancer Research UK was not involved enough, yet at the same time, too involved. A large organisation with a lot of institutional momentum, Cancer Research UK raises hundreds of millions a year, and a new online project just wasn't important enough for them to take ownership (either through money or resource). On the other hand, since the game was officially 'in aid of' the charity, it had to abide by a large range of rules that were difficult to interpret for this particular game.

Fundraising

Raising funds for Cancer Research UK was the top priority for Operation: Sleeper Cell, as set by LCTG and Cancer Research UK itself (as opposed to, say, raising awareness). To put Operation: Sleeper Cell in context, it was the first ever game to raise funds as part of its core gameplay.

We believe that the fundraising model was fundamentally sound, and would have been performed well if player numbers scaled up.
Ultimately, the game did not attract the number of players expected, limiting the fundraising total. A large part of the fundraising relied on advertising, which in term was predicated on large player numbers. We needed a dedicated sponsorship/advertising manager to go out and approach companies; in the absence of this person, we didn't sell the idea very well on the website.
However, the amount raised per player was high (although that came with a new fear of 'bleeding people dry').
Some players wanted an official sponsorship form to collect donations from their friends and colleagues. Due to Cancer Research UK's rules, we could not provide this.

Project Management

Many of the project management difficulties faced by Operation: Sleeper Cell are similar to those seen by the open source community, where teams are also usually volunteers and widely distributed. Even after many years, the number of successful open source projects unfortunately remains a tiny fraction of the total that are begun.

General

Considering that the entire team were volunteers working over Skype and Basecamp, with most people not even knowing each other, things went very well.
However, there were still significant problems with the project management that might have been alleviated with a better plan at the beginning of development (see above).
We didn't track responsibilities and decisions as well as we could have done. Lots of ideas were never followed through (or killed early enough).
Decision making was too slow - involving everyone or aiming for complete consensus on every decision was impractical. We needed a smaller group of decision-makers.
Throughout development, a lot of time was spent recruiting new volunteers and getting them up to speed. This time may have been better spent elsewhere.

Communication

We used Basecamp and a group Skype channel extensively for communication and held weekly meetings of the team leaders over Skype for which leaders wrote brief reports in advance. However, despite this, important information often did not reach the correct people and there were misunderstanding. We should have thought more carefully about how to structure communications and the weekly meetings.
The development involved a lot of dependencies, which slowed things down given that people were working asychronously.
The few face to face meetings and development camps the team had were highly productive, partly because everyone was working at the same time, and partly due to the artificial deadline of the end of the meeting. Everyone agreed that more meetings would have helped massively.

Timeline

The long development period - nine months - was an issue for many in the team. People felt that they had spent a lot of time working, with little to show, and that things were dragging. This affected morale. While it may not have been possible to shorten the development period, building in sprints, betas, or website demos may have helped.
Several team members came and went during development, and many others changed their role. During these transitions, information was often lost, and decisions forgotten or postponed.
The change in team leaders - twice - caused disruption.
Deadlines were occasionally set, and sometimes missed, with the people involved feeling demoralised. Ideally, we needed a better way of setting artificial deadlines, such as public demo or development camps, to increase productivity.

Marketing

During the LCTG competition, and in the early stages of development, it was believed that Cancer Research UK would be able to offer significant publicity and marketing resources to help promote the game. Unfortunately these resources did not materialise, causing unforeseen problems.

More marketing and PR resources from Cancer Research UK would have helped. We made incorrect assumptions about the level of help they would give us and should have clarified exactly what they could do earlier.
We should have given Cancer Research UK more precise dates so that they could prepare. A prioritised list of what we needed from them would likely have helped both parties.
Marketing and PR is an easy thing to deprioritise during development, since it's not 'needed' for launch in the same way that the website is - but it's absolutely essential for building the player base.
We needed a dedicated PR guru. More marketing and PR would have increased the funnel of players.

Mission Design

Any kind of puzzle design is extremely challenging. Many of the volunteer mission designers for Operation: Sleeper Cell were also players of Perplex City, a hybrid puzzle/alternate reality game, which had a range of very easy to very difficult puzzles.

The game needed more easy missions. Most were too hard, having been created by people who loved (difficult) puzzles. As a result, we lost a lot of casual gamers.
The 'Mission Czar' should have been involved from the very beginning of development, to set the tone, difficulty, direction and requirements of missions. This person should ideally have been one of the original team members as this was too vital and time-consuming role to expect a newcomer to take on.
Having sample missions to show to designers would have helped.
At the same time, we should have created more missions for people that didn't like puzzles or 'user-generated content' missions.
There needed to be a better process for graphics and missions team to work together; there were frequent miscommunications and misunderstandings.
Until Alex fell ill, mission roll-out during the live phase was fine - meaning that we needed a better contingency for people falling ill.
It would have been good to have Flash game missions, as originally planned.
Proof-reading and testing could have been better.
The 'manually marked' missions caused a minor fuss among players; any missions involving human marking need to be handled very carefully for consistency and timeliness of marking.

Game Design

Story

The story and gameplay (missions/metapuzzles) should have been much more integrated. The missions and story were almost like two seperate entities; they were also essentially written in isolation from each other.
The writers wanted to tie the missions and story more closely together, but not enough time was left. We should have thought about this earlier.
Generally speaking, we liked the story and characters, and felt there was enough content.
However, there was no real incentive or reward for players to follow story; people could just focus on missions.
It was fairly easy to find and follow the story, which took place over just a few different sites.
From a user experience viewpoint, it wasn't obvious from the game's front page that there even *was* a story; it needed to be displayed much more prominently.

Game Mechanics

We aren't sure whether awarding points for missions was a good idea or not. It helped spur on the competitive players, particularly through the leaderboard, but it made the game less co-operative.
The use of teams was good for fundraising, but with such a small player community, teams only put up more barriers to co-operation.
Some players may have felt that they needed to join a team in order to play; we should have better explained that this wasn't the case, and made it easier for people to play on their own (and as casual gamers).
The big 'metapuzzle' had an appropriate difficulty, but it was not integrated into story as well as it could have been and suffered from some of the missions being released later than intended.

General

An extended face to face meeting at the beginning of development involving the original core team, to pin down the overall game design, mission design, story and structure, would have saved a lot of time and confusion later on.
We did some usability testing on the website, with was very useful, but doing even more would have helped hugely since there were still issues after launch.
Having a side theme of tea and biscuits work quite nicely; players liked this and it provided a theme for the missions.
It would have been good to put in more cancer-related information (e.g. videos) into game (but not as part of story), as originally planned.

Live Events and Website

We had plans for several live events, which didn't end up happening. Why?
- We expected far more players than we actually had.
- We expected a higher percentage of players to sign up to events.
- There was a lot of red tape involved from Cancer Research UK concerning safety and fundraising, e.g. we weren't allowed to collect money from players during live events.
- In general, live events involve a lot of effort; people on the ground, and money up-front.
Live events require a disproportionate amount of effort considering their returns. We needed to better decide what we were doing them for - PR, fundraising, or as part of the game?
We needed to think more imaginatively about how to hold the live events (e.g. tying into the Hide and Seek events)
Many of the planned events were harder to organise than first thought. For example, distributing items to Cancer Research UK shops, and giving shopkeepers passwords, etc, were essentially impossible.
When the live events didn't happen, this was a big blow to morale in team; expectations had been set too high.
One team member pointed out that we were right to plan events that we did - if we had the player numbers we expected, they would have been great headline grabbers.
The online live components (blogs, twitter) worked fine.

Tech

Technology and website development is usually not a strong point in alternate reality games; in fact, most ARGs prefer to have as little technology as possible. The smooth development and running of the Operation: Sleeper Cell is an achievement that should not be underestimated.

The lack of comment here reflects how smoothly the website went!
It would have been helpful for PR and marketing purposes, as well as game design to have seen a beta website much earlier.
Development really started to move when the deadline approached!

Overall

Operation: Sleeper Cell was an amazing success for a group of volunteers scattered around the country - most similar projects never even launch at all. We were all impressed by amount of effort people put into it, and to have raised any money at all with very little input other than sweat is impressive.
Cancer Research UK never really got involved enough. This was due to a combination of the size of the organisation, its conservative nature, and a lack of understanding of game's concept. We didn't have any real institutional buy-in, and in retrospect, we should have worked with Innovation Team rather than Community Fundraising (meaning that we were treated in same way as fun-run organisers). Working with smaller charity may have been a better fit, in terms of the impact our fundraising would have had on their operations, and their flexibility.
This was a valuable learning experience for everyone involved. We were all thrown in at deep end, which was highly stressful, but the best way to learn.
One comment: "If I hadn't been making Operation: Sleeper Cell, I would just have had better gear in World of Warcraft."
Ultimately, the game was a good demonstration for other groups interested in 'serious' ARGs for charity or education, particularly those run with volunteer effort (in this case, many of whom didn't even know each other). Someone had to do it first, and it was us.

Player Comments

"My main disappointment (and I am sure for you as well) was that there weren't more people playing. I had hoped that this would reach out to a new set of people who hadn't played an ARG before, but in the end the keenest players were familiar faces and there were just too few of them."

"The quality of the puzzles was excellent. My favourite was Bookshelf, which must have taken an age to develop. If I had one very mild criticism, there were possibly too many puzzles of the type 'put some apparently unconnected things into google and see if you can find the answer'. I appreciate that it is difficult to create puzzles that aren't googlable, though."

"The non-puzzle activities were a good idea and fun - if only they hadn't hit a busy time with me, I would have enjoyed doing the later ones. I think you underestimated how competitive people can get on these things!"

"The unlocking method was interesting as a fund-raiser. I think that on the whole it worked - after all, all the puzzles did get unlocked in the end - but perhaps the puzzles cost too much. The expensive ones were basically all unlocked by the same, very generous, player. Perhaps several players could have been able to pay to unlock a puzzle early, with a longer paid-only period, or something."

"The ARG side of things was well done, but sometimes a little slow to develop. I can see that with volunteers it would be a tall order to have more posts, but sometimes for days at a time there were neither posts nor new puzzles appearing, and us puzzle addicts need feeding!"

"the endgame was rather rushed - all those final cell-puzzles to do in a week and then cracking the meta-puzzle - it would have been nice to have had more time to work on those."

"Thanks for making the game! I really enjoyed it, wished it could have gone on longer. That's my main criticism really, as you posted many difficult puzzles, especially 'Fishy Business' right near the end of the competition so no one solved in time. Most of the puzzles were fun and challenging, some were annoyingly hard, like Microfilm 3. What was great was how you included so many different types of codes and stuff, so the same solve was never used twice."

"The existence of the Forum was also a great thing - meant that even if you got stuck you could look for ideas there."

"Not sure how well it worked with sponsorship - people didn't seem to quite get what it was they were sponsoring someone for. (i.e. what result were they expecting from you - it's not like running a marathon where they could tell how you'd done.)"

"I also don't know whether the site was clear enough in that it was about raising money for charity, rather than just being somewhere you could happily while away some free time..?"

"The forum was a good idea and should be encouraged as a way of teams helping others."

"...an absolutely fantastic job done by all at Law 37. I loved the puzzles, they were a clever mixture of easy fun ones and real challenges, which I would never have solved without the help of the fine community of players on the forum. The story was great too and I loved the blogs from all the agents - I never thought Nova would turn out to be bad, but I feel so full of loveliness that she's already forgiven! The set up worked seamlessly for me, the website was always up, and any difficulties I had with solutions were all my own fault!"

" would like to be be more fulsome in my praise to the wider community because as volunteers I think you have done an awesome job - basically I've assumed you guys were professionals (ie paid) all the way through (even though I know you weren't) because the quality of delivery has been so high from start to finish."

"What you attempted - and achieved - with OSC was simply brilliant; you must be proud of it. Dozens (hundreds?) of original, high-quality puzzles; a convincing and well-constructed theme; stylish design; a great website; neat ways of inducing people to donate (even persuaded me to part with a few quid, doesn't happen that often) e.g. by unlocking cells to get a head-start. The forum was a particularly good idea, providing help when stuck and giving a sense of community."

Statistics

AMOUNT RAISED: £3,638

AMOUNT RAISED PER WEBSITE VISITOR:68.6p

NUMBER OF DONATIONS: 109 (by 66 people)

NUMBER OF PLAYERS WITH 4000+ POINTS: 42

NUMBER OF PLAYERS WITH 1500+ POINTS: 72

NUMBER OF PEOPLE REGISTERED: 605

NUMBER OF FORUM POSTS: 1296

NUMBER OF PLAYERS WHO MADE FORUM POSTS: 62

NUMBER OF MEMBERS OF FACEBOOK GROUP: 135

NUMBER OF AGENT HERRING TWITTER FOLLOWERS: 34

NUMBER OF PLAYERS ATTENDING END-OF-GAME SOCIAL: 2

NUMBER OF UNIQUE WEBSITE VISITORS: 5300

NUMBER OF WEBSITE PAGE VIEWS: 183000

AVERAGE TIME ON SITE: 11.5 minutes

TOTAL TIME SPENT BY PLAYERS ON SITE: 3000 hours