Advanced

Results of Expert User scenarios

The Good

Users did the following things which demonstrate good strategy or proficiency:

Right-clicked as a shortcut for default commands

Used hotkeys when they needed increased efficiency

Selected multiple units with all three methods (drawing a rectangle around units to select them all, shift-select to add or remove a unit from a group, and alt-select to select a previous group)

Changed the speed of the game to suit their needs

Moved units using the small map, rather than scrolling the screen (using the small map is faster but less accurate; it is more appropriate in some cases)

Attacked the tower before the other structures (the tower is the only structure that attacks and should therefore be destroyed first)

Used the battleship's "attack ground" function to test its range (Battleships cannot fire too far from the shore. If you try to attack an enemy that is out of range, there is no feedback. The users employed the "attack ground" function to make the ship fire at the ground, thereby testing its range.)

Determined what the conjurers did by experimentation (one user had never seen conjurers before, but he was able to discover their abilities and make good use of them)

The Bad

Users complained about not being able to do the following things:

Select all units of a certain type, as can be done in Starcraft

Have their units automatically destroy all buildings in their range, as they do in Age of Empires

Give their units a specified path and tell them to kill everyone along that path

Have their clerics and paladins automatically heal weak units

Have their units automatically stop and fight if they were attacked in transit

Have other units realize what was happening and make themselves useful (e.g. destroy the unguarded enemy building, attack the nearby enemy that is trying to destroy a building)

Users complained about the following deficiencies of Warcraft I as compared to Warcraft II:

Only being able to select four units at a time (as opposed to nine in Warcraft II)

Not being able to select units by drawing a rectangle around them (it is actually possible to do this, but the users did not know they had to press CTRL to do so)

Rough screen scrolling

The Ugly

Users made the following mistakes by accident:

Underestimated the area that would be affected by the "rain of fire" spell and attacked their own units

Cast the rain of fire spell directly on their own units by trying to select a unit, not realizing they had previously selected rain of fire

Cast the wrong spell when they meant to use rain of fire

Users did the following things which demonstrated poor strategy:

Sent in their catapults ahead of their other units (catapults' strength is their power at long range; they are completely helpless at close range and are easily destroyed)

Did not destroy the appropriate buildings first (destroying certain buildings cripples the enemy; destroying other buildings has no effect)

Attacked with conjurers (conjurers have valuable magic powers but are easily killed in combat; they should be carefully protected)

Attacked a grunt rather than the necrolyte who was healing him (the grunt was effectively made invincible by the necrolyte's magic and sustained no damage; the user should have attacked the necrolyte first)

Didn't move their units when a poison cloud appeared over them (in which case they died), or were very slow to do so (in which case they were severely injured)

Were not aggressive enough, allowing the enemy to keep producing units when the user had no production capability

Discussion

A Spiral Curriculum For Strategy Development

The expert users in the study had no trouble controlling many different types of units. Warcraft makes it easy to learn how to use new units, introducing them gradually through a spiral curriculum. Rather than immediately giving the user access to all of the different types of units, the designers of Warcraft decided to start them off with just a few types. When the user advances to a new level, a new type of unit is usually made available to him. The user integrates this unfamiliar unit with the units he has already mastered, and by the end of the level he is proficient with his entire arsenal.

However, all of the expert users demonstrated several acts of poor strategy. This shows that users can become experts in many aspects of a system without learning the best way to do things. In the case of a game, this does not have much consequence. But when someone is learning an application system for their profession, poor strategy means poor productivity. Just as a Warcraft user can win the game without learning the best order in which to destroy enemy buildings, a spreadsheet user can become an expert in most of the functionality without learning the most efficient way to merge two spreadsheets.

In Warcraft, good strategy can be taught through a spiral curriculum. Consider the case of a user who destroys enemy buildings in whichever order is most convenient, rather than destroying key buildings first. Although this is poor strategy, it will probably not stop him from winning the game, so he is not likely to change his tactics. But what if this strategy did stop him from winning the game? What if a level started with an enemy base right next to the user's base, and the enemy was so strong that it was not possible to survive unless the first thing the user did was destroy the enemy's barracks? In this case, the user either learns a piece of good strategy or he loses the game (and hopefully tries again). The successful user might be rewarded with a message such as "Congratulations! You found the enemy's weakness! They are helpless once their barracks is destroyed!" The user learns to incorporate this piece of strategy into their overall strategy, and they progress to the next level.

This idea can be used in other domains as well. Application systems that attempt to train users through tutorials may successfully teach users much of the system's functionality, but they are not necessarily effective at teaching strategy. If a tutorial includes scenarios where a certain task must be completed, the user will learn some functionality. If a tutorial includes scenarios where a certain task must be completed in the optimal way, the user learns some functionality and some strategy.

Mitigating the Production Paradox

A user who buys Warcraft will probably want to start playing it as soon as possible. But having the most fun requires suspending game play long enough to read certain parts of the manual. The user is therefore trapped in a production paradox; he must temporarily postpone his goal of enjoying the game in order to best achieve that goal.

The production paradox caused difficulty for the expert users in the study. When they played Warcraft I, all of them complained about not being able to select multiple units by drawing a rectangle around them, as they could do in Warcraft II. It is actually possible to do this, but the control key must be held down while the rectangle is drawn. Not only did the users not consult the manual during the study, they had never read the manual at all. They started playing the game immediately, without learning this technique that would have spared them some aggravation.

Users' failure to read the manual is a common problem for system designers. Windows 95 tried to solve this problem with a welcome screen that gives a tip when you start Windows. There are two major problems with this. First, many users hate the welcome screen and quickly disable it by unchecking a checkbox labeled, "Show this Welcome Screen next time you start Windows." Second, many users who leave the welcome screen enabled don't consistently read the tips closely. They may think they know all the tips when there are actually a few they haven't read. (In Windows 95, a text file called navigate.scm cannot be directly opened by double-clicking it, because it does not have a .txt extension. However, it can be opened by dragging it onto a notepad icon. Many proficient, "tip-reading" Windows 95 users do not know this, even though it is in the tips.)

We suggest three ways to help mitigate the production paradox. (1) Don't give the user the option of disabling tips until every tip has been displayed at least once. If the user attempts to disable the tips immediately, a message should explain that it is very important to read all the tips, and so they cannot be disabled until the rest have been read. This will at least guarantee that all tips are eventually displayed, and the message about the importance of the tips may encourage users to read them.

(2) Give context-sensitive advice. The first time a user builds an oil tanker, display a message saying, "Oil tankers are used to build oil platforms so they can extract oil. Send your tanker out to find an oil patch, and build an oil platform on top of it." The user gets timely, relevant advice, and these messages are not annoying because they are only shown once.

(3) Use key parts of the manual as the copyright protection. Many games have some kind of copyright protection to ensure that the user has the manual, and therefore probably has a legal copy of the software. A typical example of copyright protection is to show a picture of a building and ask, "What is the armor level of this structure?" This question must be answered correctly to continue playing the game. Why not ask multiple-choice questions such as, "How do you select multiple units?" If the user does not know the answer, he will be forced to look it up in the manual. As well as enforcing the copyright protection, questions such as these will give users some useful tips.