Direct Manipulation is a term coined by Ben Schniederman. It is the foundation for systems in which the user manipulates objects representative of the real world. However, the concepts behind Direct Manipulation Systems include opportunistic planning and feedback. Direct Manipulation is basically an application of the two features. System feedback is necessary in order for the user to successfully progress to be able to backtrack, if necessary.
One property of a Direct Manipulation system is "continuous representation of the objects and actions of interest". This means that the system should provide indication of what the current status is, what errors have occurred, and what actions are appropriate now. Another property is the need to have "Physical actions instead of complex syntax". We want the system to simulate real world objects as much as possible. Therefore, we also want the actions necessary to manipulate these objects. Lastly, Direct Manipulation systems should provide "immediate and continuous feedback". This will allow the users to quickly realize solutions along the way and adjust their planning as needed to accomplish their goal.
A simple example of this, given by Ben Schneiderman in "Designing the User Interface", is the procedure of driving a car. The scene is directly visible through windows. This supports the first property of "continuous representation of the objects and actions of interest" To turn left, the driver simply rotates the steering wheel to the left. This incorporates "Physical actions instead of complex syntax". The response is immediate and the scene changes, providing feedback. Now, imagine that instead of physical actions and immediate feedback, we had to give a command "TURN LEFT 30 DEGREES", and then had to give another command to see the new scene. This is a simplistic, but good example of the advantages of having a direct manipulation system. The main objective in HCI is to apply these concepts to computer systems. It is also important to mention that most computer-based Direct Manipulation systems employ graphical interfaces to aid in providing the above three properties (Ben Schneiderman, 1987).
The major aspect of direct manipulation systems is "directness". Designers tend to ask "How do we make a system less difficult to use?", "How do we provide a feeling of directness". In other words, "How do we require minimum work for the user such that the system usage feels natural?". The answer to this is to reduce the cognitive distance and increase the direct engagement. The cognitive distance is mainly the mental requirements necessary for the user to perform tasks with the system. Direct engagement is what the user should feel is occurring. Actually, the user should not be aware of the system.
In discussing cognitive distance, there are actually four distances to be concerned with. The first two are the gulf of execution and the gulf of evaluation. Both of the gulfs are concerned with the distance between the user's goals and the physical requirements of the system. The Gulf of Execution can be defined as the distance between the input of the goals and the input requirements of the system. The Gulf of Evaluation is the distance between the feedback of the system and the actual verification that the goal was achieved.
The other two distances covered in the paper are Semantic Distance and Articulatory Distance. These can be considered as subsets of each of the gulfs, but in reference to input and output. Semantic distance is the distance between the user's goals and the meaning of expressions. Articulatory Distance is the distance between the meaning of expressions and its physical form. The "Direct Manipulation Interface" article refers to the Semantic Distance and Articulatory Distance as Interface Languages (Hutchins, Hollan, Norman). This is because they separate the user and the system based on the input and output interface provided to the user.
The main objective in reducing the cognitive distance is as follows. We want to minimize the effort to translate goals into the system's language. We also want to minimize the effort required to interpret the results of the feedback. In summary, we want to decrease the gulf of execution and evaluation, as well as the semantic and articulatory distances.
The Semantic and Articulatory distance fits within the gulfs of execution and Evaluation. In understanding them we start with the goals. A user would form their intentions from the goals. This is the mental activity that covers the Semantic Distance within the Gulf of Execution. The user would then form an action specification which is the mental activity of determining what expressions need to be used. This is the Articulatory Distance within the Gulf of Execution. Next, the user executes the steps formed from the action specification. Finally, feedback is perceived and interpreted. This is the Articulatory Distance in the Gulf of Evaluation. In any of the above steps, we determine if adjustments need to be made in the planning/actions. Inter-referential input and output aids in the concept by allowing output of one phase to be input to another phase (Hutchins, Hollan, Norman, et al, 19??).
Here is an abbreviated form of an exercise in going from goals to evaluation.
| _ | _ | Goal | = | Get a hard copy of a document |
| Gulf of Execution | Semantic Distance | Intention | = | Get the desired file in the computer system |
| _ | Articulatory Distance | Action Specification | = | The mental sequence of action needed to get a file.
1. Go to menu 2. Click on file 3. Select file 4. Wait for output |
| _ | _ | Execution | = | Performing the actions. |
| Gulf of Evaluation | _ | Perception | = | Seeing the document displayed. Noticing that feedback occurs, if any. |
| _ | Articulatory
Distance |
Interpretation | = | Get output of document on the screen. User mentally asks "What does this feedback imply?" Did we get a document or not? |
| _ | Semantic Distance | Evaluation | = | Compare the document on the screen with the user's goal. What happens next? |
Next, we ask "how do we bridge the distances. We could bring the user goals closer to the systems requirements. This requires more cognitive work for the user. We could bring the system closer to the user. This is the key to Direct Manipulation. You can bridge the semantic distance and the articulatory distance. This involves creating an interface that allows one to specify goals in a straightforward manner. It should also be easy to compare feedback with the goals in order to evaluate the accomplishment of the goal. This also involves providing an interface in which actions are specified in a direct manner and interpretation of any feedback should be natural.
Direct Engagement generally means giving the user a feeling of directly manipulating objects on the screen. There are two ways to refer to objects. Either using the Conversation metaphor or the Model World metaphor. The conversation metaphor is an interface based on implicit representations of the object. The user has to describe actions necessary to manipulate the objects. The model world metaphor is an interface in which the user performs actions on models of real world objects. It involves state changes based on the user's actions. An example was given earlier with the automobile example. A simple example with computer systems is file manipulation via a "Menu-Based System" .vs. "Macintosh Graphical System".
Direct Manipulation interfaces minimize distance and maximize engagement. The solutions to providing Direct Engagement includes all of the previous discussion on semantic and articulatory directness. In addition, we want execution of tasks to be direct and intuitive. We want evaluation to require minimal cognitive effort. I/O languages should be inter-referential. This means input expressions should make use of a previous output expressions. The system should be responsive with very little delay between execution and results, if any. The user should not be aware of the interface. Otherwise the feeling of directness is compromised.
Direct Manipulation usually leads to an increase in performance. One advantage of Direct Manipulation systems is that the system is easier to learn for both the novice and experienced users. These systems make learning less frustrating for the novice user because they can learn by observing a more experienced user. The expert user can actually work more efficiently by exploring a familiar Direct Manipulation system at an advanced pace. Real world metaphors are useful in this case. Tasks are also performed more rapidly due to the ease of use and the availability of immediate feedback. There is an increase in retention of information because the user does not have to memorize complex syntax. Users can immediately see if their action are furthering their goal and make adjustments accordingly. This may lead to modification of the planned route but is necessary to provide efficient systems with less opportunity for errors. We have reduced error rate in Direct Manipulation systems because there is less syntax interpretation. In addition to a reduction in errors, we have error control due to users being able to easily detect and back out of dangerous situations.
Although Direct Manipulation allows users to do less work when learning and training, there are some situations where these systems can be more detrimental than helpful. Users can become more confused if a Direct Manipulation system is designed poorly. The wrong metaphor, or a too cluttered graphical presentation can be harmful rather than helpful. Another problem is having to know or learn the meaning of graphical representations. If the representations are not intuitive to everyone, then it can not always be advantageous. It is also known that Direct Manipulation systems are not the perfect tools to use in all situations. Repetitive operations are done best via scripts rather than graphical manipulation. For instance, mathematical symbols can be easier to do via text commands rather than point and click methods. Also, for the experienced typist, moving a mouse may be slower than typing. Another problem with Direct Manipulation systems is that creativity is compromised. New ways of thinking or doing things are hindered if we provide metaphors that always support the "customary" way of thinking. Direct Interfaces requires the user to be more precise in controlling their action which is sometimes not possible with graphical systems. Therefore, accuracy becomes a problem. Finally, Direct Manipulation systems require more resources. For example, a mouse, keyboard, high resolution screens, and a decrease in screen space due to graphics are some cost concerns.