Edward A. Fox
Department of Computer Science,
Virginia Tech, Blacksburg VA 24061-0106
This Unit has three articles that should be studied. In addition, the videotape on Interactive Digital Video can be viewed in the Library. In the lecture a general overview will be given using KMS or Acrobat. Finally, demonstrations and videotapes will help round out the picture.
The area of multimedia is rapidly emerging, in part because it allows computers to communicate in ways more convenient and effective with humans, and in part because of rapid improvements in technology. Improvements have enabled media such as audio and video to be presented by computers, then to be captured in small amounts, then to be compressed in larger amounts, and finally to be more easily managed by computers and networks.
As is discussed in the first reading, many areas of computer science can and must be applied. A whole new set of jargon has emerged, relating to storage units, compression techniques, networking approaches, and special computer systems.
Of crucial importance is compression. When images are compressed, say by using the new JPEG standard (see 2nd article), approximately 20:1 savings in space is realized. This allows images to be handled on current computers, and large image collections to be managed using CD-ROMs or network servers. JPEG makes use of the discrete cosine transform (DCT), Huffman or arithmetic coding, and some other special tricks that altogether can be handled by special chips or fast host processors.
Standards for video (e.g., MPEG and px64) build upon the JPEG techniques, adding in other methods to remove temporal redundancy (i.e., repetition from one frame of data to the next).
For multimedia to catch on, the next real barrier is software. Without real time operating systems and fast software, multimedia information cannot be presented and manipulated interactively by users. Without good software, developing multimedia applications is prohibitively expensive and time consuming. Without the use of modern object-oriented development efforts, both of the above types of efforts are doomed to early demise because of failures to keep up with rapid change in hardware, extension to myriad operating systems, and cross-platform portability requirements. The third (and fourth, optional) reading deals with many of these issues.
Though just exposed briefly to these concepts, students will find this area to be quite exciting, and will be equipped with some of the key knowledge and concepts needed to comprehend and work in this emerging industry.
From the Course Objectives an important goal is: read and understand research contributions ...; you will gain experience by reading the three CACM articles, by the in-class discussion relating to those, and by the videotape presentations and demonstrations of research systems.
This Unit has the following objectives, for students to be able to:
There are two main types of effort required. First, the readings (see the next section) should be carefully studied, keeping unit objectives in mind (see the previous section). Second, if possible, the videotapes and lectures, and the demonstrations, should supplement the readings.
Students should run the
program xv on the /u1/data/images/4.* files using
fox.cs.vt.edu. Also, do ls -l 4.* in that directory to
see the file sizes. Note that file 4.60.jpg was
produced by command
tifftopnm 4.tif | cjpeg -Q 60 > 4.60.jpg
run in that directory.
If you cannot run X, you can do all this by going to
the WWW version.
Based on the file sizes and your assessment of quality, which of the versions of this image would you recommend putting out on WWW? Please explain briefly.
Students should run the program mpeg_play on the /home/video/abdulla/*.mpg files using video.cs.vt.edu. Or, use the WWW and run a local MPEG player for the *.mpg files there.
Send to the instructor the name of the X terminal used, and the average frames per second for each of 5 files. (Hint: you may want to pick the smallest files, since that will take less time.) If you are not running X, but work with the WWW, see if you can determine how close to normal speed (30fps) these are. Please explain why these values are not 30.
Also, view a movie about Hawaii and tell the instructor briefly what that MPEG movie is about.
Please be sure to look at the WWW course notes for this unit and if your computer supports it, experiment with some of the audio and video files. Tell the instructor the 3 WWW pages you visited that you found most interesting, and explain briefly why.
You should be able to answer each of the following questions.
The four articles are relatively diverse, covering various aspects of the field. Only the first three are required. Respectively, they provide an overview, detailed discussion on compression methods and standards, one approach to systems software support, and finally an approach to authoring software.
This article provides an overview. It should be read carefully, and students should study and be able to refer to the definitions of acronyms, terms, and phrases. Note particularly the various standards and the approaches to compression.
This article should be carefully read up through page 35. The rest can be skimmed. It is important to understand the goals of JPEG and how DCT and Huffman coding work with quantization and zig zag encoding to yield a (variably reduced) compressed bitstream. If the images on page 42 cannot be clearly seen, you may wish to look at the original journal issue, on reserve in the library.
Intel's software for DVI is discussed in this article. It is quite interesting, tying in with work on object oriented system, software development, co-processor architectures, and general operating system issues. Please study the glossary carefully. Indeed, the whole article should be read closely. It is interesting to see the failings of the original DVI software, how a new better conceptual model was developed, and how it was implemented using an object oriented approach. Unfortunately, Intel has since discontinued DVI.
MediaView is an interesting system. It builds upon the metaphor of a long scrolling article, that can be enhanced with annotations that involve various media types. Though the article will not be required, it is of interest in that it shows how a simple approach to authoring can go a long way, and how important object-oriented development tools are when building such a complex piece of software.