CS4624 Text - Ch 7

Please skim this chapter while covering Unit 3.
When covering Unit 5, the chapter should be read again.

The figures should all be understood, as well as the key concepts discussed below. Sections where many of the details can be skipped include: 7.3.4 and 7.4.1.

See Study Questions.

See Course Notes for further explanation and to see most crucial points.

7 Time-Based Media Representation and Delivery

7.1 Introduction

• Synchronization = orchestration = coordinating different media sequences of data (or real-world events)
• Timeline, temporal relationships, representations
• Physical level: transmit or store
• Service level: intermedia snc for presentation/playout
• Human interface level: browsing, interactions

• Time dependencies (e.g., static = none; live = data originate in real time; persistent = maintained in DB; natural = real-world time dependencies; transient = short duration)
• Continuous media = sequences of discrete data elements played out contiguously in time, with fine-grained AV synch

• Instants are a special case of intervals.
• Intervals can relate in time 13 ways.

• Representation scheme: language, schedule, precedence, timing, concurrency
• Specify time in suitable manner for authoring
• TAC = temporal access control operations
• Integrate with spatial organization, document layout models

• TIB = temporal interval based
• Can handle all 13 time relations, synch, reverse, partial playout

• reverse, fast-backward, fast-forward
• midpoint suspension/resumption
• random access, looping
• browsing

• Coercion (fit one by compression/expansion)
• Expand static image to fill period by continuing

• scaling, cueing, inverting, translation (shifting)
• virtual time, unitless
• units: clock ticks, samples, frames
• groups: strands, ropes

• fading of audio and video
• mixing channels
• enhancement of images
• Must coordinate these spatial changes with temporal ones in some uniform repr.

• pictorial, visual, iconic
• timelines, flow-graphs, timed petri nets, temporal hierarcies

• extended CSP
• HyTime language and standard
• Other languages must represent parallel, sequential, looping

• loose coordination like browsing, submitting queries, user control
• tight coordination like VR, driving simulation, interactive movies
• hypertext controls: nonlinear, cross-reference

• soft deadlines
• access, query, database aspects

• relax to within requirements
• jitter, skew
• reference event vs. synchronized event, tolerances

• start with TPN and RDBMS model and produce:
• temporal hierarchy, with internal nodes specifying temporal relations

• Where are sync points with variable bit rate (VBR) compression?
• Can we start at an arbitrary (random) point?

• handle: storage latencies, network delays (end-to-end)
• use: buffering

• late arriving data
• lost or discarded data
• varying playout rate
• reconstruct missing values using interpolation