Post-Rendering 3D Warping

William R. Mark, Leonard McMillan, Gary Bishop

Proceedings of 1997 Symposium on Interactive 3D Graphics (Providence, Rhode Island, April 27-30, 1997), pp. 7-16.

Abstract

A pair of rendered images and their Z-buffers contain almost all of the information necessary to re-render from nearby viewpoints. For the small changes in viewpoint that occur in a fraction of a second, this information is sufficient for high quality re-rendering with cost independent of scene complexity. Re-rendering from previously computed views allows an order-of-magnitude increase in apparent frame rate over that provided by conventional rendering alone. It can also compensate for system latency in local or remote display.

We use McMillan and Bishop's image warping algorithm to re-render, allowing us to compensate for viewpoint translation as well as rotation. We avoid occlusion-related artifacts by warping two different reference images and compositing the results. This paper explains the basic design of our system and provides details of our reconstruction and multi-image compositing algorithms. We present our method for selecting reference image locations and the heuristic we use for any portions of the scene which happen to be occluded in both reference images. We also discuss properties of our technique which make it suitable for real-time implementation, and briefly describe our simpler real-time remote display system.

Full Paper (except plates): PDF (0.3M), or compressed postscript (0.5M), or uncompressed postscript (5.0M)
Color Plate Page: Adobe PDF (0.6M), or compressed postscript (1.7M), or uncompressed postscript (3.7M)

Individual Color Plates:

Captions (B&W postscript)
Plate #1 (24-bit TIFF)
Plate #2 (24-bit TIFF)
Plate #3 (24-bit TIFF)
Plate #4 (24-bit TIFF)

[Note: The caption for Plate#3 in these downloadable files incorporates a correction for a minor typo that appeared in the published paper.]

Video Clips

• Sequence #1: Conventional rendering at 5 frames/sec
• Sequence #2: Our algorithm. Reference frames at 5 frames/sec, interpolated to 30 frames/sec.
• Sequence #3: Conventional rendering at 30 frames/sec, for comparison purposes
• Sequence #4: Split screen -- Our algorithm on left, Conventional 5 frames/sec on right
• Sequence #5: Address Recalculation (standard projective warp) only; Reference frames at 5 frames/sec.
• Sequence #6: Like #2, but with larger average prediction error (5.0 cm instead of 2.0 cm)
• Sequence #7: Like #6, but reference frames are generated at only 2.5 frames/sec. Interpolated frames are still at 30 frames/sec.
• Sequence #8: Gives a feel for how the algorithm works. Discontinuities are shown in green. This uses only one reference image at a time, and is at reduced resolution.

• Sequence #9: Our real-time remote display system, captured from videotape at 8 frames/sec. [Set playback rate to 8 frames/sec].
  QuickTime movie (23 MB);
  MPEG (1 MB)