CPS 890 - Independent Study

matt shooting VR

Developing QuickTime VR movies
(and how to serve them on the web)

by Matt MacQueen, July 1996

Advisor: Bob Matson, MSU Computer Laboratory

Contents of This Page

Objectives of This Independent Study:

There were 4 major objectives of this Independent Study course:
  1. To learn how to use the equipment necessary for capturing raw VR source material
  2. To master the software tools necessary to author QuickTimeVR movies
  3. To serve these QTVR movies on the Web
  4. To present this information to the MSU Multimedia Interest Group and pass the knowledge and process on to other multimedia developers.
Learning the necessary equipment for source material entailed getting proficient at setting up and using the digital camera, the tripod, the special QTVR tripod-head and powerbook connection to download the images.

Mastering the software tools to make QTVR movies involved learning how to program with Macintosh Programmers Workshop Pro (MPW-Pro) and the QuickTime VR Authoring Tools Suite.

Serving the QTVR movies on the Web required using special internet movie tools and an HTML embedding process. I also used the Internet for development tips and to joined a QTVR developers email list at Apple that was extremely helpful.

The final objective of a presentation was carried out for the MSU Multimedia Interest Group. About 20 members attended this demonstration. I explained how the shooting and authoring software worked. Then I did a demonstration VR movie of the meeting at the MSU Faculty Facility for Creative Computing (FFCC) which included shooting, stitching, dicing, and putting the movie on the Web. This presentation was important to transfer the knowledge and experience to others at MSU interested in authoring with QuickTime VR.

Introduction to QuickTime VR:

QuickTimeVR is a new multimedia technology from Apple. QuickTimeVR allows a user to view a scene (photographic or rendered) "in the round." Users may navigate in and around a scene with just their mouse and the proper software. While surely not immersive in the "goggles and glove" sense of a virtual reality experience, QuickTimeVR is uniquely called "VR for the rest of us" -- meaning you don't need the funding of NASA to create and use these virtual experiences.

I find this technology intriguing for it's potential uses in multimedia such as CD-ROM virtual "tours" of places, as well as it's potential uses on the web. There is currently no course at MSU available for any type of VR development like this. Luckily, Bob Matson has taken a special 2-day course from Apple on how to make these movies.

Bob and I had several discussions about our eagerness to learn more about QuickTimeVR and realized that he had only scratched the surface of the development process at the seminar. Together we decided to really get our hands in the guts of the development process and learn the process thoroughly from the ground up. The Independent Study (CPS 890) was for me to learn how to make QuickTimeVR movies (and also have access to the necessary photographic equipment, software, and high-powered development computers available at the MSU FFCC.

This webpage attempts to explain the QTVR development process, provide tips gained from experience (read: making mistakes), and also provide other web resources for QTVR development and demonstration.

up to Contents

Equipment, Hardware and Software:

QuickTimeVR movies are composed of still shots, and despite the term "movie," no video is used in the process.

You will need something to actually capture the still images with. A single-lens-reflex 35mm camera is recommended for the highest resolution, though Apple's QuickTake 150 digital camera works as well (the resolution and lighting do suffer somewhat). If you want QTVR movies that you can zoom way into and still retain detail, you must use 35mm film and develop to Photo CD (a digital archive format). The QTVR movies taken with a QuickTake 150 really break down in image quality when you zoom in more than 50%.

Just as important as the camera is the tripod head that rotates the camera 360 degrees while shooting. The series of overlapping still images are later digitally "stitched" to make one very long picture. Thus, the uniformity of how these images are shot is crucial to the finished product looking seamless. The tripod head -- with it's levels and rotational degree markings -- assures this uniformity.

Because of the peculiar stitching process done to the images to create VR movies, the focal point of the camera must remain in the exact center while the camera rotates. Not only does the camera have to be exactly level, but it's focal point must be centered exactly over the panning axis. To do this requires a special tripod head specially made for QTVR shooting.

Equipment providers such as Kaidan make these specialized tripod heads, including the one the FFCC bought for the QuickTake 150 here at MSU. They also make QTVR tripod heads for 35mm cameras. The Apple Developer Catalog Online carries a wide line of Kaidan panorama tripod systems and components.

Other QTVR shooting equipment providers are Peace River Studios (email: peace.river@applelink.apple.com) and VRToolWorks (email: vrtoolwork@aol.com).

In our case we used the QuickTake 150 camera and the Kaidan tripod head made especially for this camera. (Costwise, this is also the cheapest possible QTVR setup). Hopefully the Computer Lab might someday invest in a rig that will support 35mm cameras. An advantage of the QuickTake, however, is that there is no scanning or Photo CD's to be made once the images are taken, they can simply be downloaded onto a nearby computer.

As for software, there are 2 free tools available from Apple for developing QTVR movies, though they are very basic (beta versions) and crude compared to what you can do with the real development kit. The free utilities are Make QTVR Object 1.0b3 and Make QTVR Panorama 1.0b4. These tools are meant to be a taster to what can be done with the real QTVR developers kit. These tools are very limited and not even recommended for use with multi-node QuickTimeVR projects.

For serious development we used the QuickTimeVR Authoring Tools Suite 1.0, which also includes MPW Pro. This suite can be purchased on-line. The tools suite includes all of the capabilities of the free beta-level tools but much greater flexibility. With this software you can "stitch" separate photos together to create a 360 degree panorama. You can also add "hotspots" to panoramas that trigger actions.

This suite of Macintosh tools contains all you need
to create linked panoramic and object movies
from photographic, video, or computer generated images.
The suite includes documentation for planning,
designing, photographing, and creating QuickTime VR
panoramas, objects, and hot spots to link them. 
 

The authoring tools include:

DeBabelizer is another useful piece of software for batch-processing images. This automates the process of reducing the source images to half-size for easier stitching, if so desired.

Hardware needs are pretty extreme as well. MPW-Pro acts like a giant Photoshop when it processes all the images and stitches them together into a enormous cylindrical PIC file, then re-sizes and sharpens it. These are all very memory-intensive operations. A PowerMac is necessary with a bare minimum of 40mb of RAM for these operations. We used at PowerMac 8100/120 at the MSU FFCC for this.

up to Contents

The Shooting Process:

In early May, Bob and I met up on a Saturday morning to shoot QTVR of various places around campus. Using the QuickTake 150 and a Powerbook, we started on the footbridge behind the Computer Center.

Because the QuickTake only stores 16 images at once, we configured the tripod head to overlap each picture at 16 clicks around to equal 360 degrees. Depending on how many pictures you need to take to get a full panorama, you can adjust the tripod head accordingly. This makes the rotation of the camera much easier than trying to do it with tiny degree markings every time. With this setup, the pictures overlap enough to be stitched. A 30% overlap from shot to shot is recommended, though this varies.

First I made sure the tripod was completely level. There is a small circular bubble-level on the top of the Kaidan head, and looking down at that was the best we could do. Better tripod heads have two horizontal levels that cross each other. This is much more accurate for true leveling. If the shots are not level, when stitched they will appear to wobble up and down as the user pans across the QTVR panoramic movie.

Once the leveling step is complete, I cleared the QuickTake's memory (if there were photos stored in it). Then we surveyed the area surrounding us to see if people or cars (or usually construction equipment) would interfere with the shot. Another crucial thing to watch for is changing light patterns. We were shooting in daylight, which makes for a much more even exposure - on a clear day. We were shooting on a partly-cloudy day which can be very problematic.

Partly cloudy days are difficult because once you start shooting in the round, if the sun goes behind a cloud or suddenly becomes brighter the light exposure will differ from one shot to another. When stitching the images together, this will create a blurred line from a darker image to a lighter one. The goal of stitching is for the many images to combine like one extremely long picture. If there are bands from shot to shot depending on lighting, this ruins the seamless effect.

At times I would start to shoot a panoramic view and then have to wait several minutes during the rotation to see if the sun would come back out from behind the clouds so the exposures would be even. The best days to shoot outside are clear, sunny days. These are sometimes hard to get in Michigan on the days you want to shoot.

On the bridge, once we shot 16 pictures (one for each click on the tripod head) until we had completed the panoramic shot for that area. Since the camera's memory was then full -- containing just enough images for a full panorama -- we hooked it up to a Powerbook and downloaded the shots immediately onto the hard drive. Then we relocated to a new location, cleared the camera's memory, and repeated the process again for another node.

We tried to shoot nodes so that you could see a clear shot to the next node. Thus, to create a multi-node VR movie, a user could click through hotspots assigned in the single VR movie to get around from one node to the other. These separate VR movies can be linked together to form walkthroughs.

The QuickTake Image Access software on the Powerbook is very simple to use. It acts as a control panel which mounts the camera (like it would an external hard drive) and puts a camera icon on the desktop. You copy the photos out of the camera just like you would a disk, except the data travels through a serial cable, rather than a SCSI cable. Once the download is complete, the cable is disconnected from the camera. The Image Access software even builds document icons for each image.

We moved around the campus shooting and downloading nodes for the entire day, until the sun was too low in the sky to have any more even exposures. I shot down by the river, behind the administration building, on the library bridge, through Beal Gardens, at the fountains in front of the library, and out towards Beaumont Tower. Altogether we shot over 20 VR nodes that day. By that time the sun had sunk too low to get an even exposure all the way around the panorama. Once the images for the nodes are shot and downloaded they need to be processed into QuickTime VR movies.

up to Contents

Processing The Images into VR

I transferred the images from the Powerbook into the PowerMac 8500. There are three things to do to the images at this point:

  1. Stitch them into one big picture
  2. Dice this big picture into frames that form a QuickTime movie
  3. Transform this diced QuickTime movie into a QuickTime VR movie.

After trying unsuccessfully to stitch the full-size (640x480) images together -- 40mb was not enough RAM -- we used Debabelizer to shrink the batch of images to half size, so each image was 320x240. The stitcher was able to handle this size.

The "stitch" command is used by the MPW-Pro shell software to overlap all the images at their seams and blend them seamlessly into one long panoramic picture. First the stitcher lines up the first two images and calculates roughly what adjustments are necessary to stitch these first two images successfully. Then the rest of the images are stitched based on this initial calculation process. Our panoramas were 240x3000 pixels, though you can specify larger or smaller panoramas.

Through a systemic trial and error process -- documentation is cryptic at best -- we developed a script that worked well for our images. There are many variables to tweak depending on the many input values for various stitching controls. Getting just the right script took many weeks of debugging tests and often the machine crashed if it ran out of memory in the middle of an execution. Debugging became a systematic process of looking at the processing output script for clues of what values needed to be changed for a successful stitch. This output is generated with the "-verbose" command is included.

Notice in the script below, MPW-Pro needs to access the files on the hard drive in folders, to all directory paths must be correct. The final script we developed looks like this:

###############
#doing same triple-process for N-Node (N- south door Union)
#change node letters to "N"
 
#stitching
stitch -fovy 27 7 -files 00-15 -rotate 90  -offset 180 0 ¦
-range 50 10  -wrap  -cropOut  -sharpen  -blend  -fill  ¦
-show  -dir "A/V WORKSTATION:QuickTake pix 5/11:N- south door Union"  ¦
-out "A/V WORKSTATION:Matt output:N-Node"  ¦
-outHeight 3000  -outWidth 240  -verbose -cyldim 500 4000
 
#dicing
p2mv 1 24 "A/V WORKSTATION:Matt output:N-Node.mov"  ¦
	-in "A/V WORKSTATION:Matt output:N-Node"   ¦
	-codec "cvid"	
	
#makes it QTVR movie
msnm "A/V WORKSTATION:Matt output:N-Node.srcmov"   ¦
	-source "A/V WORKSTATION:Matt output:N-Node.mov" 1 24 ¦
	-windowSize 320 200  -vPanRange 14.5 -14.5
 
 

As stated above, the output script for this input can be examined as well.

Each command represents a variable or filename. The three commands, stitch, p2mv (dice), and msnm (makes it a QTVR document) can be executed together as long as each command finds the files it needs in the correct directory path. All three of these commands required about 7 minutes of processing time for the PowerMac 8100. A stitched pict looks like this, but the actual size is over 3000 pixels long:

long panorama

The procedure described is for creating what is called a single node movie. A multi-node movie is created by linking two or more single nodes together using a software tool called the Scene Editor. The Scene Editor is a crude hack of a software program. It is a Hypercard stack that you duplicate a "virgin" copy of and then plug your files into. Like MPW-Pro, various files need to be put (with appropriate file extensions) in pre-named folders. In the Hypercard interface, you drag nodes around a screen (a makeshift map of your site) and your nodes pop up as QTVR movies in two windows. Then in a squished thumbnail version of your long panoramic pict, you drag a marquee around the areas you want to be "hotspots." Once a hotspot area is defined, open the QTVR movies where you want to link from, and the appropriate view of the movie you are linking to.

There are various parameters to enter at all stages, such data about the VR movies and amount of desired zoom when changing nodes. This is one crude software tool, it's interface is marginally better that MPW-Pro, which has no interface to speak of. At least in some ways there are buttons and menus to achieve a given task. The result of the linking process is the export of several PICT resources and a worksheet script that is fed back into MPW-Pro. Thus, the scene editor does not even link the nodes, it just generates a set of parameters and resources that MPW-Pro needs to link the movies. The appropriate script is executed in MPW-Pro and, using data from the Scene Editor along with the exported PICT resources, the final multi-node movie is created.

The QuickTime VR movies can stand alone (opened with QTVR Movie Player), or be imbedded in various multimedia authoring tools, such as MacroMedia Director, Apple Media Tool, or Hypercard. They can also be served on the Web.

up to Contents

Putting QTVR on the Web

Recently Netscape announced support for QTVR movies to play in-line in their web browser, meaning Netscape does not have to call up a helper application, but instead the movie loads in a Netscape window, even in a particular frame if desired.

netscape icon netscape plug-in icon netscape plug-in icon quicktime extension

To see use QuickTimeVR via the web, you must be using Netscape 3.0b and have the QT Plug-in and the QT VR Components installed. If you don't have them,

To embed QuickTime VR movies in a webpage, you need to open it up in Movie Player and save it as a self-contained movie (playable on non-Apple computers). Then, drag and drop your movie on Internet Movie Tool, a utility that can be downloaded for free. This process assures that the QTVR movie will stream down faster with critical information about the movie at the beginning of the download rather than at the end. This is referred to as the QuickTime Plug-In's fast start feature. Finally, to reference the QTVR movie in a webpage, simply use the "embed" command, the syntax looks like this:

<EMBED; SRC="SampleQTVR.mov" HEIGHT=200 WIDTH=320>

Replace the name "SampleQTVR.mov" with the name of your movie, and the values for height and width of the movie with the dimensions of your movie. For further detailed and excellent instructions on different parameters you can specify when putting VR movies in web pages, consult The QuickTime WebMaster's Page. This guide explains all the details.

Some Samples of the MSU campus in QTVR I shot and processed:

Note that over 20 different QTVR nodes of MSU Campus exist off-line. I have these movies available on a harddrive and also burned on a CD-ROM for archiving purposes. These QTVR movies listed here will play "in line" in your Netscape 3.0b browser, so if you don't have the proper plug-ins set up properly (see instructions above) then things will definitely not look right.

QTVR Developer Resources and Springboards:

A few highlights of QTVR Web Samples
(QTVR service providers, specialists, and experimenters)

up to Contents

Conclusions

There is no doubt that QTVR is "VR for the rest of us." It's a fun technology to use. Being able to navigate a scene in the round with just your mouse is exciting and just about any level of user can get the hang of it. If you can move your mouse around a window then you can navigate in a QTVR movie. QTVR is ideal for scenes that are visually dynamic across space, and can reveal things about a scene that a still photo of regular video cannot. Surely giving the control of the "eye" to the user is a great advantage as well.

But QTVR is still very young, even in computing years. It's difficult to develop software with QTVR because it's still a very bleeding edge technology. The authoring tools are crude and user-hostile. This is the authoring tools sure v1.0 and that's very obvious. The tools need to get much better for authoring and implementation into media projects. Only then do I predict that this will really take off gain popularity with a wider range of multimedia developers, rather than just specialists (which it is at the time I write this).

Lots of this is predicted to change, if QTVR 2.0 lives up to it's amazing sounding press release. Of particular interest to me is also the web implementations of the new version. Once QTVR movies can become dynamic documents in a web-page (and hot-spots can call up different URL's) then we can really begin to navigate a virtual scene on the web. Currently once the movies load, they can't become clickable items or links. Once this changes I predict a whole new way to explore a physical space on-line will hatch.

I wish to thank Bob Matson for having interest in this project and for his long and patient hours in helping me. Of particular note is that MSU may be using some of the QTVR movies we created on the next version of the university homepage!

by Matt MacQueen, August 6, 1996

up to Contents

Comments? Email: macqueen@pilot.msu.edu