Lapas attēli
PDF
ePub

and almost as easy to use as conventional products. I think that we could make a substantial improvement to understanding the complex world around us by encouraging the use of the appropriate parts of this technology. (Subsequent to this symposium, Starks wrote to tell me more about the Goldstar prototype TV, which he claims is soon to appear as a consumer item. It has line doubling, edge enhancement, and some other simple tricks to improve image quality. Unlike previous line doublers, this works on a field-by-field basis, so it is 3D compatible. Starks claims that in 2D it was nearly the equal of HDTV and in 3D it was stunning, especially since it is supposed to be available for just a few hundred dollars more than a standard set and is 100% compatible with all existing video hardware and software. Starks feels that the effective bandwidth of the monitor is about 40 MHz.)

In Japan, work in stereo and 3D imaging spans the same broad subfields as in the West except that there is a decided difference in emphasis. (At the symposium, Starks (3D TV) noted that most of the relevant information is not given either as journal or conference papers but rather in patent applications. In his paper he presents over 500 cited patent references to substantiate this point. Luckily, the Japanese patent system is the world's first to be computerized and researchers can now access this information from anywhere in the world.) In Japan, there is much more research than in the United States on the study of stereo and 3D in concert with high definition TV and somewhat more than in the EC. This is natural as both Japan and the EC have HDTV products that are either on the market or nearly so. What is somewhat surprising, though, is that the Japanese are also much more active in research concerning autostereoscopic and glassless imaging, and especially research connected with lenticular sheets. A main

motivation seems to be related to video conferencing, where it is felt that having all the participants view each other with glasses would be very unnatural.

The basic idea in this approach is also not new. An object is photographed with two cameras corresponding to left and right eye. Then images are displayed on a sequence of narrow vertical stripes, left eye image, right eye image, left, right, etc., a corduroy or interdigitated display. These days flat panel display devices are often used for the displays. Immediately in front is an array of half cylindrical lenses roughly matching the pitch of the display. Out in front of all this sits the observer, who can have an authentic stereo image if he/she is positioned in exactly the right place. As mentioned above, little head movement is possible and the observer must be seated at exactly the right position. Multiple observers can view this kind of display at the same time, although each observer must be correctly positioned. Also it is possible for observers to get a pseudostereoscopic image (right image to left eye, and left to right).

It is also possible to interdigitate more than two images, and this reduces the observer positioning problem, although it also decreases the resolution capability of the display devices, a problem with pixel-based displays, less so with fully optical displays. NTT incorporates an "eye-tracker" to follow the observer and interchanges the views from the lenses if a pseudostereoscopic view would be observed, but this makes it more difficult to use with more than one observer. At the University of Tokyo, Prof. Hamasaki's group has been working with images based on two Fresnel lenses and 24 prisms in order to make large color photographs. Care is required to produce these, but in the examples I saw, the stereo effect is exceptionally realistic and the resolution is as good as in normal color television. Viewers can move their head about 10 inches

left-right allowing a great deal of look around, and viewing distance can vary over several meters.

Hamasaki, the symposium director, has been working in the area of 3D imaging for almost 30 years and, in fact, will be retiring from his current University of Tokyo position in March 1992. In addition to the photographic project mentioned above, his laboratory is filled with other interesting experiments associated with autostereoscopic imaging. One of the most careful involves a

multiplano lenticular display that interdigitates eight views behind a lenticule, using an array of eight solid-state sensor cameras mounted on a base and viewing through a large Fresnel-type optic into a CRT. Using eight (rather than two) cameras increases the viewing zone, allows the observer's location to be less critical, and provides some look around ability. But the image is still limited in the number of observers that can view it. Hamasaki is hoping to increase the size of the system to 20 cameras. There is a continuing resolution problem because the flat LCD requires one pixel red-green-blue (RGB) set for each camera. There are also promising experiments on projection systems using arrays of LC panels. Considering the variety of projects, the laboratory is doing very impressive research. Symposium participants felt that there were few places in the world engaged in comparable work; however, one observed that the group needs at least five times the resources available to it in order to make timely progress.

The Heinrich-Hertz Institute in Berlin is also doing very advanced work in the design of lenticular screens and applications to glassless 3D systems. R. Borner presented a summary of these projects, which have been going on for more than 7 years. They are currently experimenting with an 18-channel front projector and hope to increase this to 24 channels, and they are also working on two rear projection systems, one

with a 640x480 pixel LC overhead projection panel in 8 grayscales and the other with an 18-inch direct view electroluminescent (EL) display with 16 grayscales and 1024x864 pixel resolution. To the best of my knowledge, the Borner group's work designing screens with varying pitch corrections is the most advanced anywhere.

Several of the U.S. participants at the symposium were surprised at the amount of work going on in Japan in the area of glassless autostereoscopsis. (Except for Benton's holographic work at MIT, there was not much reported from the United States on this topic.) All researchers would like to dispense with glasses, but most believe that practical systems will require them for the remainder of this decade. At the moment, the main problem with practical autostereoscopic systems is that the viewing position can be critical. One American remarked to me that he didn't understand why there was so much interest as there seemed to be major technical problems and, in fact, there might even be a wall that could not be breached. Another commented that he found the difficulties encountered when moving from viewing lobe to viewing lobe (i.e., head movement) in glassless lenticular systems to be far more problematic than properly presented glasses approaches. A third said that he saw no Japanese systems that were anywhere near being productizable and some that were much more than a decade away.

Two Westerners expressed serious concern that many of the scientists they met from Japanese companies (especially the younger ones) had huge gaps in their knowledge of what had already been done, and that a number of things were being worked on from scratch again, albeit with new materials. For example, one attendee commented to the authors that the NTT eye-tracker (above) had been done at HewlettPackard 10 years ago.

I was surprised to hear this criticism, as one of the only basic books on this topic was translated into English in 1976 from the Japanese edition (1972) and is widely known here.

Three Dimensional Imaging
Techniques

by Takanori Okoshi
Academic Press 1976
(Japanese book: Sanjigen Gazo
Kogaku, 1972)

My own view is somewhat different. The problem of stereo or 3D imaging is old enough that many fundamental ideas old enough that many fundamental ideas have already been proposed. Some of these may have failed in the past because the implementation technology was not up to the demands placed upon it. But it may be appropriate to look more carefully again. Good examples are recent research prototypes by NHK, one of a 50-inch autostereoscopic 3D display using a 1.5 Mx3 pixel projector (1440x1024 resolution) and an air-filled lenticular screen and the other a 9-inch lenticular screen and the other a 9-inch 3D electroluminescent display. Also, Hamasaki's photographic work clearly indicates that the viewer positioning problem can be solved, or very significantly reduced. Even in the case of restricted viewer position, there are obvious applications, such as sitting in front of a computer monitor looking at the image of a molecule. Also, the interest in video conferencing is very great est in video conferencing is very great in Japan. As one participant put it, you can have a successful company selling shutter glasses, or you can make real money designing video conferencing for businesses. Finally, whoever said for businesses. Finally, whoever said that the Japanese were daunted by 10-year time frames?

At least one speaker H. Tilton (Visonics) disagreed with the claim that 3D HDTV was far away. He felt that a very modest increase in HDTV channel space compared to 2D transmission was all that was needed, and that if an appropriate standard was adopted a

wide variety of receiver designs as well as compatibility with 2D systems could be achieved. Tilton's idea is to transmit one 2D HDTV signal, and another signal describing depth, which he claims can be done with a bandwidth of about one-fourth that of the luminance signal. However, it was clear that most attendees felt that bandwidth was a serious problem and that related work on image compression was very important.

Several interesting symposium sessions were devoted to applications of 3D to other disciplines. These included analyzing electron micrographs of GaP_{1-x} for dislocations, medical endoscopy, nondestructive investigation of animals' internal organs, etc. There was a paper by Sony researchers on an almost commercial stereolithography system (largest size, 1 meter), although frankly this only had a very weak connection with other aspects of this symposium. One of the most impressive medical applications was from researchers at Chiba University Hospital, whose goal is to provide a tool "with which many doctors can, with their own eyes, simultaneously and clearly observe a 3D diagnostic image of an object organ from any direction which is selectable and controllable by the doctor's hand." For some years the stereoviewer has been available for use with angiographic images, but these cannot be viewed by several doctors simultaneously. Similarly holographic images are sometimes used, with all their current difficulties. Of course, 2D displays of synthesized computed tomography (CT) and magnetic resonance imaging (MRI) slices are also widely used. The Chiba experiment uses x-ray images recorded on a video tape or 8-mm camera. Images are generated via a total circumference radiograph system (circa 1982) that can take 240 images in 8 seconds over 360°, i.e., with a 1.5° pitch. The current system takes these images, enhances them, and makes them available 12 at a time in a frame

buffer, which then displays them via a CRT with lenticular screen with 1 mm pitch and 12 dots per pitch (corresponding to 18 degrees of rotation). The authors are hoping to improve the system in the future, but they note that a major difficulty is that there are no currently available flat lenticular glass sheets installed within the glass envelope of a CRT and that they are looking into using LCDs.

One of the most obvious applications of 3D is to remote sensing, photogrammetry, and geographic information systems. S. Murai gave an exciting paper of what can be done, automatically, from digital elevation models (DEMs). DEMs can be obtained from stereo aerial photography, stereo space imagery, and rasterized contour maps. Each of these requires different technology and Murai listed various projects that groups in Japan have been working on. For example, image matching (which is needed for aerial and space images) is being studied by the Japan Society of Photogrammetry and Remote Sensing, a research committee set up by MITI, the Japan Highway Corporation, the Geographic Survey Institute, etc. The key to progress in this area is how to automate the digital data. While not discussing any details of this work, Murai gave several fascinating practical examples including DEMs generated automatically by image matching of stereo aerial photos and then used for a highway plan, 3D drainage maps generated automatically, a bird's eye view of a shaded hillside in Bhutan obtained by processing an input contour map, a 25-second 3D animation "flying" along a fault line in Shikoku Island and, finally, a "fly-by" of Mount Fuji. I think that everyone in the audience was very impressed with the fact that this could be done essentially automatically, as well as with the information content of the images.

As part of the Symposium on 3D Imaging Technology and Arts, the organizers arranged for foreign participants to visit four laboratories in the Tokyo area to see representative work in this field. Visits were short and focused. The description below is meant only to give a rough sense of what we saw and put it in perspective.

NHK Science and Technical Research Laboratories

This laboratory celebrated its 60th anniversary in 1990. Three hundred twenty people (270 research staff) are working in three general areas related to broadcast engineering. The annual budget is about ¥7.7B ($62M), approximately 1.3% of NHK's total budget. General research thrusts are as follows.

When I wrote about virtual reality TOURS last year (see reference above), I mentioned work by S. Tachi (University of Tokyo). Prof. Tachi showed a very impressive movie at the symposium based on research done over more than 10 years in Japan. His work has focused on developing anthropomorphic robots that will work in synchrony with a human (so-called master-slave system). The operator, using a helmet and data glove, moves around and the remote robot duplicates his movements. (This robot duplicates his movements. (This concept is not new, but Tachi's work concept is not new, but Tachi's work seems to be one of the best implementations.) The weight of the visual display is carried by a link mechanism with six degrees of freedom, which cancels gravitational force by use of a counter balance mechanism. The master arm has 10 degrees of freedom; 7 degrees of freedom are allocated for the arm and an additional three are used to comply with the body movement. The operator has input to his goggles not only from TV, etc., mounted on the robot, but also from integrated computerized displays that allow him to "see" what the robot is seeing even if the robot's vision is obscured, for example, by smoke. Thus, this system assumes a known environment to work in. We watched a demo of the robot lifting eggs from a basket as well as turning off a valve in a smoking room. The most interesting aspect of this demonstration was that Tachi assured the audience that the operator needed no training to use the system because it operated very naturally. He put it more dramatically: operators already come to the system pretrained by living.

The "art" portion of the symposium was provided by J. Gjessing (Norway), who demonstrated various aspects of his anaglyphic slide work. Conference attendees also had the opportunity to learn about "virtual drugs" at an evening session. I don't think that any of the Westerners attended.

• Putting new broadcast media into practice. In August 1990 the third Japanese broadcast satellite was launched, BS-3. HDTV is being transmitted from this 8 hours each day as well as some data-broadcasting. NHK has been working on HDTV since 1964 and demonstrated a stereo HDTV system in 1989. We were told that there is no plan yet for a commercial product, but one is expected soon. Broadcasting equipment, transmission systems, and home receivers are studied at the laboratory. Currently it is possible to purchase for home use a 50-inch HDTV monitor for about $25K (in Japan this is called High Vision TV) that will allow viewing of prerecorded tapes from a VCR or viewing of decoded signals from the satellite. However, the satellite signals need to be decoded. At the moment this requires another box (MUSE decoder) about the size of a VCR, about $10K. We were told that an

implementation via large scale integration (LSI) was in progress. The laboratory is also working on a 33-inch, wall-mounted, plasma discharge panel (6-mm-thick) HDTV display and is heavily involved in studio and international standards.

• Improved conventional broadcast services. Better reception, noise and interference reduction research, such as optical cable TV. Work on machine translation (as well as some speech and pattern recognition), which is already in use for making subtitles for news programs. New cameras for use under a very wide range of light (sunlight to starlight) and wireless (45-GHz) broadcast cameras. Experimentation with synthesizing HDTV and real-time images using "chromakey" (see also discussion of Sony below). There is also a very interesting experiment with digital broadcasting that allows use of 35- to 40-cm-diameter antennas for nearly all of Japan reception and even mobile reception.

Research into future broadcast systems. Materials for signal processing, recording, display, and LSI are studied. Also ISDB, Integrated Service Digital Broadcasting, the satellite equivalent of ISDN, which will allow broadcasting of TV, audio, fax, software (data), telemusic, etc., as well as 3D video and audio are major projects.

We saw several examples on enhanced reality work at NHK. One was a wall of 160 small speakers that could be driven at appropriate phases so that the sound appeared to emanate from a point several meters in front of the wall. NHK scientists told us that they could also localize (in space) different kinds of sounds, such as different instruments. This type of research has been done in other places. The

difficult part is how to encode all the
different phase information on a tape
or disk and then distribute it to the
speakers properly. We did not have any
opportunity to discuss this aspect. We
were given a demonstration of field
sequential stereo HDTV based on film
(not computer generated). In principle
this is fairly direct, but the implemen-
tation details are difficult and NHK
pulled it off very well. Everyone was
impressed with the visual effect; bright- Matsushita Electric Works
ness, clarity, and depth were excellent.
In fact R. Kroiter, who is one of the
founders of Imax, commented that the
apparent resolution was even greater
than what he normally expected even
from HDTV. (See also the remark by
from HDTV. (See also the remark by
Starks above about information con-
tent in stereo images.) We were shown
a demonstration of the autostereoscopic
system based on a 50-inch lenticular
projection system. The resolution is
not as good as the HDTV system, and
not as good as the HDTV system, and
viewers had to be carefully seated, but
no glasses were needed and the image
was amply bright and clear.

The final tour stop was a standards laboratory, where a set of "standard" HDTV still pictures was being investigated for various characteristics such as color, sharpness, etc. Standard moving images have not yet been selected. Our host at NHK, who also presented a paper at the symposium summarizing the company's work, was Dr. Hideo Kasaka.

This virtual reality exhibit was one small part of a large public exhibit area where many of the company's consumer products were on display including such low-tech ones as light bulbs. The VR system incorporated a conventional head-tracker, goggles, data glove, and stereo headphones. As usual with these systems, the LCD resolution in the goggles was not as good as one would like, although it was about as good as one normally sees. Also, several of the participants complained that the stereo video effects could be done better and that the attendant who moved around their hand was too intrusive. Nevertheless, the most interesting part of this demonstration was that it was included as part of a kitchen planning system. The user is placed inside a kitchen that he/she can walk through and, by moving the data glove and "grasping," can open cabinets, turn on faucets, radio, etc. Carping from the experts aside, I thought that it was a good demonstration and will be effective as a sales tool. Matsushita claims that it is the first commercial application of VR.

One of the most interesting demon-
strations (and the furthest into the
future) was by Mahito Fujii in the Visual
future) was by Mahito Fujii in the Visual
Science Research Department. Fujii's
work is directed toward developing a
fully 3D image beginning with only two
or three static 2D TV pictures. The
resulting 3D image can be rotated (a
little). In other words, occluded infor-
mation is interpolated as needed. His
approach is to use the fact that edges
probably have the same orientation in
the two images. He has built a four-
layer neural network to extract binocu-
lar disparity by the use of edge infor-
mation and the model can specify relative Sony Media World
positions in 3D space that can then be
interpolated to form textured surfaces.
Fujii admitted that this was still in the
very early stages; his neural model runs
on NHK's Convex, much slower than
real time. (I was told that there is also
related work at the Heinrich-Hertz
Institute in Germany.)

We spent one afternoon at Sony's public display area in which a large number of its fanciest products and prototypes were on exhibit. This included a multipurpose teleconferencing room, HDTV displays (both 2D and field

sequential), and an extremely sophisticated intelligent TV studio where just about every digital processing technique was available from within a "videocockpit" to control production, editing, and transmission. This included chromakey, converting 2D images to 3D, and others. In the stereo TV demonstration we were shown two films composed of still photos of Japanese museum pieces and a third (much more effective) showing a training film for ophthalmic surgery. The latter revealed just how helpful the sensation of depth can be in transmitting information. In both this demo and in NHK's, it was pointed out just how difficult the photographing of multiple images can be and how much care must be exercised during all steps of the process. Sony's exhibits, like Matsushita's, covered much more than we were able to see during one afternoon.

« iepriekšējāTurpināt »