Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia

Enhancing Operator Performance of Remote Container Landing: An Assessment of a 3D Stereoscopic Control & Display System

K. Y. Lim and Roy S. M. Quek

Centre for Human Factors Engineering,

Nanyang Technological University, School of MPE,

50, Nanyang Avenue, Singapore 639798

Email:   mkylim@ntu.edu.sg   royquek@cyberway.com.sg

Keywords: 2 dimensional display, 3D stereoscopic display, depth perception, remote control crane

Abstract

A freight company in Singapore has developed and implemented the world’s first semi-automated camera-based remote control crane system to pick-up and land freight containers. A centralised computer commands the crane to move a container between locations, while the operator performs the skilled task of vertically landing/picking up the container. However, the 2 dimensional (2D) camera-based system compromises somewhat the depth perception required for the container landing/pick-up task. For instance, the operator may experience difficulty in judging container height relative to the prime mover chassis. Their ability to control container landing/pick up quickly and thus, throughput, may be affected. Similarly, container landing impact may be controlled poorly. To address these problems, depth perception needs to be restored. A desk-top virtual reality system comprising a three-dimensional (3D) stereoscopic display, has thus been developed to meet this requirement. Subject tests with a scale model has shown that the 3D stereoscopic display can enhance operator performance of container landing impact (P<0.01). For the small scale test rig, no significant difference in container landing time performance is found between the displays as expected. Thus, it may be concluded that a 3D stereoscopic display has shown promise in enhancing operator performance of container landing with apparently no loss in throughput.

1.      Background

Commercial industries have long realised the potential of automated operation as a means to enhance efficiency and productivity. For this reason, a freight company in Singapore has invested US$ 4 billion to develop and implement the world's first remote crane control system. This semi-automated system incorporates a centralised computer which removes from the operator the task of positioning a crane or container at a designated location. The operator now performs only the skilled component of manipulating the spreader to land and hoist/pick up a container. This new 2-dimensional (2D) camera-based system improves the working conditions of operators by removing them from being physically in the crane. Thus, work-stress related problems caused by poor working posture and vibration are avoided (see Figure 1). In contrast, the remote operators now work in a comfortable seated position in a centralised air-conditioned control room (see Figure 2).

Currently, to enable remote operation, four cameras are mounted directly above the spreader (the container grasper) to relay to the operator, a view of four corners of a container (see Figure 3). A composite of these four corners of a container is displayed in 2D, termed a quad display view.

Previous studies by Ng et al (2000) have compared the efficacy of various 2D display formats, for enhancing remote task performance of lateral positioning and vertical landing of a container. 2.5D perspective displays have also been assessed. The results reveal that a 2D view of two corners of a container could enable the same level of performance as a 2D quad view display. However, to date, the 2D quad-view display continues to be used commercially as it is already implemented.

Although the 2D quad view display is useful for positioning a container directly over a prime mover, it deprives the operator of physiological depth perception cues required for vertical container pick-up and landing. As such, the operator may find it difficult to judge the height of the container to either the prime mover chassis (container dispatch operation), or to another container (container storage/stacking operation). Therefore, the operator may experience difficulty in controlling the speed and impact of container landing. In this respect, it should be noted that the operator needs to monitor container height and speed, to maximise throughput and avoid damaging either the contents in the container or the prime mover chassis, that may arise from an overly heavy container landing impact.

To address the above concerns, a desktop stereoscopic 3D display system is developed to determine its potential for restoring depth perception cues to a remote operator. Although a 3D stereoscopic display may provide some physiological depth perception cues necessary for the container-landing task, it is unclear to what extent, such a display could enhance operator performance over the existing 2D display system. This is because both displays would provide psychological depth cues such as relative size, motion parallax, shadow and occlusion. Further, it is also uncertain to what extent operators could adapt to and compensate for the shortcomings of a 3D display (such as potential motion sickness). Previous studies by Pang et al (2000) reveal that a 3D stereoscopic display seems promising. However, the primitive test rig that is used for these studies lacks fidelity when compared with the commercially operated remote control system. As such, the results have to be further substantiated.

To address these questions and concerns, a more advanced remote control test rig is constructed. The objective of the study is to evaluate the efficacy or otherwise, of using a 3D stereoscopic display system to restore stereoscopic depth perception to the operator. In this regard, system efficacy is assessed in terms of operator performance of vertical container landing, measured in terms of landing time and impact. An account of the test rig and experimental study follows.

2.      Construction of a Scale Model of the Remote Crane Control System

The remote control test rig is shown in Figure 4. The test rig includes a container whose position and height are controlled remotely via a 4-channel digital proportional radio controller. The trolley from which the container is hung, can travel in the y-axis (front & back) and x-axis (left & right). Cameras are mounted onto the test rig to provide subjects a quad view display (Figure 3) and a 3D stereoscopic display (Figure 5).

The stereoscopic display system simulates how the human eye perceives depth, by relaying slightly different images from two CCD video cameras placed 6.5cm apart (corresponding approximately to the human inter-pupillary distance) to a stereo-decoder. The latter projects both images which are slightly displaced onto a monitor, where the left and right images are synchronously time-multiplexed. The stereo-decoder then polarises light from the display alternately, by using a light polariser screen or a filter that slots over a monitor such as the Stereographics Z-Screen (see Figure 5). The image targeted at the left eye is polarised in the horizontal direction, while the image for the right eye is polarised in the perpendicular direction. For stereoscopic viewing, an observer needs to wear spectacles with opposing polarised lenses. When the images are viewed through the lenses, only light polarised in the direction of the filter can pass through.  The result is different images seen by each eye. The images are transmitted to the brain via optic nerves, which come together at the optic chiasma near the brain (Pedrotti and Pedrotti, 1998). The two images are then integrated into a single image in a process referred to as binocular vision. The fusion of images and convergence of the eyeballs, are interpreted by the brain as depth information (Schiffman, 1996). In respect of the equipment for generating 3D stereoscopic vision, it should be noted that they are also selected with the requirements of the commercial operation context in mind. In particular, the circular polarizing spectacles that are selected are light and comfortable (no battery requirements), and would not constrain the crane operator in any way. They can thus be worn for an entire working day. For the same reason, the display system is targeted at a desk top monitor.

3.      Experiment Design

The main purpose of the experiment is to compare subject performance of the container landing task using 2- and 3-D displays. In the study, only the container landing task is examined, since in the commercial remote control system, the operator is only required to land and pick up containers (the computer handles all other operations). As the study aims to compare the relative efficacy of a 2- and 3-D display system, confounding cues are removed. Specifically, shadow is removed using lamps located strategically, and sound from the motor drive is silenced using earmuffs (noise reduction rating of 31 decibels).

48 subjects (equal number of female and male university students) are recruited for the study. Crane operators need not be used in this case, as the study is only concerned with comparing relative performance. In other words, test validity concerns pertaining to subjects would not arise as the quantification of absolute container landing performance, is not presently of interest. The subjects are all required to perform the container landing task by direct viewing of the container, and indirectly using a mediated view provided by the 2D quad view display and the 3D stereoscopic display. The direct viewing test condition provides an approximation of onsite task performance, and thus serves as the best case reference for comparison. To ensure reliability, each subject is required to perform the task 10 times. An average value for their performance is then taken. The order of subject tests with respect to the type of display being used, is also balanced to account for sequence effects.

The subjects are instructed to land, as fast and as lightly as they could, a red container weighing 2.1kg (20.8N) onto a target area that is marked out on a force platform. Landing impact (corresponding to the spike observed on the step graph profile) and time data for each landing (at point of impact) are recorded by a computer using Bioware, a software that comes with the force platform. A force-time graph is plotted by the software. In all, a total of 1,920 graphs are obtained in the study. Figures 6 and 7 show force-time graphs that indicate contrasting force of landing impact as performed by the subjects. Figure 6 shows a low impact landing, while Figure 7 shows a high impact landing (indicated by the long spike).

The data extracted from these force-time graphs, are then subjected to statistical analysis using a one-way repeated measures ANOVA.

4.      Results and Discussion

Table 1 shows the ANOVA results for container landing impact. The results reveal that the type of display is a significant factor (P<0.01) affecting subject performance of the vertical container landing task. To ascertain the source of performance differences, a Tukey test is performed next. The results in Table 2 reveal that subject performance across all the displays are significantly different. Using a stereoscopic display to support the container landing task is found not to be equivalent to the support that is provided by direct viewing. The results indicate clearly that onsite operations are most effective due perhaps to the most natural and comprehensive availability of both psychological and physiological depth perception cues. Although shadow and auditory cues have been removed for the tests, other psychological depth perception cues such as occlusion and perspective, are still available in the direct viewing test condition but are not as rich in the 3D stereoscopic display test condition. The better performance achieved by subjects using direct viewing over a 3D stereoscopic display, may be due to these depth perception cues. Nevertheless, a 3D stereoscopic display does enable subjects to perform better than with a 2D quad view display. Specifically, direct viewing enables subjects to achieve the lowest impact landings (17.9N), followed by a 3D stereoscopic display (23.6N) and then a 2D quad-view (29.5N).

In summary, the results for container landing impact indicate that depth information provided by a 3D stereoscopic display and direct viewing, enabled subjects to gauge better the container height and thus control its landing velocity more appropriately to achieve a lower impact landing. Conversely, the 2D quad-view display provides the least depth perception cues, and the subjects are thus less able to gauge the container height and speed parameters required for a low impact landing.

Table 1. ANOVA table for container landing performance (impact force)

Table 2. Tukey test for container landing performance (impact force)

Table 3 shows the ANOVA results for subject performance in terms of container landing time. The results show that the type of display that is used is a significant factor affecting subject performance of container landing (P<0.01). As before, to uncover the source of performance differences, a Tukey test is performed. Table 4 shows the results of the Tukey test. The results reveal that subject performance of container landing using both displays is significantly different from direct viewing performance (2D quad-view display at P<0.05 and 3D stereoscopic display at P<0.01). For the 2D quad-view display, the significance is marginal. Thus, remote container landing performance is slower as expected, due to less effective depth perception cues derivable from an intervening display (as opposed to being onsite). The Tukey test results also show that there is no significant difference in subject performance across 2D quad-view and 3D stereoscopic displays. In summary, the study reveals that direct viewing enables subjects to achieve the fastest landing time (7.1 s) followed by the 2D quad-view display (7.4 s), and finally the 3D stereoscopic display (7.7 s).

Taken together, the two sets of studies have confirmed that a 3D stereoscopic display can enhance remote operation performance expressed in terms of container landing impact. The performance enhancement is also achieved without incurring a loss in throughput as compared with the existing 2D quad-view display.

Table 3. ANOVA table for container landing performance (time)

Table 4. Tukey test for container landing performance (time)

5.      Conclusion

A scale model remote control test rig has been built to enable a comparison of the efficacy of 2D quad-view and 3D stereoscopic displays, for supporting the task of vertically landing a container onto a target platform. As reference, subject performance is compared with direct viewing which serves as an approximation of onsite operation. The objective of the study is to determine whether a 3D stereoscopic display has promise for restoring some physiological depth perception cues to a remote operator. These cues are lost with the current use of a 2D quad-view display for remote container handling. To avoid introducing confounding variables into the tests, shadow and sound cues are removed from the test set up. In the tests, subject performance of container landing is assessed in terms of landing impact and time.

ANOVA results from the test rig experiments indicate that remote operation using either of the two displays, would remain inferior to onsite task performance. Thus, it is concluded that the depth perception cues that are presented by both the 2D quad view and 3D stereoscopic display, can not match either/both the quality and/or quantity of depth perception cues available onsite. However, the results reveal that subjects do perform significantly better at vertically landing a container with the least impact force (P<0.01), when they work with a 3D stereoscopic display. With the latter display, subjects are better able to gauge the container height and control its speed to achieve a lighter impact container landing. The time taken to land a container is not affected significantly, when compared with the performance that is achieved using a 2D quad-view display. Thus, it is concluded that a 3D stereoscopic display has promise for enhancing operator performance of container landing tasks, without incurring a loss in throughput. However, the latter result needs to be verified in a large scale with an actual remote control crane and at the operation height of 26m. Further, the efficacy of providing the operator with wider depth perception cues such as shadow and sound which are available onsite, should be investigated. If these cues are found to be promising, consideration may then be given for their integration with the present 3D stereoscopic display system. As a follow up study is ongoing, the results will be reported at a later date.

6.      References

·         Ng M.C. and Lim K.Y., (2000). An Assessment of Various Two-Dimensional Display Designs for a Camera Based Remote Freight Handling System, In Lim K.Y. et al, Proceedings of the Joint Conference of APCHI 2000 & ASEAN Ergonomics 2000, Singapore, Elsevier Science, pp. 68-74.

·         Pang T.K., Lim K.Y. and Quek S.M. Roy, (2000). Design and development of a Stereoscopic Display Rig for a Comparative Assessment of Remote of Freight Handling Performance, In Lim K.Y. et al, Proceedings of the Joint Conference of APCHI 2000 & ASEAN Ergonomics 2000, Singapore,  Elsevier Science, pp. 62-67.

·         Pedrotti L.S. and Pedrotti F.L., (1998). Optics and Vision, Prentice Hall International Inc, New Jersey, USA, pp. 194-223.

·         Schiffman H.R., (1996). Sensation and Perception. An Integrated Approach, 4^th edition, John Wiley & Sons, Inc, England, pp. 215-245.