Can you tell which one of the two is following you and which one is not?
Picture perception is ubiquitous in one's daily life. Picture perception is not limited to appreciating a piece of painting hanging in a museum or looking at a picture on a magazine; instead, it also encompasses watching a movie or talking to your loved ones or coworkers through online video-mediated telecommunication software. In this context, picture perception essentially entails perceiving the depicted 3D structure of a scene or a figure or an object on a flat surface. The ubiquity of picture does not mean that people necessarily "enjoy" it. For instance, during the COVID-19 pandemic, remote working and constant "Zooming" with your coworkers have people complaining about "Zoom fatigue", where video-based conference tends to be more attentionally taxing and tiring than a face-to-face meeting. This project aims to examine the perceptual factors that distinguish visual perception in one's immediate surrounding and that through a 2D surface.
To understand their differences, one has to appreciate how pictures are created. For our purposes, you can think about a pinhole camera: The image of a scene is captured through a tiny aperture and the image that is captured reflect the location of that aperture relative to the scene that lies beyond the aperture. The captured image is geometrically correct, following the laws of linear perspective. When viewing the captured image, the viewer has to assume the location of the camera based on which the image was taken to obtain a geometrically correct view of the depicted scene. However, as one can imagine, this is rarely the case, as the viewer may be a little too close, or be a little too far to the side. This results in affine distortions, where the angular information is lost. The consequence of affine distortions is manifested in the Uncle Sam's recruitment poster or Mona Lisa - Uncle Sam is always pointing at you whereas Mona Lisa is always looking at you, regardless of where you are.
If assuming the position of the camera at which the picture was taken can connect you to the depicted 3D scene that the camera captures, then deviating from this position would break the connection. Given that it is almost impossible to even obtain the original position of the camera, there is an intrinsic disconnect between the space that you occupy and that is depicted in the picture. This disconnect could be the reason behind why video conferencing is inferior to face-to-face interactions, in terms of communication efficiency, rapport building, developing trust, and henceforth, since a lot of things that we may have taken for granted, such as direction and distance between the observer (you) and other people (your coworker on the screen) and objects, is no longer present between the two spaces.
This project uses the power offered by virtual reality (VR) to explore what can help to reconnect the two spaces. VR allows us to manipulate various types of visual information to simulate the situations where one is looking at a picture versus where one is looking at something in his/her surrounding. In addition, we are also able to control the availability of different types of visual information, in particular motion parallax and stereopsis, to examine the intermediate conditions between looking at a picture and looking in real life. Results from this study would inform us of the potential ways through which we can improve video-based telecommunication and offer its users a more immersive experience.
Wang, X.M., Thaler, A., Bebko, A.O., Troje, N. (in preparation). Relating pictorial and visual spaces with an exocentric pointing task: The roles of motion parallax and stereopsis.
Wang, X.M., Thaler, A., Bebko, A.O., Troje, N. (in preparation). Relating pictorial and visual spaces with an exocentric pointing task II: Motion parallax eliminates the affine distortions in the pictorial space.
Wang, X.M., Thaler, A., Bebko, A.O., Troje, N. (in preparation). Perceptual distortions of window and screens: Self-motion as a link between stereopsis and motion parallax.