Virtual reality has long been used for training simulations in fields from medicine to welding to vehicular operation, but simulations involving more complex cognitive skills present new design challenges. Foreign language learning, for example, is increasingly vital in the global economy, but computer-assisted education is still in its early stages. Immersive virtual reality is a promising avenue for language learning as a way of dynamically creating believable scenes for conversational training and role-play simulation. Visual immersion alone, however, only provides a starting point. We suggest that the addition of social interactions and motivated engagement through narrative gameplay can lead to truly effective language learning in virtual environments. In this paper, we describe the development of a novel application for teaching Mandarin using CAVE-like VR, physical props, human actors and intelligent virtual agents, all within a semester-long multiplayer mystery game. Students travel (virtually) to China on a class field trip, which soon becomes complicated with intrigue and mystery surrounding the lost manuscript of an early Chinese literary classic. Virtual reality environments such as the Forbidden City and a Beijing teahouse provide the setting for learning language, cultural traditions, and social customs, as well as the discovery of clues through conversation in Mandarin with characters in the game.

This research is aimed at examining the added value of using Virtual Reality (VR) in a driving simulator to prevent road accidents, specifically by improving drivers' skills when confronted with extreme situations. In an experiment, subjects completed a driving scenario using two platforms: A 3-D Virtual Reality display system using an HMD (Head-Mounted Display), and a standard computerized display system based on a standard computer monitor. The results show that the average rate of errors (deviating from the driving path) in a VR environment is significantly lower than in the standard one. In addition, there was no compensation between speed and accuracy in completing the driving mission. On the contrary: The average speed was even slightly faster in the VR simulation than in the standard environment. Thus, generally, despite the lower rate of deviation in VR setting, it is not achieved by driving slower. When the subjects were asked about their personal experiences from the training session, most of the subjects responded that among other things, the VR session caused them to feel a higher sense of commitment to the task and their performance. Some of them even stated that the VR session gave them a real sensation of driving.

Motion parallax is a crucial visual cue produced by translations of the observer for the perception of depth and selfmotion. Therefore, tracking the observer viewpoint has become inevitable in immersive virtual (VR) reality systems (cylindrical screens, CAVE, head mounted displays) used e.g. in automotive industry (style reviews, architecture design, ergonomics studies) or in scientific studies of visual perception. The perception of a stable and rigid world requires that this visual cue be coherent with other extra-retinal (e.g. vestibular, kinesthetic) cues signaling ego-motion. Although world stability is never questioned in real world, rendering head coupled viewpoint in VR can lead to the perception of an illusory perception of unstable environments, unless a non-unity scale factor is applied on recorded head movements. Besides, cylindrical screens are usually used with static observers due to image distortions when rendering image for viewpoints different from a sweet spot. We developed a technique to compensate in real-time these non-linear visual distortions, in an industrial VR setup, based on a cylindrical screen projection system. Additionally, to evaluate the amount of discrepancies tolerated without perceptual distortions between visual and extraretinal cues, a "motion parallax gain" between the velocity of the observer's head and that of the virtual camera was introduced in this system. The influence of this artificial gain was measured on the gait stability of free-standing participants. Results indicate that, below unity, gains significantly alter postural control. Conversely, the influence of higher gains remains limited, suggesting a certain tolerance of observers to these conditions. Parallax gain amplification is therefore proposed as a possible solution to provide a wider exploration of space to users of immersive virtual reality systems.

Meta!Blast (http://www.metablast.org) is designed to address the challenges students often encounter in understanding cell and metabolic biology. Developed by faculty and students in biology, biochemistry, computer science, game design, pedagogy, art and story, Meta!Blast is being created using Maya (http://usa.autodesk.com/maya/) and the Unity 3D (http://unity3d.com/) game engine, for Macs and PCs in classrooms; it has also been exhibited in an immersive environment. Here, we describe the pipeline from protein structural data and holographic information to art to the threedimensional (3D) environment to the game engine, by which we provide a publicly-available interactive 3D cellular world that mimics a photosynthetic plant cell.

This paper describes the teaching of an immersive environments class on the Spring of 2011. The class had students from undergraduate as well as graduate art related majors. Their digital background and interests were also diverse. These variables were channeled as different approaches throughout the semester. Class components included fundamentals of stereoscopic computer graphics to explore spatial depth, 3D modeling and skeleton animation to in turn explore presence, exposure to formats like a stereo projection wall and dome environments to compare field of view across devices, and finally, interaction and tracking to explore issues of embodiment. All these components were supported by theoretical readings discussed in class. Guest artists presented their work in Virtual Reality, Dome Environments and other immersive formats. Museum professionals also introduced students to space science visualizations, which utilize immersive formats. Here I present the assignments and their outcome, together with insights as to how the creation of immersive environments can be learned through constraints that expose students to situations of embodied cognition.

Virtual Reality was a technological wonder in its early days, and it was widely held to be a domain where men were the main practitioners. However, a survey done in 2007 of VR Artworks (Immersive Virtual Environments or VEs) showed that women have actually created the majority of artistic immersive works. This argues against the popular idea that the field has been totally dominated by men. While men have made great contributions in advancing the field, especially technologically, it appears most artistic works emerge from a decidedly feminine approach. Such an approach seems well suited to immersive environments as it incorporates aspects of inclusion, wholeness, and a blending of the body and the spirit. Female attention to holistic concerns fits the gestalt approach needed to create in a fully functional yet open-ended virtual world, which focuses not so much on producing a finished object (like a text or a sculpture) but rather on creating a possibility for becoming, like bringing a child into the world. Immersive VEs are not objective works of art to be hung on a wall and critiqued. They are vehicles for experience, vessels to live within for a piece of time.

We propose a novel markerless 3D facial motion capture system using only one common camera. This system is simple and easy to transfer facial expressions of a user's into virtual world. It has robustly tracking facial feature points associated with head movements. In addition, it estimates high accurate 3D points' locations. We designed novel approaches to the followings; Firstly, for precisely 3D head motion tracking, we applied 3D constraints using a 3D face model on conventional 2D feature points tracking approach, called Active Appearance Model (AAM). Secondly, for dealing with various expressions of a user's, we designed 2D face generic models from around 5000 images data and 3D shape data including symmetric and asymmetric facial expressions. Lastly, for accurately facial expression cloning, we invented a manifold space to successfully transfer 2D low dimensional feature points to 3D high dimensional points. The manifold space is defined by eleven facial expression bases.

We investigate the suitability of the Microsoft Kinect device for capturing real-world objects and places. Our new geometry scanning system permits the user to obtain detailed triangle models of non-moving objects with a tracked Kinect. The system generates a texture map for the triangle mesh using video frames from the Kinect's color camera and displays a continually-updated preview of the textured model in real-time, allowing the user to re-scan the scene from any direction to fill holes or increase the texture resolution. We also present filtering methods to maintain a high-quality model of reasonable size by removing overlapping or low-precision range scans. Our approach works well in the presence of degenerate geometry or when closing loops about the scanned subject. We demonstrate the ability of our system to acquire 3D models at human scale with a prototype implementation in the StarCAVE, a virtual reality environment at the University of California, San Diego. We designed the capturing algorithm to support the scanning of large areas, provided that accurate tracking is available.

In this paper, we propose a new face relighting algorithm powered by a large database of face images captured under various known lighting conditions (a Multi-PIE database). Key insight of our algorithm is that a face can be represented by an assemble of patches from many other faces. The algorithm finds the most similar face patches in the database in terms of the lighting and the appearance. By assembling the matched patches, we can visualize the input face under various lighting conditions. Unlike existing face relighting algorithms, we neither use any kinds of face model nor make a physical assumption. Instead, our algorithm is a data-driven approach, synthesizing the appearance of the image patch using the appearance of the example patch. Using a data-driven approach, we can account for various intrinsic facial features including the non-Lambertian skin properties as well as the hair. Also, our algorithm is insensitive to the face misalignment. We demonstrate the performance of our algorithm by face relighting and face recognition experiments. Especially, the synthesized results show that the proposed algorithm can successfully handle various intrinsic features of an input face. Also, from the face recognition experiment, we show that our method is comparable to the most recent face relighting work.

This text will examine how avatars and the socially interactive, online virtual worlds in which they become embodied may lead to an understanding of identity and of self-perception, how such shifts in awareness may relate to the notion of the undividedly holistic 'self' and whether such perceptual shifts may be instrumental in bringing forth virtual states of experiential creative activity which may also have their precursors in the literary pseudonym, particularly as evidenced in Fernando Pessoa's conception of 'heteronyms.' The output of my study is a self-observational social system of my own creation, of which the agents are a coterie of avatars of both sexes, endowed with distinct physical attributes, both human and non-human; with uniquely emergent personalities which have progressed towards further idiosyncrasy over a period of three years. I, their creator am also the observer of their undertakings, their interactions, and their creative output, all of which manifest as disparate facets of my own persona and my artistic activity.

ManifestAR is an international artists' collective working with emergent forms of augmented reality as interventionist public art. The group sees this medium as a way of transforming public space and institutions by installing virtual objects, which respond to and overlay the configuration of located physical meaning. This paper will describe the ManifestAR vision, which is outlined in the groups manifesto.

Augmented reality is way of both altering the visible and revealing the invisible. It offers new opportunities for artistic exploration through virtual interventions in real space. In this paper, the author describes the implementation of two art installations using different AR technologies, one using optical marker tracking on mobile devices and one integrating stereoscopic projections into the physical environment. The first artwork, De Ondas y Abejas (The Waves and the Bees), is based on the widely publicized (but unproven) hypothesis of a link between cellphone radiation and the phenomenon of bee colony collapse disorder. Using an Android tablet, viewers search out small fiducial markers in the shape of electromagnetic waves hidden throughout the gallery, which reveal swarms of bees scattered on the floor. The piece also creates a generative soundscape based on electromagnetic fields. The second artwork, Urban Fauna, is a series of animations in which features of the urban landscape become plants and animals. Surveillance cameras become flocks of birds while miniature cellphone towers, lampposts, and telephone poles grow like small seedlings in time-lapse animation. The animations are presented as small stereoscopic projections, integrated into the physical space of the gallery. These two pieces explore the relationship between nature and technology through the visualization of invisible forces and hidden alternate realities.

Our contemporary imaginings of technological engagement with digital environments has transitioned from flying through Virtual Reality to mobile interactions with the physical world through personal media devices. Experiences technologically mediated through social interactivity within physical environments are now being preferenced over isolated environments such as CAVEs or HMDs. Examples of this trend can be seen in early tele-collaborative artworks which strove to use advanced networking to join multiple participants in shared virtual environments. Recent developments in mobile AR allow untethered access to such shared realities in places far removed from labs and home entertainment environments, and without the bulky and expensive technologies attached to our bodies that accompany most VR. This paper addresses the emerging trend favoring socially immersive artworks via mobile Augmented Reality rather than sensorially immersive Virtual Reality installations. With particular focus on AR as a mobile, locative technology, we will discuss how concepts of immersion and interactivity are evolving with this new medium. Immersion in context of mobile AR can be redefined to describe socially interactive experiences. Having distinctly different sensory, spatial and situational properties, mobile AR offers a new form for remixing elements from traditional virtual reality with physically based social experiences. This type of immersion offers a wide array of potential for mobile AR art forms. We are beginning to see examples of how artists can use mobile AR to create social immersive and interactive experiences.

As cities world-wide adopt and implement reforestation initiatives to plant millions of trees in urban areas, they are engaging in what is essentially a massive ecological and social experiment. Existing air-borne, space-borne, and fieldbased imaging and inventory mechanisms fail to provide key information on urban tree ecology that is crucial to informing management, policy, and supporting citizen initiatives for the planting and stewardship of trees. The shortcomings of the current approaches include: spatial and temporal resolution, poor vantage point, cost constraints and biological metric limitations. Collectively, this limits their effectiveness as real-time inventory and monitoring tools. Novel methods for imaging and monitoring the status of these emerging urban forests and encouraging their ongoing stewardship by the public are required to ensure their success. This art-science collaboration proposes to re-envision citizens' relationship with urban spaces by foregrounding urban trees in relation to local architectural features and simultaneously creating new methods for urban forest monitoring. We explore creating a shift from overhead imaging or field-based tree survey data acquisition methods to continuous, ongoing monitoring by citizen scientists as part of a mobile augmented reality experience. We consider the possibilities of this experience as a medium for interacting with and visualizing urban forestry data and for creating cultural engagement with urban ecology.

The proliferation of technological devices and artistic strategies has brought about an urgent and justifiable need to capture site-specific time-based virtual reality experiences. Interactive art experiences are specifically dependent on the orchestration of multiple sources including hardware, software, site-specific location, visitor inputs and 3D stereo and sensory interactions. Although a photograph or video may illustrate a particular component of the work, such as an illustration of the artwork or a sample of the sound, these only represent a fraction of the overall experience. This paper seeks to discuss documentation strategies that combine multiple approaches and capture the interactions between art projection, acting, stage design, sight movement, dialogue and audio design.

A study was conducted to examine the impact, in terms of cognitive demands, of a restricted field of view (FOV) on semi-natural locomotion in virtual reality (VR). Participants were divided into two groups: high-FOV and low-FOV. They were asked to perform basic movements using a locomotion interface while simultaneously performing one of two memory tasks (spatial or verbal) or no memory task. The memory tasks were intended to simulate the competing demands when a user has primary tasks to perform while using an unnatural interface to move through the virtual world. Results show that participants remembered fewer spatial or verbal items when performing locomotion movements with a low FOV than with a high FOV. This equivalent verbal and spatial detriment may indicate that locomotion movements with a restricted FOV require additional general cognitive resources as opposed to spatial or verbal resource pools. This also emphasizes the importance of this research, as users of a system may allow primary task performance to suffer when performing locomotion. Movement start and completion times were also measured to examine resource requirements of specific aspects of movements. Understanding specific performance problems resulting from concurrent tasks can inform the design of systems.

The United States military is increasingly pursuing advanced live, virtual, and constructive (LVC) training systems for reduced cost, greater training flexibility, and decreased training times. Combining the advantages of realistic training environments and virtual worlds, mixed reality LVC training systems can enable live and virtual trainee interaction as if co-located. However, LVC interaction in these systems often requires constructing immersive environments, developing hardware for live-virtual interaction, tracking in occluded environments, and an architecture that supports real-time transfer of entity information across many systems. This paper discusses a system that overcomes these challenges to empower LVC interaction in a reconfigurable, mixed reality environment. This system was developed and tested in an immersive, reconfigurable, and mixed reality LVC training system for the dismounted warfighter at ISU, known as the Veldt, to overcome LVC interaction challenges and as a test bed for cuttingedge technology to meet future U.S. Army battlefield requirements. Trainees interact physically in the Veldt and virtually through commercial and developed game engines. Evaluation involving military trained personnel found this system to be effective, immersive, and useful for developing the critical decision-making skills necessary for the battlefield. Procedural terrain modeling, model-matching database techniques, and a central communication server process all live and virtual entity data from system components to create a cohesive virtual world across all distributed simulators and game engines in real-time. This system achieves rare LVC interaction within multiple physical and virtual immersive environments for training in real-time across many distributed systems.

In this paper, we present a novel method of visualizing flow in blood vessels. Our approach reads unstructured tetrahedral data, resamples it, and uses slice based 3D texture volume rendering. Due to the sparse structure of blood vessels, we utilize an octree to efficiently store the resampled data by discarding empty regions of the volume. We use animation to convey time series data, wireframe surface to give structure, and utilize the StarCAVE, a 3D virtual reality environment, to add a fully immersive element to the visualization. Our tool has great value in interdisciplinary work, helping scientists collaborate with clinicians, by improving the understanding of blood flow simulations. Full immersion in the flow field allows for a more intuitive understanding of the flow phenomena, and can be a great help to medical experts for treatment planning.

We present a human-computer interface that enables the operator to plan a surgical procedure on the musculoskeletal (MS) model of the patient's lower limbs, send the modified model to the bio-mechanical analysis module, and export the scenario parameters to the surgical navigation system. The interface provides the operator with tools for: importing customized MS model of the patient, cutting bones and manipulating/removal of bony fragments, repositioning muscle insertion points, muscle removal and placing implants. After planning the operator exports the modified MS model for bio-mechanical analysis of the functional outcome. If the simulation result is satisfactory the exported scenario data may be directly used during the actual surgery. The advantages of the developed interface are the possibility of installing it in various hardware configurations and coherent operation regardless of the devices used. The hardware configurations proposed to be used with the interface are: (a) a standard computer keyboard and mouse, and a 2-D display, (b) a touch screen as a single device for both input and output, or (c) a 3-D display and a haptic device for natural manipulation of 3-D objects. The interface may be utilized in two main fields. Experienced surgeons may use it to simulate their intervention plans and prepare input data for a surgical navigation system while student or novice surgeons can use it for simulating results of their hypothetical procedure. The interface has been developed in the TLEMsafe project (www.tlemsafe.eu) funded by the European Commission FP7 program.

We examined what effect the reaching distance had on the prediction of a visually perceived location using reaching action. A system presenting a virtual object must execute the process of interaction when a body is directly at the visually perceived location of the virtual object to enable direct interaction between an observer's body and the virtual object. Conventional techniques assume that the visually perceived location is the same as the location defined by binocular disparity. However, both locations are often different. We proposed a new technique in our previous studies to predict the visually perceived location using an observer's action. We also demonstrated prediction using an action where an observer reached out to a virtual object. This study was an examination into the range of applications of our proposed approach. An observer in an experiment reached out to a virtual object, and the reaching distance was the experimental variable. The results did not support the effect of the reaching distance on prediction. We demonstrated that our technique could be applied to a wide range of reaching distances.

As technology grows people are diverted and are more interested in communicating with machine or computer naturally. This will make machine more compact and portable by avoiding remote, keyboard etc. also it will help them to live in an environment free from electromagnetic waves. This thought has made 'recognition of natural modality in human computer interaction' a most appealing and promising research field. Simultaneously it has been observed that using single mode of interaction limit the complete utilization of commands as well as data flow. In this paper a multimodal platform, where out of many natural modalities like eye gaze, speech, voice, face etc. human gestures are combined with human voice is proposed which will minimize the mean square error. This will loosen the strict environment needed for accurate and robust interaction while using single mode. Gesture complement Speech, gestures are ideal for direct object manipulation and natural language is used for descriptive tasks. Human computer interaction basically requires two broad sections recognition and interpretation. Recognition and interpretation of natural modality in complex binary instruction is a tough task as it integrate real world to virtual environment. The main idea of the paper is to develop a efficient model for data fusion coming from heterogeneous sensors, camera and microphone. Through this paper we have analyzed that the efficiency is increased if heterogeneous data (image & voice) is combined at feature level using artificial intelligence. The long term goal of this paper is to design a robust system for physically not able or having less technical knowledge.

This article deals with virtual tools based on principles of open source philosophy in implementation area of composite lay-up technology. It describes virtual software and hardware elements that are necessary for work in augmented reality environment. In the beginning it focuses on general problems of applications of virtual components and in composite lay-up process. It clarifies the fundamental philosophy of new created application and the process called visual scripting that was used for program development. Further it provides closer view on particular logical sections, where necessary data are harvested and compared with values from virtual arrays. Also the new device is described with adjustment of operating desk, what enables detailed control of any realized manufacturing process. This positioning table can determine and set the position of the working plane using the commands in computer interface or manual changes. Information about exact position of individual layers are obtained in real time thanks to the built-in sensors. One of them manages the data change of the desk position (X, Y, Z), other checks the rotation around main axis situated in the center of the table. New software consists of 4 main logical areas with data packets comming from internal computer components as well as from external devices. In the end the displaying section is able to realize the movement process of virtual item (composite layer) according to its trajectory. Article presents new attitude in realization of composite lay-up process. Finally it deals with possible future improvements and other application possibilities.