How can virtual reality (VR) be experienced haptically, i.e., through the sense of touch? This is one of the fundamental questions that modern VR research is investigating.

Computer scientist André Zenner, who is based in Saarbrücken, Germany, has come a significant step closer to answering this question in his doctoral thesis—by inventing new devices and developing software-based techniques inspired by human perception. He has now been awarded the “Best Dissertation Award” at the world’s leading VR conference.

The award-winning work is about how physical props (technical term: “proxies”) can be used to make objects in virtual environments tangible.

“Of course, you can’t have a proxy for every virtual object, then the approach wouldn’t be scalable. In my dissertation, I, therefore, thought about what devices could look like that could be used to simulate the physical properties of several different virtual objects as effectively as possible,” explains Zenner, who completed his doctorate at the Saarbrücken Graduate School of Computer Science at Saarland University and is now conducting research at Saarland University and the German Research Center for Artificial Intelligence.

This resulted in the prototypes for two special VR controllers, “Shifty” and “Drag:on.” VR controllers are devices that can be held in the user’s hand to control or manipulate objects in virtual reality using tracking technology.

“Shifty” is a tubular controller in which a movable weight is installed. The weight can be moved along the lengthwise axis by a motor, changing the center of gravity and inertia of the rod.

“In combination with corresponding visualizations in virtual reality, Shifty can be used to create the illusion that a virtual object is getting longer or heavier,” explains Zenner. In experiments, he was able to prove that objects are perceived as lighter or smaller when the weight is close to the user’s hand and that, coupled with the corresponding visual input; they are perceived as longer and heavier the further the weight in the rod moves away from the user.

“This is mainly due to changes in the inertia of the controller, as the overall weight does not change,” explains Zenner. The research and development department of gaming giant Sony is already experimenting with this concept and cites Zenner’s work in the development of new VR controllers.

The second controller, “Drag:on,” consists of two flamenco fans that can be unfolded using servomotors, thus increasing the air resistance of the controller. This means that the further the fans are unfolded, the more force the user has to exert to move the controller through the air.

“Coupled with the right visual stimuli, Drag:on can be used to create the impression that the user is holding a small shovel or a large paddle, for example, or that they are pushing a heavy trolley or are twisting a knob that is difficult to turn,” explains Zenner.

Both controllers are basic research and so-called “proof of concepts.” This means that the prototypes can be used to show in user experiments that different controller states can improve the perception of different VR objects. Still, specific products using this technology are not yet available on the market.

With the controllers, the Saarbrücken-based computer scientist first addressed the so-called ‘similarity problem.” The aim here is to ensure that virtual and real objects feel as similar as possible. In the second part of his work, he dealt with the so-called “colocation problem,” i.e., the question of how the proxy can be spatially located in real life where the user sees it in virtual reality.

This is particularly challenging as the controllers act as proxies for different virtual objects. Consequently, the user must be given the illusion that they are reaching for various objects, although in reality, they will always grasp the same proxy.

To achieve this, the researcher made use of the already established method of “hand redirection.” As the name suggests, this involves redirecting the movement of the hand in virtual reality so that the user thinks they are reaching to the left, for example, even though they are actually stretching their hand forward.

“We conducted experiments to investigate the point at which users realize that their hand has been redirected. Our results showed that this point was reached quickly, so we thought about how we could better conceal the hand redirection,” says Zenner.

The solution: he tricked the brain by only redirecting the hand when the brain was blind to visual changes—namely during blinking. Together with a student under his supervision, he developed the appropriate software and used the eye trackers built into many VR headsets.

In control studies, the team was then able to show that their new controllers, in combination with hand redirection algorithms, led to more convincing VR perceptions than previously possible.

Provided by
Universität des Saarlandes

Using NASA’s first two-way, end-to-end laser relay system, pictures and videos of cherished pets flew through space over laser communications links at a rate of 1.2 gigabits per second—faster than most home internet speeds.

NASA astronauts Randy Bresnik, Christina Koch, and Kjell Lindgren, along with other agency employees, submitted photos and videos of their pets to take a trip to and from the International Space Station.

The transmissions allowed NASA’s SCaN (Space Communications and Navigation) program to showcase the power of laser communications while simultaneously testing out a new networking technique.

“The pet imagery campaign has been rewarding on multiple fronts for the ILLUMA-T, LCRD, and HDTN teams,” said Kevin Coggins, deputy associate administrator and SCaN program manager at NASA Headquarters in Washington. “Not only have they demonstrated how these technologies can play an essential role in enabling NASA’s future science and exploration missions, it also provided a fun opportunity for the teams to ‘picture’ their pets assisting with this innovative demonstration.”

This demonstration was inspired by “Taters the Cat”—an orange cat whose video was transmitted 19 million miles over laser links to the DSOC (Deep Space Optical Communications) payload on the Psyche mission. LCRD, DSOC, and ILLUMA-T are three of NASA’s ongoing laser communications demonstrations to prove out the technology’s viability.

The images and videos started on a computer at a mission operations center in Las Cruces, New Mexico. From there, NASA routed the data to optical ground stations in California and Hawaii. Teams modulated the data onto infrared light signals, or lasers, and sent the signals to NASA’s LCRD (Laser Communications Relay Demonstration) located 22,000 miles above Earth in geosynchronous orbit. LCRD then relayed the data to ILLUMA-T (Integrated LCRD Low Earth Orbit User Modem and Amplifier Terminal), a payload currently mounted on the outside of the space station.

Since the beginning of space exploration, NASA missions have relied on radio frequency communications to send data to and from space. Laser communications, also known as optical communications, employ infrared light instead of radio waves to send and receive information.

While both infrared and radio travel at the speed of light, infrared light can transfer more data in a single link, making it more efficient for science data transfer. This is due to infrared light‘s tighter wavelength, which can pack more information onto a signal than radio communications.

This demonstration also allowed NASA to test out another networking technique. When data is transmitted across thousands and even millions of miles in space, the delay and potential for disruption or data loss is significant. To overcome this, NASA developed a suite of communications networking protocols called Delay/Disruption Tolerant Networking, or DTN. The “store-and-forward” process used by DTN allows data to be forwarded as it is received or stored for future transmission if signals become disrupted in space.

To enable DTN at higher data rates, a team at NASA’s Glenn Research Center in Cleveland developed an advanced implementation, HDTN (High-Rate Delay Tolerant Networking). This networking technology acts as a high-speed path for moving data between spacecraft and across communication systems, enabling data transfer at a speed of up to four times faster than current DTN technology—allowing high-speed laser communication systems to utilize the “store-and-forward” capability of DTN.

The HDTN implementation aggregates data from a variety of different sources, like discoveries from the scientific instrumentation on the space station, and prepares the data for transmission back to Earth. For the pet photo and video experiment, the content was routed using DTN protocols as they traveled from Earth to LCRD, to ILLUMA-T on the space station. Once they arrived, an onboard HDTN payload demonstrated its ability to receive and reassemble the data into files.

This optimized implementation of DTN technology aims to enable a variety of communications services for NASA, from improving security through encryption and authentication to providing network routing of 4K high-definition multimedia and more. All of these capabilities are being tested on the space station with ILLUMA-T and LCRD.

As NASA’s Artemis campaign prepares to establish a sustainable presence on and around the moon, SCaN will continue to develop ground-breaking communications technology to bring the scalability, reliability, and performance of the Earth-based internet to space.