When robots come across unfamiliar objects, they struggle to account for a simple truth: Appearances aren’t everything. They may attempt to grasp a block, only to find out it’s a literal piece of cake. The misleading appearance of that object could lead the robot to miscalculate physical properties like the object’s weight and center of mass, using the wrong grasp and applying more force than needed.

To see through this illusion, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers designed the Grasping Neural Process, a predictive physics model capable of inferring these hidden traits in real time for more intelligent robotic grasping. Based on limited interaction data, their deep-learning system can assist robots in domains like warehouses and households at a fraction of the computational cost of previous algorithmic and statistical models.

The Grasping Neural Process is trained to infer invisible physical properties from a history of attempted grasps, and uses the inferred properties to guess which grasps would work well in the future. Prior models often only identified robot grasps from visual data alone.

Typically, methods that infer physical properties build on traditional statistical methods that require many known grasps and a great amount of computation time to work well. The Grasping Neural Process enables these machines to execute good grasps more efficiently by using far less interaction data and finishes its computation in less than a tenth of a second, as opposed to seconds (or minutes) required by traditional methods.

The researchers note that the Grasping Neural Process thrives in unstructured environments like homes and warehouses, since both house a plethora of unpredictable objects. For example, a robot powered by the MIT model could quickly learn how to handle tightly packed boxes with different food quantities without seeing the inside of the box, and then place them where needed. At a fulfillment center, objects with different physical properties and geometries would be placed in the corresponding box to be shipped out to customers.

Trained on 1,000 unique geometries and 5,000 objects, the Grasping Neural Process achieved stable grasps in simulation for novel 3D objects generated in the ShapeNet repository. Then, the CSAIL-led group tested their model in the physical world via two weighted blocks, where their work outperformed a baseline that only considered object geometries.

Limited to 10 experimental grasps beforehand, the robotic arm successfully picked up the boxes on 18 and 19 out of 20 attempts apiece, while the machine only yielded eight and 15 stable grasps when unprepared.

While less theatrical than an actor, robots that complete inference tasks also have a three-part act to follow: training, adaptation, and testing. During the training step, robots practice on a fixed set of objects and learn how to infer physical properties from a history of successful (or unsuccessful) grasps.

The new CSAIL model amortizes the inference of the objects’ physics, meaning it trains a neural network to learn to predict the output of an otherwise expensive statistical algorithm. Only a single pass through a neural network with limited interaction data is needed to simulate and predict which grasps work best on different objects.

Then, the robot is introduced to an unfamiliar object during the adaptation phase. During this step, the Grasping Neural Process helps a robot experiment and update its position accordingly, understanding which grips would work best. This tinkering phase prepares the machine for the final step: testing, where the robot formally executes a task on an item with a new understanding of its properties.

“As an engineer, it’s unwise to assume a robot knows all the necessary information it needs to grasp successfully,” says lead author Michael Noseworthy, an MIT Ph.D. student in electrical engineering and computer science (EECS) and CSAIL affiliate.

“Without humans labeling the properties of an object, robots have traditionally needed to use a costly inference process.”

According to fellow lead author, EECS Ph.D. student, and CSAIL affiliate Seiji Shaw, their Grasping Neural Process could be a streamlined alternative: “Our model helps robots do this much more efficiently, enabling the robot to imagine which grasps will inform the best result.”

“To get robots out of controlled spaces like the lab or warehouse and into the real world, they must be better at dealing with the unknown and less likely to fail at the slightest variation from their programming. This work is a critical step toward realizing the full transformative potential of robotics,” says Chad Kessens, an autonomous robotics researcher at the U.S. Army’s DEVCOM Army Research Laboratory, which sponsored the work.

While their model can help a robot infer hidden static properties efficiently, the researchers would like to augment the system to adjust grasps in real time for multiple tasks and objects with dynamic traits. They envision their work eventually assisting with several tasks in a long-horizon plan, like picking up a carrot and chopping it. Moreover, their model could adapt to changes in mass distributions in less static objects, like when you fill up an empty bottle.

Joining the researchers on the paper is Nicholas Roy, MIT professor of aeronautics and astronautics and CSAIL member, who is a senior author. The group recently presented this work at the IEEE International Conference on Robotics and Automation (ICRA 2024), held in Yokohama, Japan, May 13–17.

Most people call an internet outage an annoyance. The researchers at USC’s ANT Lab (Analysis of Network Traffic) call it a data point.

The ANT Lab, based out of USC Viterbi’s Information Sciences Institute (ISI), has been scanning the public internet looking for outages since 2014. USC Ph.D. student Xiao Song and John Heidemann, principal scientist at ISI and research professor of Computer Science at USC Viterbi School of Engineering, were considering this data when the COVID-19 epidemic started in 2020.

Heidemann said, “We were inspired by the network at ISI, where we could see four or five bumps every week, corresponding with the work week. After the Martin Luther King Holiday we saw four bumps in that work week because nobody came in on that Monday. And then we saw COVID hit, and all of a sudden there were no bumps.”

The “bumps” they were seeing were ISI employees’ laptops and IP addresses connecting to the ISI network when they were at work. Heidemann and Song thought perhaps this could be generalized and applied across the entire internet, to see if they could pick up signals of human activity from internet usage data.

Their resulting paper, “Inferring Changes in Daily Human Activity from Internet Response,” is the first demonstration of inferring changes in human activity, such as the transition to work-from-home, from IP responsiveness, and an important example of using the internet to understand our world.

Heidemann and Song presented their paper at the 2023 Internet Measurement Conference, held in Montreal, Canada, from October 24–26, 2023.

The internet isn’t out, but employees are

Since 2013, the ANT Lab has had an ongoing project that actively probes the internet to detect outages worldwide (currently 5 million networks measured every 11 minutes).

Heidemann and Song used this existing data to look for and analyze changes in internet usage worldwide that could indicate something about human behavior. They developed algorithms to clean the data, extract underlying trends, and detect changes in activity.

They found that, by using their algorithms, they could identify work-from-home orders that were put in place due to the emergence of COVID-19 in 2020. They could also identify other changes in human activity, such as national holidays and government-mandated curfews.

Song explained, “We looked for significant changes in our human behavior change maps and compared those change event dates to news reports for the same location. For example, around late March 2020, network usage plummeted in Manila, Philippines. The news timeline confirmed that the change we saw correlated with Manila’s COVID lockdown which began on March 15, 2020.”

Real world events spotted via internet activity: Two case studies

China in January 2020

Using their technique, the team detected activity changes in China in late January 2020. They correlated these changes with two concurrent events: the Wuhan COVID lockdown and the week-long Spring Festival, a national holiday where people typically do not go into their offices. Since the Wuhan lockdown and Spring Festival were concurrent events, they cannot attribute network changes specifically to either one.

India in February and March 2020

The team detected network changes for several days in India in both February and March 2020. When they looked at news in the area, they found that the February network activity correlated with riots in India associated with immigration law protests. While the March network activity corresponded with the first COVID-related curfew in India and subsequent lockdown order.

These case studies suggest that changes in human behavior that lead them to work from home can have multiple causes, but their outcome on the internet is similar.

What’s next?

Heidemann said, “Our first goal was just curiosity—can we see human activity in the internet?” Now that the team has shown they can, what can they do with and learn from this information. He continued, “In the context of COVID, we could explore questions like: what countries have lockdowns or stay-at-home orders? When do they have them? If you have a stay-at-home order, do people actually follow it or do they not comply and go into work anyway?”

This ability to detect trends in human activity from the internet data provides a new ability to understand our world, complementing other sources of public information such as news reports and wastewater virus observation.

Heidemann explained, “People use wastewater detection systems to understand the background level of COVID in a city. And that’s great, because it’s an anonymous and reliable method to judge what’s happening in a city as a whole. I would hope that our technique could provide a similar kind of anonymous, independent, third party observation about what’s going on, and that might feed into public health decisions.”

The new Tandem Dual-Antenna Spaceborne Synthetic Aperture Radar (SAR) Interferometry (TDA-InSAR) system, addresses the limitations of current spaceborne Synthetic Aperture Radar (SAR) systems by providing a more reliable and efficient method for 3D surface mapping. The system’s innovative design allows for single-pass acquisitions, significantly reducing the time required for data collection and enhancing the precision of 3D reconstructions in various terrains, including built-up areas and vegetation canopies.

Synthetic Aperture Radar (SAR) interferometry (InSAR) is a powerful tool for producing high-resolution topographic maps. However, traditional InSAR techniques face challenges such as the ill-posed 2D phase unwrapping problem and the need for multiple acquisitions over time, which can introduce errors due to atmospheric and orbital changes. The TDA-InSAR system overcomes these challenges by utilizing dual-antenna and dual-satellite configurations to acquire optimal interferograms for an asymptotic 3D phase unwrapping algorithm.

Researchers from Fudan University and the Chinese Academy of Sciences have developed a novel Tandem Dual-Antenna Spaceborne SAR Interferometry (TDA-InSAR) system, designed to achieve optimal multi-baseline interferograms for fast 3D reconstruction. The study, published on 6 May 2024, in the journal Journal of Remote Sensing, presents a systematic investigation into the performance of various baseline configurations and the impact of different error sources on the system’s accuracy.

The TDA-InSAR system employs a dual-antenna and dual-satellite approach to capture optimal interferograms, which are then processed through an asymptotic 3D phase unwrapping algorithm. This method allows for rapid and accurate 3D reconstruction with minimal acquisitions, overcoming the limitations of previous technologies.

The study’s simulations demonstrated that the TDA-InSAR system could achieve a remarkable relative height precision of 0.3 meters in urban areas and 1.7 meters in dense vegetation, outperforming existing SAR interferometry methods. The research also explored various baseline configurations, finding that a bi-static mode with a flexible satellite baseline provided the best results.

“The TDA-InSAR system represents a significant advancement in SAR interferometry,” said Fengming Hu, the lead researcher of the study.

“By tailoring the system to work with an asymptotic 3D phase unwrapping algorithm, we’ve been able to achieve a relative height precision of 0.3 meters in built-up areas and 1.7 meters in vegetation canopies, which is a substantial improvement over existing technologies.”

The TDA-InSAR system has significant implications for various applications, including terrain mapping, target recognition, and forest height inversion. Its ability to perform rapid 3D reconstruction in a single flight makes it a valuable tool for both scientific research and practical applications such as disaster response and environmental monitoring.

Provided by
Journal of Remote Sensing