The public release of ChatGPT and other large language models (LLMs) has allowed developers worldwide to start experimenting with these models to enhance the interactive capabilities of their own systems. Similar generalizable models for robotic manipulation, however, remain scarce.

Researchers at University of California, Berkeley (UC Berkeley), Stanford University and CMU recently introduced Octo, an open-source generalist model for robotic manipulation that could allow different robotic systems to effectively manipulate a wide range of objects. This model, presented in a paper pre-published on the server arXiv, could open new avenues for the development of robots that can tackle manual tasks.

“Much of the current progress in AI is driven by large datasets and large models,” Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black and Oier Mees, told Tech Xplore. “In the robotics community, we recently assembled the Open X-Embodiment dataset, a big manipulation dataset that pools data from many research institutions. While this new dataset is a really exciting resource, at the time there weren’t many models that could make use of it yet.”

The recent work by this research team had two main objectives. The first was to develop a good generalist robotics model that could be applied to various robots and the second was to create open-source code that would allow other researchers to build similar models in the future.

“Octo is what we call a ‘generalist’ robot model, a neural network that can control many different types of robots and make them fulfill requests like ‘pick up the spoon,’ ‘close the drawer,’ ‘wipe the table’ etc.,” Ghosh, Walke, Pertsch, Black and Mees explained.

“Being a generalist and working on many robots is key, because if you look at research labs around the world, many of them use different robots, so the only way to ensure Octo can be used by many researchers is by supporting a wide range of robots.”

Within the technology research and development community, highly performing computational tools that can be applied across multiple systems are often referred to as foundational models. An example of these models is ChatGPT, which can be used to equip various agents and systems with natural language processing (NLP) capabilities.

“We want to build similar foundation models, but for robot control, or in other words, models that can control many robots and make them solve many different tasks,” Ghosh, Walke, Pertsch, Black and Mees said.

“Octo is a first step towards that goal. Its training looks very similar to models like ChatGPT: we curate a large and diverse dataset, in our case robot data instead of text, and train a large model to predict the next action the robot should execute given the current robot state and a task instruction.”

Octo, the model developed by Ghosh, Walke, Pertsch, Black and Mees is based on the same type of neural networks as ChatGPT, known as transformers. A key advantage of Octo over other previously developed robotics models is the scale of the data used to train it and its flexibility.

The model was trained on the largest dataset of robotic manipulation trajectories compiled to date; the Open X-Embodiment dataset. Octo can also process a diverse range of sensory inputs including different types of images, robot joint readings, language instructions, goal-related images and more.

“Octo can also control many different types of robot arms, from small single arms that can barely pick up a soda can, to larger, more powerful robot arms and even bi-manual setups,” Ghosh, Walke, Pertsch, Black and Mees said. “This flexibility is what makes Octo more applicable to the diverse setups roboticists actually have around the world.”

The researchers evaluated their model in a series of initial experiments, deploying it on nine different robotic systems developed at UC Berkely, Stanford and CMU. Octo succeeded in controlling these robots and allowed them to complete various manipulation tasks, even in instances where it had not encountered data collected by these robots’ sensors or their unique design during training.

“It was really cool to see that we can take our Octo model and use it to control many different robots,” the researchers said. “Since we released the model, we saw quite a few people who tried running it on their own robots and we have been using the codebase we built for Octo in our next projects as well. These are some encouraging signs that Octo will indeed help foster the next generation of improved foundation models for robotics.”

For the researchers, the development of Octo was merely a small milestone towards their goal of building a generalist model for robotic manipulation. In their next studies, they plan to continue working towards this goal and hope that research groups at other institutes will also start experimenting with their code.

“Right now, chances are that the model will not work on your robot out of the box and you need to collect a few examples of the task you want your robot to solve to teach it to Octo, even if it’s a mundane task like picking up a coke can in a new kitchen,” they added.

“This is to say, the generalization ability of the current model is still pretty limited and we’re working on new models that will push this a bit further. We’re not yet at the point where you can just download a model to your robot, tell your robot what you’d like it to do and it will succeed 9 out of 10 times, but we’re working towards this goal.”

© 2024 Science X Network

In a new study, scientists describe three new, but long-extinct, walnut species on an island above the Arctic Circle. The fossils were discovered further north than any known walnut species, living or extinct, and represent some of the oldest-known records of this group. The work is published in the International Journal of Plant Sciences.

Today, the Canadian island of Axel Heiberg is a frozen desert devoid of nearly all life. But 45 million years ago, it supported a lush rainforest on the shores of the Arctic Ocean. Since then, the forest has been buried first beneath layers of sediment, then accumulated ice, leaving it frozen in time.

“When you walk into the site, the first thing you notice are these big stumps, a meter or more in diameter, and they’re still rooted in the soil that they grew in. It’s completely out of place. The closest living trees are 3,000 kilometers away,” said study co-author James Basinger, professor emeritus of geological sciences at the University of Saskatchewan.

The stumps are so conspicuous, they can be spotted from the air. In 1985, staff from the Geological Survey of Canada discovered the Axel Heiberg fossil forest while conducting a survey of the area from a helicopter. A year later, paleobotanists returned to the site and found fossils unlike anything they’d seen before.

“There aren’t really that many places around where you can go to see fossils that are preserved that well,” said Steven Manchester, lead author of the study and curator of paleobotany at the Florida Museum of Natural History.

In most cases, fossilization is characterized by organic matter being replaced with minerals over time. In other cases, organic matter is heated and compressed into coal or burned in forest fires and preserved as charcoal. But this isn’t the case with the Axel Heiberg fossils. The wood, leaves, cones, nuts and fruit are seemingly unchanged. This unique form of preservation is referred to as mummification, and it only takes place under a very specific and rare set of circumstances.

“Things can be broken down by bacteria and fungi, they can be rumbled along in a riverbed and destroyed; there are lots of ways of losing the material before it becomes fossilized,” Basinger said. But the ancient forests on Axel Heiberg were buried rapidly under swamp and lake sediments. As the global climate cooled, these processes were slowed.

Basinger was among the first researchers to study the forest. The Arctic’s barren surfaces and strong winds made it remarkably easy to collect specimens. “You can see a few fossils on the surface and pick what you can. But you go back next year, when there’s been a little erosion, and there’s a few more on the surface. Over a number of years, you can actually get a large collection,” he said.

The walnuts had been eroded from the soil and were sitting on the surface. “In one case, the walnuts are concentrated at one spot, possibly cached there by animals,” Basinger said. Some of the fossil nuts also have gnawed holes, indicating they were a food source for local animals.

Over a period of fifteen years, Basinger and his colleagues retrieved over a thousand nut and seed fossils and returned with them to Saskatchewan to be studied.

Visualizing a globally warm planet

If you looked back 45 million years ago to the middle Eocene, the Earth’s poles would be unrecognizable. At the time, Antarctica and the Arctic Circle were warm and blanketed with forests, in stark contrast to the freezing deserts that we associate with the region today.

Due to their high latitude, polar regions had relatively short growing seasons, but they made up for it with exceptionally long summer days, receiving up to twenty hours of sunlight. Inversely, the winters were characterized by near-total darkness, yet temperatures seldom reached freezing.

Paleontology and geology records indicate there was more CO₂ in Earth’s atmosphere at the time, which resulted in temperatures that were much higher than they are now. This global greenhouse, in turn, created warm ocean circulations that kept the Arctic Ocean free of ice.

“The far north supported redwood-style forests,” Basinger said. There were cypress swamps and upland forests, where statuesque trees grew up to 40 meters in height. The canopy included dozens of trees, such as redwoods, cedars, hickories, pines, spruces, hemlocks, larches, birches, ginkgos and, of course, walnuts.

What we can learn from the Axel Heiberg walnuts

As an expert in the evolutionary history of the walnut family, Manchester helped bring the decades-long project to completion. He performed CT scans on walnuts from the island and described three previously unknown species.

“The CT scans allow us to show details of the internal structure of these nuts that were once really hard to get,” Manchester said. Before CT scanners, traditional methods for studying fossils involved tediously dissecting and slicing the specimens in various orientations, destroying them in the process.

After scanning several of the most completely preserved fossils, Manchester compared them to walnuts from both modern and extinct walnut species. National repositories, like iDigBio, allow researchers to easily locate museum specimens stored anywhere in the United States. The fossils didn’t match anything that had yet been discovered and were thus found to represent three new species in the genus Juglans.

Based solely on genetic data from living species, researchers once thought the walnut family originated somewhere in Asia. More recently, however, fossil data indicates they instead first appeared in the warm, moist environments of North America or Europe. As the family diversified, some species adapted to cooler conditions, which allowed them to extend their range into higher latitudes.

The fossils from this study add a clearer picture of how walnuts evolved during periods of intensely shifting climates and where our modern species came from.

Near the end of the last century, Bill Gates saw the prospect of unifying citizens of nearly 200 countries, speaking more than 7,000 languages, coming together in common dialogue through the suddenly burgeoning web community.

“The Internet is becoming the town square for the global village of tomorrow,” he declared.

The Internet certainly has since drawn the world closer and has enriched global communications, commerce, research and entertainment immeasurably.

But a recent report reminds us—as if we really needed reminding—that along with progress sometimes come problems.

Researchers at Amazon Web Services Artificial Intelligence Lab and the University of California, Santa Barbara, say that after examining more than 6 billion sentences across the web, they have found that more than half had been translated into two or more different languages. The translations, they found, were often poor. And with each successive translation into other languages, some up to eight or nine, the results became worse.

The report, “A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism,” was uploaded to the preprint server arXiv Jan. 11.

“The low quality of these … translations indicates they were likely created using machine translation,” the authors report. “Our work raises serious concerns about training models such as multilingual large language models on both monolingual and bilingual data scraped from the web.”

The researchers said texts are not only being translated by artificial intelligence but are being created by AI as well. They observed rates of AI-generated translations were highest among lower-resource languages, such as Wolof and Xhosa, African languages.

“We find that highly multi-way parallel translations are significantly lower quality than two-way parallel translations,” the authors continue.

That means that as trillions of bits of data are ingested for AI training operations, regions under-represented on the web, such as African nations and other countries with more obscure languages, will face greater challenges in establishing reliable—and grammatical—large language models. With few native resources to draw upon, they must heavily rely on tainted translations flooding the market.

Mehak Dhaliwal, a former applied science intern at Amazon Web Services, told Motherboard in an interview, “We actually got interested in this topic because several colleagues who work in machine training and are native speakers of low resource languages noted that much of the internet in their native language appeared to be machine training generated… Everyone should be cognizant that content they view on the web may have been generated by a machine.”

The Amazon researchers found bias in selection of content used for AI training.

They state, “Machine generated, multi-way parallel translations not only dominate the total amount of translated content on the web in lower resource languages, it also constitutes a large fraction of the total web content in those languages.”

Such content, they suggested, tends to be simpler, lower-quality passages “likely produced to generate ad revenue.” Since fluency and accuracy are lower for machine-trained material, numerous translations will lead to even less accurate content and increase the odds of AI hallucination.

Sometimes, computer-generated translations over the years have led to unintentionally humorous or embarrassing interpretations.

Google misinterpreted a phrase “Russia is a great country” and referred instead to Mordor, a fictional village in J.R.R. Tolkien’s “The Lord of the Rings.” Facebook’s translation software in 2019 inadvertently referred to China’s President Xi Jinping as “Mr. S***hole” several times in an English article translated from Burmese text. Facebook immediately apologized and blamed the mishap on a “technical error.”

And a medical prescription translation tool for Armenian speakers provided some unfortunate advice for a patient with a headache.

English: “You can take over-the-counter ibuprofen as needed for pain.”

Translation to Armenian: “You may take anti-tank missile as much as you need for pain.”

© 2024 Science X Network