For the sixth consecutive year, California officials are delaying the Bay Area’s commercial Dungeness crab season because of the “high abundance” of migrating humpback whales already getting ensnared by old crab-pot fishing lines and other gear—while conservation groups are again expressing frustration with what they say is a too-slow transition to safer fishing methods.

The upcoming season normally would start Nov. 15 in the waters from the Sonoma/Mendocino County line south to the Mexican border.

The decision came down on Friday afternoon from Charlton H. Bonham, director of the California Department of Fish and Wildlife, and was made in consultation with representatives of the fishing industry, environmental organizations and scientists.

According to the nonprofit conservation group Oceana, a participant in those discussions, during the period from May to Oct. 21 of this year, four humpback whales were confirmed entangled in California commercial Dungeness crab fishing gear. Ten others were observed entangled in “unknown fishing gear that may be Dungeness crab gear.”

Other entanglements were sighted the past few days, said marine scientist Caitlynn Birch, Oceana’s campaign manager.

Both she and representatives from the Center for Biological Diversity spoke out in favor of the use of rope-less or pop-up gear to give whales a safe migration south.

“I’m glad state officials are taking precautions to avoid entangling whales in the area, but with viable pop-up gear available, these season delays are getting increasingly archaic,” said Ben Grundy, oceans campaigner at the Center for Biological Diversity, in a statement.

He noted that if the state had authorized the use of pop-up gear—which he said has performed well in tests—”crab fishers could be prepping to put their traps in the water right now.”

Although another assessment will take place on or around Nov. 22, that date will make it impossible for crab to make an appearance on Thanksgiving dinner tables on Nov. 28. That assessment could, however, lead to an opening date in early December.

The recreational Dungeness season will be allowed to start as scheduled on Nov. 2—but with restrictions. Crabbers may not use trap gear, according to the state order; only hoop nets and crab snares will be allowed until further assessment.

Since 2015, there have been delays in all but one commercial Dungeness season in the Bay Area. A toxin, domoic acid, that could sicken anyone who eats the tainted crab destroyed Northern California’s 2015–2016 commercial season and created delays in other years.

In 2018, the commercial season began without a hitch although recreational crabbers had to postpone their fishing.

In 2019 and 2020, the fishing line danger to whales resulted in a crabbing delay of several weeks. The 2020 crabbing season was officially set to begin Dec. 23, but price negotiations between crab fleets and seafood processors delayed the start until early January 2021.

With delays to protect whales, the truncated 2021–22 season ran from Dec. 29 to April 8, and the 2022–23 season from Dec. 31 to April 15. The 2023–24 season didn’t begin until Jan. 18, 2024, and closed early, on April 8.

2024 MediaNews Group, Inc. Distributed by Tribune Content Agency, LLC.

Around 7000 years ago, long knives, bracelets and other stone goods fashioned by skilled Parisian crafters were reaching people hundreds of kilometres away, via complex trade networks that are now being mapped for the first time.

By combining archaeology with computer modelling, Solène Denis at the French National Centre for Scientific Research in Nanterre and Michael Kempf at the University of Basel in Switzerland have reconstructed the lengthy and winding paths taken to supply people from what is now Normandy…

In the classic cartoon “The Jetsons,” Rosie the robotic maid seamlessly switches from vacuuming the house to cooking dinner to taking out the trash. But in real life, training a general-purpose robot remains a major challenge.

Typically, engineers collect data that are specific to a certain robot and task, which they use to train the robot in a controlled environment. However, gathering these data is costly and time-consuming, and the robot will likely struggle to adapt to environments or tasks it hasn’t seen before.

To train better general-purpose robots, MIT researchers developed a versatile technique that combines a huge amount of heterogeneous data from many sources into one system that can teach any robot a wide range of tasks.

Their method involves aligning data from varied domains, like simulations and real robots, and multiple modalities, including vision sensors and robotic arm position encoders, into a shared “language” that a generative AI model can process.

The work is published on the arXiv preprint server.

By combining such an enormous amount of data, this approach can be used to train a robot to perform a variety of tasks without the need to start training it from scratch each time.

This method could be faster and less expensive than traditional techniques because it requires far fewer task-specific data. In addition, it outperformed training from scratch by more than 20% in simulation and real-world experiments.

“In robotics, people often claim that we don’t have enough training data. But in my view, another big problem is that the data come from so many different domains, modalities, and robot hardware. Our work shows how you’d be able to train a robot with all of them put together,” says Lirui Wang, an electrical engineering and computer science (EECS) graduate student and lead author of the paper on this technique.

Wang’s co-authors include fellow EECS graduate student Jialiang Zhao; Xinlei Chen, a research scientist at Meta; and senior author Kaiming He, an associate professor in EECS and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research will be presented at the Conference on Neural Information Processing Systems, held 10–15 December at the Vancouver Convention Center.

Inspired by LLMs

A robotic “policy” takes in sensor observations, like camera images or proprioceptive measurements that track the speed and position a robotic arm, and then tells a robot how and where to move.

Policies are typically trained using imitation learning, meaning a human demonstrates actions or teleoperates a robot to generate data, which are fed into an AI model that learns the policy. Because this method uses a small amount of task-specific data, robots often fail when their environment or task changes.

To develop a better approach, Wang and his collaborators drew inspiration from large language models like GPT-4.

These models are pretrained using an enormous amount of diverse language data and then fine-tuned by feeding them a small amount of task-specific data. Pretraining on so much data helps the models adapt to perform well on a variety of tasks.

“In the language domain, the data are all just sentences. In robotics, given all the heterogeneity in the data, if you want to pretrain in a similar manner, we need a different architecture,” he says.

Robotic data take many forms, from camera images to language instructions to depth maps. At the same time, each robot is mechanically unique, with a different number and orientation of arms, grippers, and sensors. Plus, the environments where data are collected vary widely.

The MIT researchers developed a new architecture called Heterogeneous Pretrained Transformers (HPT) that unifies data from these varied modalities and domains.

They put a machine-learning model known as a transformer into the middle of their architecture, which processes vision and proprioception inputs. A transformer is the same type of model that forms the backbone of large language models.

The researchers align data from vision and proprioception into the same type of input, called a token, which the transformer can process. Each input is represented with the same fixed number of tokens.

Then the transformer maps all inputs into one shared space, growing into a huge, pretrained model as it processes and learns from more data. The larger the transformer becomes, the better it will perform.

A user only needs to feed HPT a small amount of data on their robot’s design, setup, and the task they want it to perform. Then HPT transfers the knowledge the transformer gained during pretraining to learn the new task.

Enabling dexterous motions

One of the biggest challenges of developing HPT was building the massive dataset to pretrain the transformer, which included 52 datasets with more than 200,000 robot trajectories in four categories, including human demo videos and simulation.

The researchers also needed to develop an efficient way to turn raw proprioception signals from an array of sensors into data the transformer could handle.

“Proprioception is key to enable a lot of dexterous motions. Because the number of tokens is in our architecture always the same, we place the same importance on proprioception and vision,” Wang explains.

When they tested HPT, it improved robot performance by more than 20% on simulation and real-world tasks, compared with training from scratch each time. Even when the task was very different from the pretraining data, HPT still improved performance.

“This paper provides a novel approach to training a single policy across multiple robot embodiments. This enables training across diverse datasets, enabling robot learning methods to significantly scale up the size of datasets that they can train on. It also allows the model to quickly adapt to new robot embodiments, which is important as new robot designs are continuously being produced,” says David Held, associate professor at the Carnegie Mellon University Robotics Institute, who was not involved with this work.

In the future, the researchers want to study how data diversity could boost the performance of HPT. They also want to enhance HPT so it can process unlabeled data like GPT-4 and other large language models.

“Our dream is to have a universal robot brain that you could download and use for your robot without any training at all. While we are just in the early stages, we are going to keep pushing hard and hope scaling leads to a breakthrough in robotic policies, like it did with large language models,” he says.

This story is republished courtesy of MIT News (web.mit.edu/newsoffice/), a popular site that covers news about MIT research, innovation and teaching.