Sometimes it seems like an AI is helping, but the benefit is actually a placebo effect—people performing better simply because they expect to be doing so—according to new research from Aalto University. The study also shows how difficult it is to shake people’s trust in the capabilities of AI systems.

In this study, participants were tasked with a simple letter recognition exercise. They performed the task once on their own and once supposedly aided by an AI system. Half of the participants were told the system was reliable and it would enhance their performance, and the other half was told that it was unreliable and would worsen their performance.

The findings are published in Proceedings of the CHI Conference on Human Factors in Computing Systems.

“In fact, neither AI system ever existed. Participants were led to believe an AI system was assisting them, when in reality, what the sham-AI was doing was completely random,” explains doctoral researcher Agnes Kloft.

The participants had to pair letters that popped up on screen at varying speeds. Surprisingly, both groups performed the exercise more efficiently—more quickly and attentively—when they believed an AI was involved.

“What we discovered is that people have extremely high expectations of these systems, and we can’t make them AI doomers simply by telling them a program doesn’t work,” says Assistant Professor Robin Welsch.

Following the initial experiments, the researchers conducted an online replication study that produced similar results. They also introduced a qualitative component, inviting participants to describe their expectations of performing with an AI. Most had a positive outlook toward AI and, surprisingly even skeptical people still had positive expectations about its performance.

The findings pose a problem for the methods generally used to evaluate emerging AI systems. “This is the big realization coming from our study—that it’s hard to evaluate programs that promise to help you because of this placebo effect,” Welsch says.

While powerful technologies like large language models undoubtedly streamline certain tasks, subtle differences between versions may be amplified or masked by the placebo effect—and this is effectively harnessed through marketing.

The results also pose a significant challenge for research on human-computer interaction, since expectations would influence the outcome unless placebo control studies were used.

“These results suggest that many studies in the field may have been skewed in favor of AI systems,” concludes Welsch.

The best way to help smartphone users manage their screen time may be to make phones progressively more annoying to use, according to new University of Michigan research.

The study, published in Proceedings of the CHI Conference on Human Factors in Computing Systems, shows that interfering with swiping and tapping functions is around 16% more effective at reducing screen time and the number of times an app is opened than forcibly locking users out of their phones.

The lockout strategy is used by many screen-time management apps today, and such apps also send users a notification offering more time before locking. Researchers discussed the findings Monday, May 13, at the Association for Computing Machinery’s Conference on Human Factors in Computing Systems (CHI 2024) in Honolulu.

“Lockout apps are pretty disruptive, so if someone is in the middle of an important task or a game, they’ll scramble to skip through the screen timer. Then, they can forget about the time limit and spend more time on the phone than they wanted to,” said Anhong Guo, U-M assistant professor of computer science and engineering and the corresponding author of the study.

The researchers’ InteractOut app is more effective at limiting screen time because it is less restrictive and harder to ignore than hard lockouts. Once the user’s designated screen limit has been reached, InteractOut can delay the phone’s response to a user’s gesture, shift where tapping motions are registered or slow the screen scrolling speed.

The strength of the delays and shifts continues to increase each time the user touches the phone, up to a pre-set maximum, and the user can decide how the app interferes with their phone use. The app’s gradual interference allows users to continue using their phone, but with a little extra difficulty.

“If we just continuously add a little bit of friction to the interaction with the phone, eventually the user becomes more aware of what they are doing because there’s a mismatch between what they expect to happen and what actually happens. That makes using smartphones a more deliberate process,” said Guo, who also is an assistant professor of information.

The researchers believe that forcing more mindfulness into otherwise mindless gesturing is the key to making smartphones less addictive.

“We want to evoke users’ awareness of using their smartphone so that they can use it more productively,” said study first author Tao Lu, a recent U-M bachelor of science graduate in computer science who is now a master’s student at the Georgia Institute of Technology.

While designing InteractOut, the researchers had to be careful not to make the phone so inconvenient that it became insufferable. To ensure their software strikes the right balance, the team tested their app’s performance in a field study of 42 participants that took place over five weeks.

In the first week of the study, the researchers reviewed how often each participant used their phones without screen time management tools. Then, each participant installed the InteractOut app on their Android phone and chose which other apps it could monitor and interfere with. The researchers fixed the participants’ daily screen time allowance to one hour, after which InteractOut began to modify swipe and tap functions inside of the specified apps.

All participants received a random swipe and tap intervention from InteractOut for a single two-week period, and their screen time was compared to a separate two-week period in which they used Timed Lockout, a widely available app that imposes hard lockouts.

The researchers found that InteractOut was not only more effective at reducing screen time for the targeted apps than a hard lockout, but it was also better received by the study participants. When the screen-time management apps were activated, around 62% of the participants kept InteractOut’s interventions on for the day, but only 36% of the participants did the same with Timed Lockout.

There is still room for improvement, however. The participants thought that InteractOut was too intrusive for some games that require precise, real-time movements. It was also less effective at limiting the amount of time spent on apps that require few tapping or swiping gestures, such as video streaming services. Guo plans to find ways to tailor the app’s interventions to be better suited for different kinds of phone apps.

The team has submitted an invention disclosure for the software and hope to eventually bring it to market.