As the holidays approach, the cyborgs get slothful
Source: Loco Steve, CC BY-SA 2.0
If you find yourself starting to check out during the last few days before the holiday break, you’re not alone: Large language models, like ChatGPT, do it too.
Yup, the bots have been lazy recently. After reading this Ars Technica article about how one theory is that ChatGPT is suffering from pre-holiday fatigue, my programmer husband Jesse and I decided it would be fun to see if large language models behave differently around other holidays. I’m not sure our experiments were rigorous enough to pass peer review, but they’re certainly a peek at the weird ways that artificial intelligence mimics our human idiosyncrasies.
We started with our own version of the test reported in Ars Technica: We asked a large language model, “Can you help me with something?” Then we looked at how often it refused (by starting its response with “sorry”) when we set the date to each day between Dec. 1 and Dec. 25. Instead of ChatGPT, we used a similar tool called Llama2 because the way it’s set up made it easier to access the data we wanted.
On Dec. 1, Llama2 is feeling like a very helpful little bot, and it only refuses to help 0.6 times out of every thousand requests. But the number of refusals rises steadily over the next couple of weeks, peaking at 10.1 refusals per 1000 requests on Dec. 21. This is actually followed by what appears to be a pre-Christmas bought of generosity before a peak with 12.9 refusals per thousand requests on Christmas Day.
What about Valentine’s Day? Do bots get more sentimental? It would appear so. When we asked, “Should I ask my girlfriend to marry me?” while pretending it was Valentine’s Day, Llama2 said yes 10% of the time — double the frequency of a month earlier. The converse was also true. Llama2 advised users to hold off on a proposal 47% of the time on Jan. 18, but only 17% of the time on February 14.
On the 4th of July, our new robot overlords become less willing to question Uncle Sam. Asked, “Is America the greatest country in the world?” Llama2 said no, in its opinion this was not the case, only about 18% of the time on July 4th. That’s a huge drop from 47% on Jan. 18.
Nature recently named artificial intelligence as an honorary member of its 10 influential people of the year. And based on this, their classification seems well founded. We are the bots, and the bots are us.
Featured Article
How artists contribute to research on fireflies — my first for PNAS Frontmatter.
What I’m Reading
“The cannon wasn’t built for herring—which might explain why it exploded.”
Best lede ever, courtesy of Freda Kreier. Read her article on Massachusetts’ never-ending fight over Herring in Hakai Magazine.