Budapest Post

Cum Deo pro Patria et Libertate
Budapest, Europe and world news

A teenager on Tiktok disrupted thousands of scientific studies with a single video

Researchers were caught by surprise after a short video sent a flood of new users to a survey platform. Thousands of scientific studies had to toss out weeks of data because of a 56-second TikTok video by a teenager.

The July 23rd video is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera.

“Welcome to side hustles I recommend trying — part one,” she says in the video, pointing users to the website Prolific.co. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.”


That video got 4.1 million views in the month after it was posted and sent tens of thousands of new users flooding to the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tools in place to make sure that it delivered representative population samples to each study. Suddenly, scientists used to getting a wide mix of subjects for their Prolific studies saw their surveys flooded with responses from young women around Frank’s age.

For researchers who rely on representative samples of the US population, that demographic shift was a major problem with no obvious cause and no immediately clear way to fix.

Doing science on the internet


Though not particularly well known, Prolific is part of a small collection of online tools that have transformed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, which was released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after it was released, behavioral scientists realized its potential value for their research, and it quickly revolutionized several research fields.

“Before Mechanical Turk existed, all social science research had to happen in the laboratory. You’d need to bring in college sophomores and put them through questionnaires and surveys and whatnot,” said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business.

“That’s an enormous time- and labor-intensive endeavor. Online research makes it so much easier. You program a survey... you put it online, and within a day, you have 1,000 responses,” Hall told The Verge. “That changed the face of social science.”

The Behavioral Lab at Stanford mainly uses the newer, smaller Prolific platform for online studies these days, Hall said. While many Mechanical Turk customers are big businesses conducting corporate research, Prolific gears its product to scientists.

The smaller platform offers more transparency, promises to treat survey participants more ethically, and promises higher-quality research subjects than alternative platforms like Mechanical Turk.

Scientists doing this sort of research in the United States generally want a pool of subjects who speak English as a first language, are not too practiced at taking psychological surveys, and together make up a reasonably representative demographic sample of the American population.

Prolific, most agreed, did a good job providing high-quality subjects. The sudden change in the platform’s demographics threatened to upend that reputation.

TikTok aftermath


In the days and weeks after Frank posted her video, researchers scrambled to figure out what was happening to their studies.


A member of the Stanford Behavioral Laboratory posted on a Prolific forum, “We have noticed a huge leap in the number of participants on the platform in the US Pool, from 40k to 80k. Which is great, however, now a lot of our studies have a gender skew where maybe 85% of participants are women. Plus the age has been averaging around 21.”

Wayne State psychologist Hannah Schechter seems to have been the first person to crack the case.

“This may be far-fetched,” she tweeted, linking to Frank’s video, “but given the timing, virality of the video, and the user’s follower demographics....”

Long-standing Prolific survey-takers complained on Reddit that Frank had made it difficult to find paid surveys to take on the overrun platform.

“Now it’s just another bullshit site to spend hours and make pennies on,” wrote one user, who said they had previously made $30 a week on the platform.

Frank, who “guesstimated” she had made a total of about $80 taking surveys on Prolific before her video, told The Verge she also noticed a difference on the platform.

“Less studies have been available for me and everyone else,” she told The Verge. “I’ve received some really mean comments accusing me of single-handedly ruining the site and being selfish — even though I received no compensation for that video.”

She added that she hoped Prolific would be able to set up a system to deal with its changed demographics.

“I also predict that a lot of people who signed up after seeing my video will forget about it, and the surge will die down,” she said.

Prolific co-founder and CTO Phelim Bradley told The Verge that many of the new users do seem to be dropping off.

“Prior to Tiktok, about 50% of the responses on our platform came from women,” he wrote in an email. “The surge knocked this up as high as 75% for a few days, but since then, this number has been trending down, and we’re currently back to ~60% of responses being from women.”

A chart showing the increase in female users over time.


According to Bradley, about 4,600 studies were disrupted by Frank’s TikTok, around a third of the total that were active on the platform during the surge. Of those, he said, the vast majority should be salvageable.

Prolific has refunded researchers whose studies were significantly impacted by the surge in women survey takers and introduced a new suite of demographic screening tools. The company announced these steps a month after Frank posted her video. The company has now re-organized, putting a team in charge of demographic balancing in order to more quickly recognize and respond to this sort of problem in the future.

“Honestly, we were somewhat caught by surprise, and we didn’t predict how large the impact was going to be,” Bradley said.

The surge isn’t all bad. Refreshing the pool of survey takers probably has long-term benefits, says Vlad Chituk, a Yale graduate student in psychology who was running several pilot studies on Prolific when the surge hit. When subjects take lots of psychological surveys, they learn the tricks scientists use to gather data, and that can impact the way they respond to future survey questions. Fresh subjects provide higher-quality data.

“Young women who enjoy TikTok are people too,” he said.

As for Frank, she says that her side hustle video is now the most popular TikTok she’s ever posted.

“It definitely didn’t occur to me that the video would blow up. I just posted it for my friends and followers, not for the reach it ended up getting,” she said. “I think it blew up because the site is genuinely so cool, and people love efficient ways to make money.”

For the time being, Frank has paused most of her own side hustles as she settles into her freshman year at Brown.

AI Disclaimer: An advanced artificial intelligence (AI) system generated the content of this page on its own. This innovative technology conducts extensive research from a variety of reliable sources, performs rigorous fact-checking and verification, cleans up and balances biased or manipulated content, and presents a minimal factual summary that is just enough yet essential for you to function as an informed and educated citizen. Please keep in mind, however, that this system is an evolving technology, and as a result, the article may contain accidental inaccuracies or errors. We urge you to help us improve our site by reporting any inaccuracies you find using the "Contact Us" link at the bottom of this page. Your helpful feedback helps us improve our system and deliver more precise content. When you find an article of interest here, please look for the full and extensive coverage of this topic in traditional news sources, as they are written by professional journalists that we try to support, not replace. We appreciate your understanding and assistance.
Newsletter

Related Articles

0:00
0:00
Close
UK Government Considers Dropping Demand for Apple Encryption Backdoor
Japanese Man Discovers Family Connection Through DNA Testing After Decades of Separation
Russia Signals Openness to Ukraine Peace Talks Amid Escalating Drone Warfare
Switzerland Implements Ban on Mammography Screening
Pogacar Extends Dominance with Stage Fifteen Triumph at Tour de France
President Trump Diagnosed with Chronic Venous Insufficiency After Leg Swelling
CEO Resigns Amid Controversy Over Relationship with HR Executive
NVIDIA Achieves $4 Trillion Valuation Amid AI Demand
Tulsi Gabbard Unveils Evidence Alleging Political Manipulation of Intelligence During Trump Administration
Centrist Criticism of von der Leyen Resurfaces as she Survives EU Confidence Vote
Trump Announces Coca-Cola to Shift to Cane Sugar in U.S. Production
FIFA Pressured to Rethink World Cup Calendar Due to Climate Change
Zelensky Reshuffles Cabinet to Win Support at Home and in Washington
"Can You Hit Moscow?" Trump Asked Zelensky To Make Putin "Feel The Pain"
Church of England Removes 1991 Sexuality Guidelines from Clergy Selection
Superman Franchise Achieves Success with Latest Release
Hungary's Viktor Orban Rejects Agreements on Illegal Migration
Air India Pilot’s Mental Health Records Under Scrutiny
Jamie Dimon Warns Europe Is Losing Global Competitiveness and Flags Market Complacency
Moonshot AI Unveils Kimi K2: A New Open-Source AI Model
Martha Wells Says Humanity Still Far from True Artificial Intelligence
Nvidia Becomes World’s First Four‑Trillion‑Dollar Company Amid AI Boom
EU Delays Retaliatory Tariffs Amid New U.S. Threats on Imports
Trump Proposes Supplying Arms to Ukraine Through NATO Allies
US Opens First Rare Earth Mine in Over 70 Years in Wyoming
Bitcoin Reaches New Milestone of $116,000
Severe Heatwave Claims 2,300 Lives Across Europe
Declining Beer Consumption Signals Cultural Shift in Germany
Emails Leaked: How Passenger Luggage Became a Side Income for Airport Workers
Polish MEP: “Dear Leftists - China is laughing at you, Russia is laughing, India is laughing”
Western Europe Records Hottest June on Record
BRICS Expands Membership with Indonesia and Ten New Partner Countries
Elon Musk Founds a Party Following a Poll on X: "You Wanted It – You Got It!"
China’s Central Bank Consults European Peers on Low-Rate Strategies
France Requests Airlines to Cut Flights at Paris Airports Amid Planned Air Traffic Controller Strike
Poland Implements Border Checks Amid Growing Migration Tensions
Emirates Airline Expands Market Share with New $20 Million Campaign
Amazon Reaches Milestone with Deployment of One Millionth Robot
Yulia Putintseva Calls for Spectator Ejection at Wimbledon Over Safety Concerns
House Oversight Committee Subpoenas Former Jill Biden Aide Amid Investigation into Alleged Concealment of President Biden's Cognitive Health
Amazon Reaches Major Automation Milestone with Over One Million Robots
Extreme Heat Wave Sweeps Across Europe, Hitting Record Temperatures
Meta Announces Formation of Ambitious AI Unit, Meta Superintelligence Labs
Robots Compete in Football Tournament in China Amid Injuries
China Unveils Miniature Insect-Like Surveillance Drone
Marc Marquez Claims Victory at Dutch Grand Prix Amidst Family Misfortune
Germany Votes to Suspend Family Reunification for Asylum Seekers
Budapest Pride Parade Draws 200,000 Participants Amid Government Ban
Southern Europe Experiences Extreme Heat
Xiaomi's YU7 SUV Launch Garners Record Pre-Orders Amid Market Challenges
×