Budapest Post

Cum Deo pro Patria et Libertate
Budapest, Europe and world news

A teenager on Tiktok disrupted thousands of scientific studies with a single video

Researchers were caught by surprise after a short video sent a flood of new users to a survey platform. Thousands of scientific studies had to toss out weeks of data because of a 56-second TikTok video by a teenager.

The July 23rd video is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera.

“Welcome to side hustles I recommend trying — part one,” she says in the video, pointing users to the website Prolific.co. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.”


That video got 4.1 million views in the month after it was posted and sent tens of thousands of new users flooding to the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tools in place to make sure that it delivered representative population samples to each study. Suddenly, scientists used to getting a wide mix of subjects for their Prolific studies saw their surveys flooded with responses from young women around Frank’s age.

For researchers who rely on representative samples of the US population, that demographic shift was a major problem with no obvious cause and no immediately clear way to fix.

Doing science on the internet


Though not particularly well known, Prolific is part of a small collection of online tools that have transformed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, which was released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after it was released, behavioral scientists realized its potential value for their research, and it quickly revolutionized several research fields.

“Before Mechanical Turk existed, all social science research had to happen in the laboratory. You’d need to bring in college sophomores and put them through questionnaires and surveys and whatnot,” said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business.

“That’s an enormous time- and labor-intensive endeavor. Online research makes it so much easier. You program a survey... you put it online, and within a day, you have 1,000 responses,” Hall told The Verge. “That changed the face of social science.”

The Behavioral Lab at Stanford mainly uses the newer, smaller Prolific platform for online studies these days, Hall said. While many Mechanical Turk customers are big businesses conducting corporate research, Prolific gears its product to scientists.

The smaller platform offers more transparency, promises to treat survey participants more ethically, and promises higher-quality research subjects than alternative platforms like Mechanical Turk.

Scientists doing this sort of research in the United States generally want a pool of subjects who speak English as a first language, are not too practiced at taking psychological surveys, and together make up a reasonably representative demographic sample of the American population.

Prolific, most agreed, did a good job providing high-quality subjects. The sudden change in the platform’s demographics threatened to upend that reputation.

TikTok aftermath


In the days and weeks after Frank posted her video, researchers scrambled to figure out what was happening to their studies.


A member of the Stanford Behavioral Laboratory posted on a Prolific forum, “We have noticed a huge leap in the number of participants on the platform in the US Pool, from 40k to 80k. Which is great, however, now a lot of our studies have a gender skew where maybe 85% of participants are women. Plus the age has been averaging around 21.”

Wayne State psychologist Hannah Schechter seems to have been the first person to crack the case.

“This may be far-fetched,” she tweeted, linking to Frank’s video, “but given the timing, virality of the video, and the user’s follower demographics....”

Long-standing Prolific survey-takers complained on Reddit that Frank had made it difficult to find paid surveys to take on the overrun platform.

“Now it’s just another bullshit site to spend hours and make pennies on,” wrote one user, who said they had previously made $30 a week on the platform.

Frank, who “guesstimated” she had made a total of about $80 taking surveys on Prolific before her video, told The Verge she also noticed a difference on the platform.

“Less studies have been available for me and everyone else,” she told The Verge. “I’ve received some really mean comments accusing me of single-handedly ruining the site and being selfish — even though I received no compensation for that video.”

She added that she hoped Prolific would be able to set up a system to deal with its changed demographics.

“I also predict that a lot of people who signed up after seeing my video will forget about it, and the surge will die down,” she said.

Prolific co-founder and CTO Phelim Bradley told The Verge that many of the new users do seem to be dropping off.

“Prior to Tiktok, about 50% of the responses on our platform came from women,” he wrote in an email. “The surge knocked this up as high as 75% for a few days, but since then, this number has been trending down, and we’re currently back to ~60% of responses being from women.”

A chart showing the increase in female users over time.


According to Bradley, about 4,600 studies were disrupted by Frank’s TikTok, around a third of the total that were active on the platform during the surge. Of those, he said, the vast majority should be salvageable.

Prolific has refunded researchers whose studies were significantly impacted by the surge in women survey takers and introduced a new suite of demographic screening tools. The company announced these steps a month after Frank posted her video. The company has now re-organized, putting a team in charge of demographic balancing in order to more quickly recognize and respond to this sort of problem in the future.

“Honestly, we were somewhat caught by surprise, and we didn’t predict how large the impact was going to be,” Bradley said.

The surge isn’t all bad. Refreshing the pool of survey takers probably has long-term benefits, says Vlad Chituk, a Yale graduate student in psychology who was running several pilot studies on Prolific when the surge hit. When subjects take lots of psychological surveys, they learn the tricks scientists use to gather data, and that can impact the way they respond to future survey questions. Fresh subjects provide higher-quality data.

“Young women who enjoy TikTok are people too,” he said.

As for Frank, she says that her side hustle video is now the most popular TikTok she’s ever posted.

“It definitely didn’t occur to me that the video would blow up. I just posted it for my friends and followers, not for the reach it ended up getting,” she said. “I think it blew up because the site is genuinely so cool, and people love efficient ways to make money.”

For the time being, Frank has paused most of her own side hustles as she settles into her freshman year at Brown.

AI Disclaimer: An advanced artificial intelligence (AI) system generated the content of this page on its own. This innovative technology conducts extensive research from a variety of reliable sources, performs rigorous fact-checking and verification, cleans up and balances biased or manipulated content, and presents a minimal factual summary that is just enough yet essential for you to function as an informed and educated citizen. Please keep in mind, however, that this system is an evolving technology, and as a result, the article may contain accidental inaccuracies or errors. We urge you to help us improve our site by reporting any inaccuracies you find using the "Contact Us" link at the bottom of this page. Your helpful feedback helps us improve our system and deliver more precise content. When you find an article of interest here, please look for the full and extensive coverage of this topic in traditional news sources, as they are written by professional journalists that we try to support, not replace. We appreciate your understanding and assistance.
Newsletter

Related Articles

0:00
0:00
Close
Putin Celebrates ‘Unprecedentedly High’ Ties with China as Gazprom Seals Power of Siberia-2 Deal
China Unveils New Weapons in Grand Military Parade as Xi Hosts Putin and Kim
Rapper Cardi B Cleared of Liability in Los Angeles Civil Assault Trial
Google Avoids Break-Up in U.S. Antitrust Case as Stocks Rise
Couple celebrates 80th wedding anniversary at assisted living facility in Lancaster
Information Warfare in the Age of AI: How Language Models Become Targets and Tools
The White House on LinkedIn Has Changed Their Profile Picture to Donald Trump
"Insulted the Prophet Muhammad": Woman Burned Alive by Angry Mob in Niger State, Nigeria
Trump Responds to Death Rumors – Announces 'Missile City'
Druzhba Pipeline Incident Sparks Geopolitical Tensions
Cost of Opposition Leader Péter Magyar's Economic Plan Revealed
Germany in Turmoil: Ukrainian Teenage Girl Pushed to Death by Illegal Iraqi Migrant
United Krack down on human rights: Graham Linehan Arrested at Heathrow Over Three X Posts, Hospitalised, Released on Bail with Posting Ban
Asian and Middle Eastern Investors Avoid US Markets
Ray Dalio Warns of US Shift to Autocracy
Eurozone Inflation Rises to 2.1% in August
Russia and China Sign New Gas Pipeline Deal
Von der Leyen's Plane Hit by Suspected Russian GPS Interference in an Incident Believed to Be Caused by Russia or by Pro-Peace or by Anti-Corruption European Activists
China's Robotics Industry Fuels Export Surge
Suntory Chairman Resigns After Police Probe
Gold Price Hits New All-Time Record
UK Fintechs Explore Buying US Banks
Greece Suspends 5% of Schools as Birth Rate Drops
Apollo to Launch $5 Billion Sports Investment Vehicle
Bolsonaro Trial Nears Close Amid US-Brazil Tension
European Banks Push for Lower Cross-Border Barriers
Poland's Offshore Wind Sector Attracts Investors
Budapest Central European Fashion Week Kicks Off
U.S. Celebrates Labor Day
Hungarian National Team Captain Scores Epic Goal
EU is getting aggressive: Four AfD Candidates Die Unexpectedly Ahead of North Rhine-Westphalia Local Elections
Japanese Customer Sways from VW to BYD after “Unbelievable” Test Drive amid Dealership Expansion
Nestlé Removes CEO Laurent Freixe Following Undisclosed Relationship with Subordinate
Pickles are the latest craze among Generation Z in the United States.
Giuliani Seriously Injured in Accident – Trump to Award Him the Presidential Medal of Freedom
Deadline Day Delivers Record £125m Isak Move and Donnarumma to City
Nvidia Reveals: Two Mystery Customers Account for About 40% of Revenue
Woody Allen: "I Would Be Happy to Direct Trump Again in a Film"
Lula and Putin Hold Strategic BRICS Discussions Ahead of Trump–Putin Summit
White House Eyes Budapest for Peace Talks
Cave Diving Beneath the Streets of Budapest
Another American Restaurant Chain Opens in Budapest
Hungarian Opposition Politician Supports Ukrainian Commander
Opposition Leader Threatens Media Outlets
American Airlines Adds New Flights to Budapest
F1 Hungarian Grand Prix Wraps Up
WhatsApp is rolling out a feature that looks a lot like Telegram.
U.S. Trade Representative says Washington still negotiating trade deals after court rules tariffs illegal
Von der Leyen says Europe drawing up 'precise' plans to send troops to Ukraine
Kremlin accuses Europe of hindering Trump’s peace efforts in Ukraine
×