ChatGPT is Writing Scientific Research Papers

Learn more about the new policies that were developed to manage the risk AI-generated content poses to scientific research.

AUSTIN, TEXAS, CZECH REPUBLIC, September 5, 2024 /EINPresswire.com/ -- Are Scientific Journals Becoming Flooded with AI-Generated Research Papers?

Since the introduction of ChatGPT, many in the scientific research community have been trying to evaluate how to incorporate the benefits of AI into their research workflow without tripping over ethically problematic areas.

The experience of Roy Kishony and his team at the Technion Israel Institute of Technology in Haifa may be typical: As an experiment, Kishony tasked ChatGPT with writing a paper from scratch that evaluated a publicly available CDC data set. Kishony found the resulting paper lacked the kind of unique insights that form a useful paper.

In a survey conducted by Nature, over 1,600 readers weighed in on the pros and cons of using AI in their research work. Many reported positive findings in using AI thanks to it providing helpful, timesaving work – such as generating ideas for hypotheses or optimizing experimental setups. But they also cautioned that AI posed an ethical risk by making it easier to commit fraud or to create irreproducible and/or biased research results.

Sleuthing the Internet to Ferret Out Scientific Papers Written by AI

How many scientific papers are being written by AI?

In late 2022, Ansible Health submitted one of the first papers that intentionally listed ChatGPT as one of the co-authors.

The move generated lots of controversy (perhaps this was their intention?), and suddenly, publications began to quickly amend their submission guidelines to disavow and exclude the very few AI-generated papers they had received to date.

Given the backlash against listing AI as a co-author (more on the policy changes made by journals and government agencies later), we can assume that most recent papers “touched” by AI have been submitted without acknowledging AI’s supporting role.

But sometimes, it’s still possible to identify the hidden hand of AI.

In one hilarious example, a published paper was “outed” as AI-generated because it mistakenly included the original prompt used to generate the paper’s introductory paragraph:

“Certainly, here is a possible introduction for your topic.”

Internet sleuths have stepped into the breach to assist in identifying AI-generated papers.

Two of the prominent internet sleuths who have made it their life’s work to identify problematic research papers are David Bimler (who goes by the online name of “Smut Clyde”) and Elizabeth Bik, a former researcher who now works full-time uncovering faked image data.

Bimler and Bik both got their start investigating problematic, low-quality scientific articles pumped out by so-called “paper mills” – an ongoing problem that pre-dates the introduction of ChatGPT.

Other researchers are using statistical word distribution methods to identify AI-generated papers.

Researcher Andrew Gray of the University College London began to notice a statistically significant rise in the occurrence of certain words, such as “meticulous,” “commendable,” and “intricate” (up 137% since 2022), which may be telltale signs of AI writing. Researchers at Stanford have noticed a similar phenomenon, with the use of the words “realm,” “showcasing,” “intricate,” and “pivotal” up as much as 160%.

Based on these statistics, Gray hypothesizes that at least 1% of all papers published in 2023 were partially written by AI. The Stanford researchers believe the number could be much higher, up to 6.3% of papers in mathematics and natural science and up to 17.5% in computer science articles.

AI-drafted Scientific Articles Can Introduce Inaccuracies and Discrepancies Deep within a Paper’s Reasoning, Supporting Diagrams, and Data Sources

Unfortunately, indiscriminate or unethical use of AI can introduce errors and omissions deep within a scientific research paper – and these can be more difficult to identify without careful case-by-case review.

This certainly makes the job of a peer reviewer more difficult.

In some cases, logic errors and other anomalies found in AI-generated research papers can be traced back to the “hallucination” problem inherent in many large language models (LLMs) that form the backbone of ChatGPT and other AI chat systems.

Here, we can think of hallucinations as a euphemism for “making up facts” to bridge a gap in the knowledge of the AI system.

Another area to be on the lookout for is faked data representations, either stemming from the AI creating what are essentially fraudulent data sets that can’t be reproduced or making changes in the AI-generated data charts and diagrams that don’t accurately reflect the underlying data.

As an aside, the problem with fake data may be with us for some time. That’s because AI systems are increasingly relying on what’s called ‘synthetic data’ for training. In the case of training AI systems, synthetic data is OK, but when it spills over into research articles, it’s bad.

Read more...

Julia Solodovnikova
Formaspace
+ +1 800-251-1505
email us here
Visit us on social media:
Facebook
X
LinkedIn