Problematic citation practices

[Upasunda]

It seems to me as if the AI Scientist suffer from the same problem as other research oriented tools which are based on LLM inferencing and draws their research backround information from Semantic Scholar. While this judgement could very well be field specific, it still seems to pose quite a large problem for the turstworthiness of academic research and its disemniation. The issue mainly seems to pertain to how the model reads and understands the input text and which aspects of it should be attributed to what author.

If we look at the generated paper "DualScale Diffusion: Adaptive Feature Balancing for Low-Dimensional Generative Models" and the very first citation made in the introduction:

"Diffusion models have emerged as a powerful class of generative models, achieving state-of-the-art results in various domains such as image synthesis, audio generation, and molecular design Yang et al. (2023)."

This is, even with the citation, borderline plagiarism of the first sentence of the abstract in the source:

"Diffusion models have emerged as a powerful new family of deep generative models with record-breaking performance in many applications, including image synthesis, video generation, and molecule design."

That isn't, however, the main issue, which instead is that Yang et al. is cited as the primary source of the statement, which I would guess is related to how the model approached the text and wether the model had access to the entire text or just the abstract. Becuase, if we read the very first sentence of Introduction in the cited source, it is clear that Yang et al. is but a secondary source on the asserted statement:

Diffusion models [ 105, 263, 268, 273] have emerged as the new state-of-the-art family of deep generative models. They have broken the long-time dominance of generative adversarial networks (GANs) [83 ] in the challenging task of image synthesis [58 , 105, 268, 273 ] and have also shown potential in a variety of domains, ranging from computer vision [ 3 , 13, 23 , 27 , 106, 108, 139, 143, 164, 184, 196, 215, 247, 249, 300, 335, 336, 358, 368 ], natural language processing [8, 111, 168, 254, 342 ], temporal data modeling [ 2, 37 , 153, 239, 279, 319], multi-modal modeling [9, 233, 245, 248, 365], robust machine learning [21, 31, 138, 293, 338], to interdisciplinary applications in fields such as computational chemistry [4, 109, 127, 160, 162, 186, 312] and medical image reconstruction [29, 45–47, 51, 192, 220, 272, 313].

Essentially, what the AI Scientist is doing here is ascribing the work of numerous other researchers to someone who has condensed their work into a statement in an abstract. This, in some instances, could be considered research misconduct, depending on national regulations. It is, however, independet of national regulations, bad research practice.

So, while the AI Scientist is an interesting project and indeed show potential, it still seems, at least to me, to be facing the same problem as other initiatives. Perhaps a way to leaviate this could be to, if possible, hook it up to other APIs, like the SCOPUS, Taylor & Francis, etc. and do full text retrievals, embed the fulltexts in some database and query the entire source papers for information when writing the paper? I guess that each in-text citation in the text could be connected to some metadata containing the correct - primary - source to facilitate the paper writing for the model.

[conglu1997]

This is on our radar and we have some upcoming solutions :)

[Upasunda]

Sweet =) I'm really looking forward to see how you're going to solve it with consistency.
Great and needed project!

[BradKML]

Perhaps document RAG (picking the best papers) and graph-based thinking agents (the papers that aggregates the ideas) would be useful? But which libraries are the best in this regard?