AI-Directed Research and the Loss of Scientific Direction

Article information

Int Neurourol J. 2026;30(3):173-177
Publication date (electronic) : 2026 September 30
doi : https://doi.org/10.5213/inj.2626edi03
1Department of Medical Informatics, Chung-Ang University, Seoul, Korea
2Department of Urology, Chung-Ang University Gwangmyeong Hospital, Gwangmyeong, Korea

The direction of scientific research is intricately related to how its questions are formulated and evaluated. What, then, becomes of this direction when artificial intelligence determines the question itself, and how does this translate into the conduct of research? Well recognized are the fabricated references and inaccurate text associated with artificial intelligence (AI)-generated research, serious but containable problems in terms of content quality and publication ethics [1, 2]. Less apparent is how the use of AI by both authors and reviewers, from generating the question to defending the manuscript, may come to determine the investigation itself. Through this repeated exchange, research may become enclosed within a cycle increasingly dependent on plausibility, conformity to existing work, and the likelihood of publication, allowing these qualities to determine its direction at the expense of its relationship to truth [3, 4].

The repeated adjustment of research through such a cycle can be understood in terms of convergence. In optimization theory, a system repeatedly adjusted toward an objective may arrive at a solution without the process itself establishing its correctness [5]. Despite appearing optimal within the range being explored, a solution reached with insufficient data or a constrained parameter space may represent a local minimum, i.e., a stable state that remains suboptimal or false when considered against a broader reality. To escape this state requires perturbation of the existing equilibrium, whether by new data, a change in the landscape, or an external influence sufficient to expose a deeper basin of attraction. Without such disturbance, local stability may indefinitely take the place of a global validity that has never been established (Fig. 1).

Fig. 1.

Stochastic gradient descent and Kuhnian scientific paradigms. Stochastic gradient descent may converge on a local minimum rather than the global optimum. In a similar manner, a Kuhnian paradigm allows scientific work to develop within a stable framework, while that stability may also confine the field to a locally coherent solution. This process occurs in conventional as well as artificial intelligence (AI)-driven science, with AI-mediated research potentially accelerating convergence and further entrenching the resulting paradigm.

Scientific research has long displayed a comparable pattern, as successive hypotheses, experiments, publications, and revisions in response to criticism gradually bring a field into what Kuhn [6] described as a paradigm. The shared assumptions, methods, and exemplary problems of this framework determine which questions are meaningful, which methods are legitimate, and, consequently, which explanations are considered adequate. In terms of optimization, the paradigm functions as a local minimum, allowing productive and internally coherent work within the assumptions it provides. Researchers inheriting these assumptions proceed to address the problems defined by them, with publication and review further reinforcing the framework from which the questions originated.

Such paradigms are, however, susceptible to the accumulation of observations that resist explanation within the existing framework [6]. As additional hypotheses are required to accommodate exceptions, the theory becomes increasingly cumbersome, its apparent coherence dependent on ad hoc explanations that eventually allow the accumulated evidence to destabilize the local minimum and make a paradigm shift possible. This is intrinsically dependent on time, as clinical observations must mature and longitudinal data accumulate, with independent attempts at replication allowed to succeed or fail before discrepancies can be distinguished from noise [7]. Through sustained contact with empirical reality, the discrepancy between the model and the world becomes too substantial to be accommodated within the framework. Thus, the recognition described by Kuhn is closely associated with prolonged empiri-cal testing, through which the limitations of a previously productive paradigm become apparent.

AI-directed research may reproduce this convergence at a rate detrimental to the empirical process required for its correction. With AI employed by both authors and reviewers throughout the research cycle, existing literature and familiar manuscript structures, together with reviewer expectations, citation patterns, and language considered scientifically acceptable, may assemble into a pseudo-paradigm [3, 4, 8]. Within this statistical and editorial framework, a question is selected for its resemblance to publishable research, its design for apparent defensibility, and its response to criticism for persuasiveness. The resulting investigation thus becomes increasingly dependent on publishability as its objective, displacing the search for truth and the clinical phenomenon that requires explanation.

The interaction between author and reviewer provides the means for this convergence. An author requesting a research topic from AI receives a question drawn from existing literature, carrying with it the resemblance to questions already investigated [9]. Further use of AI extends this familiarity into the study design and manuscript, emphasizing internal coherence while giving less prominence to conceptual difficulties. The reviewer, faced with a large volume of submissions, may similarly depend on AI to summarize the manuscript and identify familiar methodological weaknesses [3, 4]. These are then addressed by the author through another AI-generated response, measured and persuasive within the same conventions. Despite the reasonableness and possible efficiency of each use, the combined process narrows the investigation to questions and explanations that AI can generate and AI-assisted review will accept, allowing convergence on a local minimum without establishing the correctness of the solution itself [5] (Figs. 2, 3).

Fig. 2.

Coupling-induced synchronization. An illustrative coupled-oscillator model shows initially different phases aligning after coupling begins at the dotted line. (A) Individual oscillator signals. (B) Synchrony approaches 1 as the phases align. In the research analogy presented here, the use of shared artificial intelligence models and literature, together with feedback between authors and reviewers, couples their judgments around common conventions. Agreement can therefore arise through coupling alone, without establishing scientific validity. The simulation illustrates this relationship and does not provide empirical evidence of synchronization in research communities.

Fig. 3.

Self-distillation loop. (A) Generated papers re-enter the literature corpus used in subsequent artificial intelligence (AI)-mediated research. (B) Idealized distributions narrow around a conventional framing over successive cycles. This illustrates the proposed loss of conceptual diversity when recycled outputs are insufficiently challenged by independent evidence. The term “self-distillation” refers here to the feedback between the literature corpus, AI, and generated papers, without implying that each use retrains model weights. The curves are schematic and do not represent measured data.

Systems incorporating this loop have already been described by Lu et al. [10] in Nature, where The AI Scientist connects idea generation and literature search to computational experimentation, manuscript writing, and automated review. Its templatefree version further explores branches of experiments through an agentic tree search. Of three manuscripts selected by humans for submission to an ICLR (International Conference on Learning Representations) workshop, one received scores above the workshop’s acceptance threshold; all three were subsequently withdrawn under the study protocol. The system’s automated reviewer was itself evaluated against conference acceptance decisions.

As considered in a recent AutoResearch survey, this breadth of execution has yet to establish scientific autonomy [11]. Current pipelines remain limited in their capacity to revise the research question itself, leaving the scientific value of the work undetermined by completion of the workflow alone. Furthermore, clinical and biological validation retain demands specific to their respective fields, dependent on slower processes that faster computational execution cannot simply remove.

What, then, would allow feedback to make such a system abandon its initial question? To generate and evaluate research within the same conventions may allow increasing agreement to reinforce the existing attractor, without requiring its relationship to reality to be tested. The pipeline may thereby become more complete in its execution and more capable of developing the paper, while remaining unable to recognize that the question around which it is constructed is itself wrong.

Recent work on automated research incorporates the conventionally understood scientific process into a self-driving loop, potentially expanding discovery through a sufficiently broad search for questions. Even an extensive search, however, is dependent on the system’s representation of the world, and thus confined by what that representation allows it to consider. Without direct interaction with external ground truth, together with the capacity to revise its questions and assumptions through that interaction, the system remains within the representational space of inherited human research paradigms. To explore this space with unprecedented efficiency would not in itself provide recognition of what has been excluded from it.

The rapidity of convergence is therefore of concern in terms of the time available for research to encounter the real world [7, 12]. In conventional research, the interval between successive studies was occupied by clinical practice, failed treatments, unexpected patient outcomes, and difficult datasets, allowing the developing explanation to be disturbed by observations outside its immediate construction. This interval thus served as an opening through which clinical reality could exert its corrective pressure. As the interval contracts in AI-mediated research, a surprising result may be accommodated as a limitation, a weak association described as hypothesis-generating, or a clinically unimportant finding incorporated into a persuasive account, allowing the manuscript to become increasingly coherent while its connection to clinical reality weakens. The resulting convergence on a local minimum leaves insufficient time for studies and concepts to mature in the clinical setting, where their validity must be tested against patients, outcomes, and the irreducible complexity of disease.

By reducing the opportunity for such clinical discrepancies to influence subsequent work, the repeated exchange between AIassisted authors and reviewers may impair an essential corrective process in science. Traditional paradigms were susceptible to disruption as evidence accumulated over time, with clinicians continuing to encounter discrepancies already accommodated within the theory until the distance between the model and clinical reality became too great to ignore [6, 7]. If AI-directed research proceeds at a pace that prevents this accumulation from exerting its effect, the local minimum may persist indefinitely, or be escaped only after substantial clinical harm has occurred [8, 12].

The implications for the conduct of research extend to the initial recognition of the problem itself. To conduct research requires the capacity to identify a meaningful clinical problem and formulate an answerable question, with judgment of the data remaining accountable to the patient and the disease rather than exhausted by the production of a publishable manuscript. Of particular concern is how AI allows this initial recognition to be bypassed, providing a research gap, a manuscript outline, and responses to reviewers to a researcher who may remain unable to explain why the question matters. Having never taken ownership of the problem, the researcher is consequently ill-equipped to recognize when the proposed answer no longer corresponds to the clinical phenomenon.

Thus, the distinction between AI-assisted and AI-directed research is intrinsically related to the researcher’s responsibility for the investigation. To relegate the question, its framing, the anticipation of criticism, and the defense of the conclusion to AI would be to reduce this responsibility to the operation of a process for producing publications.

To retain this responsibility requires deliberate points of human judgment within research, without prohibiting the use of AI itself. Before its use, the researcher should provide a statement of ownership of the problem, identifying the clinical observation that prompted the investigation and why it matters, together with what remains unexplained by current knowledge. This should extend to the data needed to answer the question and the result that would lead to reconsideration of the initial position. AI may then examine the assumptions contained within that question, possible sources of invalidating bias, and the conditions under which the work would become clinically irrelevant, allowing scrutiny of a problem for which the researcher remains responsible. To request a publishable topic instead would bypass this responsibility at its outset. Each project must further return to ground-truth validation, whether through chart review, pathology confirmation, imaging verification, prospective follow-up, or renewed examination of the clinical phenomenon that motivated the work. In doing so, the researcher must identify what evidence would disprove the explanation, irrespective of the persuasiveness of the manuscript.

Only by retaining ownership of the clinical problem can the researcher recognize when an increasingly persuasive explanation has ceased to account for the phenomenon that prompted the investigation [6]. At this point, interruption of the AI-generated cycle allows the patient, the data, and the unresolved discrepancy to determine again what question should be asked.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

References

1. Bhattacharyya M, Miller VM, Bhattacharyya D, Miller LE. High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus 2023;15e39238.
2. Athaluri SA, Manthena SV, Kesapragada VSRKM, Yarlagadda V, Dave T, Duddumpudi RTS. Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references. Cureus 2023;15e37432.
3. Liang W, Izzo Z, Zhang Y, Zhang Y, Lepp H, Cao H, et al. Monitoring AI-modified content at scale: a case study on the impact of ChatGPT on AI conference peer reviews. ICML’24: Proceedings of the 41st International Conference on Machine Learning. PMLR 2024;235:29575–620.
4. Hosseini M, Horbach SPJM. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Res Integr Peer Rev 2023;8:4.
5. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge (MA): MIT Press; 2016. Chapter 8, Optimization for training deep models; p. 271-325.
6. Kuhn TS. The structure of scientific revolutions Chicago (IL): University of Chicago Press; 1962.
7. Ioannidis JP. Why most published research findings are false. PLoS Med 2005;2e124.
8. Mollaki V. Death of a reviewer or death of peer review integrity? The challenges of using AI tools in peer reviewing and the need to go beyond publishing policies. Res Ethics 2024;20:239–50.
9. Thorp HH. ChatGPT is fun, but not an author. Science 2023;379:313.
10. Lu C, Lu C, Lange RT, Yamada Y, Hu S, Foerster J, et al. Towards end-to-end automation of AI research. Nature 2026;651:914–9.
11. Tie G, Shi J, Song D, Huang Y, Sheng Z, Zhou X, et al. AutoResearch AI: towards AI-powered research automation for scientific discovery. arXiv [Preprint] 2026;[cited 2026 Sep 10]. Available from: https://doi.org/10.48550/arXiv.2605.23204.
12. Smaldino PE, McElreath R. The natural selection of bad science. R Soc Open Sci 2016;3:160384.

Article information Continued

Fig. 1.

Stochastic gradient descent and Kuhnian scientific paradigms. Stochastic gradient descent may converge on a local minimum rather than the global optimum. In a similar manner, a Kuhnian paradigm allows scientific work to develop within a stable framework, while that stability may also confine the field to a locally coherent solution. This process occurs in conventional as well as artificial intelligence (AI)-driven science, with AI-mediated research potentially accelerating convergence and further entrenching the resulting paradigm.

Fig. 2.

Coupling-induced synchronization. An illustrative coupled-oscillator model shows initially different phases aligning after coupling begins at the dotted line. (A) Individual oscillator signals. (B) Synchrony approaches 1 as the phases align. In the research analogy presented here, the use of shared artificial intelligence models and literature, together with feedback between authors and reviewers, couples their judgments around common conventions. Agreement can therefore arise through coupling alone, without establishing scientific validity. The simulation illustrates this relationship and does not provide empirical evidence of synchronization in research communities.

Fig. 3.

Self-distillation loop. (A) Generated papers re-enter the literature corpus used in subsequent artificial intelligence (AI)-mediated research. (B) Idealized distributions narrow around a conventional framing over successive cycles. This illustrates the proposed loss of conceptual diversity when recycled outputs are insufficiently challenged by independent evidence. The term “self-distillation” refers here to the feedback between the literature corpus, AI, and generated papers, without implying that each use retrains model weights. The curves are schematic and do not represent measured data.