Skip to content

Commit a1a4a45

Browse files
authored
Apply suggestion from @lwasser
1 parent 9a576bf commit a1a4a45

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

_posts/2025-09-16-generative-ai-peer-review.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,10 @@ LLMs are already perceived as helping developers:
9898

9999
Some contributors perceive these products as making open source more accessible.
100100

101+
### Incorrectness of LLMs and misleading time benefits
102+
103+
Although it is commonly stated that LLMs help improve the productivity of high-level developers, recent scientific explorations of this hypothesis indicate the contrary (see https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ for an excellent discussion on this). What's more is that the responses of LLM's for complex coding tasks tend to be incorrect (e.g., https://arxiv.org/html/2407.06153v1). Therefore, it is crucial that, if an LLM is used to help produce code, that the correctness of the code is evaluated separately from the LLM.
104+
101105
## Challenges we must address
102106

103107
### Overloaded peer review

0 commit comments

Comments
 (0)