Two now 3 months old O’RLY book puns “Getting ChatGPT to write your code” / “Copying and Pasting from ChatGPT”
Posted by jpluimers on 2023/04/04
Earlier this week I got reminded of the “book” so many people seem to fall for via the Tweet by [Wayback/Archive] turbo (@turboCodr) / Twitter.
The image (and text) is in fact a parody both on ChatGPT and on the Stack Overflow meme it is based on (more on my opinion on both further below).
Back to the book title referred by [Wayback/Archive] turbo on Twitter: “Something something last tech book you’ll ever buy”.:
Deploying untested code at break-neck speeds
Essential
Copying and Pasting from ChatGPT
O’REILLY
The Practical Developer
I remember having seen it way earlier so I mentioned the very first mentioning of “Copying and Pasting from ChatGPT” I could find in two replies:
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@dnsmichi @kwootman Likely…”
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@dnsmichi @kwootman A textual “Copying and pasting from chatGPT” comment to the original StackOverflow reference was already made on January 12, 2023: …”
That textual response was in fact a response to the much older meme:
Cutting corners to meet arbitrary management deadlines
Essential
Copying and Pasting from Stack Overflow
O’REILLY®
The Practical Developer @ThePracticalDev
as a reply on 2023-01-12 to [Wayback/Archive] Copying and Pasting from Stack Overflow (Essential) : ProgrammerHumor
Copying and pasting from chatGPT
It appeared around the same time as [Wayback/Archive] I wonder if it wrote the book itself : ProgrammerHumor
Hey, it’s practically the same as hiring an intern!
Getting ChatGPT to write your code
O’RLY?
Artie Fishallin-Teligents
Somehow, right before april 1st, the first meme became really popular, for instance appearing in:
- 2023-03-30: [Wayback/Archive] Last-ever programming book just released by Oreilly : ProgrammerHumor (reddit)
- 2023-03-31: [Wayback/Archive] Volker Weber: “The last O’REILLY book Copyin…” – Heise Medien on Mastodon
Google Search cannot yet keep up with them as currently quite a few of the below results are in fact about StackOverflow:
- [Wayback/Archive] “essential” “copying and pasting from chatgpt” – Google Search
- [Wayback/Archive] “essential” “copying and pasting from chatgpt” – Google Image Search
The first query also returns a few other interesting posts on why you should be careful when using ChatGPT.
In addition it returns a few other good posts of which I will quote from [Wayback/Archive] Why AI in Digital Marketing Won’t Put You Out of a Job | TEAM LEWIS which the first as one of the cons:
- Hazy plagiarism lines. Because text generators pull snippets from the entire Internet’s worth of sources, one 600-word AI-written blog post could borrow ideas from dozens of authors. Similarly, AI art generators gain inspiration from hundreds of years of art history. Best practice dictates that creators correctly attribute their sources. But when it’s unclear where the information comes from, originality and accuracy lags. Copying and pasting from ChatGPT is just as unethical as copying and pasting from a human author.
followed by
- The “it’s-good-enough” trap.
- Stretching truths, casting doubt.
Stack Overflow meme from 2016
The odd thing is that doing a Google Search by Image on the first image in fact turned back quite a few Stack Overflow meme references.
The earliest I could find was in a Tweet from March 2016: [Wayback/Archive] DEV Community on Twitter: “The last programming book you’ll ever need”
Later it gots used on a lot of blog posts of which a these two are in chronological order:
- [Wayback/Archive] Copy-and-Paste Programming | Effective Software Design
What would we do without Stack Overflow?
- [Wayback/Archive] Custom SQLFORMATs with SQLcl
But in this case, it’s OK. Because, I’m copying and pasting code from OUR GitHub project, AND my boss wrote the code.
I like both points made, so lets elaborate a bit more on both ways of copying/pasting code.
My opinions on ChatGPT and Stack Exchange sites like Stack Overflow
I used to be really active on Stack Overflow and am still occasionally contributing to some of the Stack Exchange network sites (which coincidentally are owned by the Dutch Prosus N.V.) for the same reason I stopped being active on Facebook and its groups, usenet groups, most if not all forums, and before that FidoNet echomail forums: egos playing a too important role and the declining feeling of a “common cause”.
Smaller Stack Exchange sites are still cool for me, as they are basically self moderating and the voting mechanisms really work for keeping quality up. This is unlike Experts-Exchange which was the reason for Stack Overflow to get started in the first place..
The drawback of anything published (including the internet and specifically Stack Exchange) is that hardly anybody revisits past content and updates it to the current state of the art.
This brings me back to ChatGPT: it is trained with a corpus of text that is quite unselective on quality. This basically averages out the knowledge that ChatGPT can expose.
By plain copying and pasting from ChatGPT you basically indicate that you have no other means to post average material at best.
Furthermore, since you do not know the source of the text that ChatGPT responds with, you have to check it yourself in a similar way you have to check Google Search results.
This apart from the generic shortcomings of a large language model: it is centered around human language, not programming language.
The minimum you have to do with code you copy from the internet is write adequate test code. Given how difficult even non-novice programmers have to write such test code I wonder how below average programmers will behave here.
Earlier tweets from me on both meme targets
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@jpagroenen Dat komt omdat ChatGPT put uit een zee van teksten die de kwaliteit uitmiddelt.” (that is because ChatGPT got trained from an ocean of texts which averages the quality)
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@marcowobben @benjedwards Let me elaborate why it is scary: For us, it is already consuming tremendous amounts of energy to check on all the stories of a single person. Checking every potentially “reused” ChatGPT output will need impossible amounts of energy.” (on why ChatGPT generated answers will be banned from the Stack Exchange network; read the rest of the messages in the ) which means I agree with[Wayback/Archive] Marco Wobben on Twitter: “@benjedwards @jpluimers My concern is the output not getting verified enough by knowledgeable experts. I’ve played with it a little. It is impressive, but at the same time it becomes harder to distinguish ‘accuracy’ from the ‘almost correct’, ánd to detect the error in between.” and[Wayback/Archive] Christian Findlay on Twitter: “@jpluimers @emilymbender @Felienne If you’re picking up code from anywhere, the appropriate level of suspicion is red alert, and it always has been. The remedy is good tests. It’s pointless to panic about the source of code. The most important thing is that the tests prove the correctness”
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@svpino @dvsoest How much can you, especially as novice developer or developer with a background in a different computer language, trust ChatGPT to reason about critical code when it is making so many mistakes with reasoning about natural language?”
- [Wayback/Archive] Jeroen Wiert Pluimers @wiert@mastodon.social on Twitter: “@GergelyOrosz Similar here
stackoverflow.com/users/29290/jeroen-wiert-pluimersI wasn’t there for the stats but to help out others and learn tons of things. Stopped when the testosterone people took over. I occasionally get back when there is no other way to solve a thing. Stats benefit: I see useful deleted stuff.” - [Wayback/Archive] /Fay-lee-nuh/ on Twitter: “This is what I fear too. We will invent new things, but GPT will not know the new things, since no one (human) is sharing knowledge anymore. This is not so much a GPT problem as it is a closeness problem. If all data and all prompts were open, the situation might be different.”
[Wayback/Archive] Peter Nixey on Twitter: “I’m in the top 2% of users on StackOverflow. My content there has been viewed by over 1.7M people. And it’s unlikely I’ll ever write anything there again. Which may be a much bigger problem than it seems. Because it may be the canary in the mine of our collective knowledge. A… “
Be sure to read the rest of his insightful and long tweet.
Queries
- [Wayback/Archive] “essential” “copying and pasting from chatgpt” – Google Search
- [Wayback/Archive] “essential” “copying and pasting from chatgpt” – Google Image Search
- [Wayback/Archive] Google Search by Image (based on the “copy paste from chatgpt image”)
- [Wayback/Archive] copy paste from stack overflow – Google Image Search by image (which Google automatically deducted from the “copy paste from chatgpt” book text)
- [Wayback/Archive] from:@jpluimers chatgpt – Twitter Search / Twitter
–jeroen
The Wiert Corner - irregular stream of stuff 
Leave a Reply