I'm not a regular around these parts but I thought it was worth flagging coverage of the recent "scientific research shows ChatGPT has left wing bias" story.
https://www.techspot.com/news/99838-openai-chatgpt-has-left-wing-bias-times.html
Now I don't particularly have a political dog in this fight, but I thought it was just a textbook attempt of how scientific research is poorly and uncritically covered by news sites. The fundamental problem with much media coverage for scientific research is it just parrots what was announced in the press release without taking the basic step of stopping and thinking about the multiple red flags:
And then - surprise surprise - an scientific paper with multiple red flags turns out to be complete garbage. It turns out they weren't even testing ChatGPT!
https://www.theverge.com/2023/8/18/...tgpts-liberal-bias-wasnt-even-testing-chatgpt
https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias
Anyhow, two points to make here.
That is all, Jonathan.
https://www.techspot.com/news/99838-openai-chatgpt-has-left-wing-bias-times.html
Now I don't particularly have a political dog in this fight, but I thought it was just a textbook attempt of how scientific research is poorly and uncritically covered by news sites. The fundamental problem with much media coverage for scientific research is it just parrots what was announced in the press release without taking the basic step of stopping and thinking about the multiple red flags:
- Research was trumpeted with a press release,
- Primary author is a lecturer in accounting (I mean for a paper on a pretty deep-tech subject its a bit weird),
- Author comes from second tier institution (not elitism - when assessing research quality, authors credentials matter).
- Impact Factor of journal article published in is low (1.6),
- Paper has funding from Brazilian Ministry of Education (and unsurprisingly majors on the left-wing credentials of then-opposition politician Lula).
And then - surprise surprise - an scientific paper with multiple red flags turns out to be complete garbage. It turns out they weren't even testing ChatGPT!
https://www.theverge.com/2023/8/18/...tgpts-liberal-bias-wasnt-even-testing-chatgpt
https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias
Anyhow, two points to make here.
- I understand that resources are limited and ad-funded websites are incentivised to drive engagement and clicks. But please please make the effort when covering science related stories to apply basic critical thought rather than just rewriting a press release. Generally speaking when something sounds too good to be true, it probably is. In another world this would have been a great opportunity tor Techspot to differentiate themselves from the chasing pack by asking a few simple questions before hitting publish.
- The article in question is still up on the front page unamended. At the very least you could put up an correction highlighting the fact that the original paper has material flaws, as least as originally presented. Again many of the other media outlets which reported it have failed to do this - so from a selfish point of view doing so would actually be a way to differentiate yourself from the competition!
That is all, Jonathan.
