From Wikipedia, the free encyclopedia

Simply bad

Bendegúz Ács asked a little while ago for more information about the sentence that "A source can also simply be bad, where biases in criteria make it less than ideal." Here's a short explanation, in case anyone's interested:

We have criteria that help editors identify sources that are generally better (e.g., peer-reviewed sources, secondary sources, recent sources). However, no single criterion definitely makes a source be good ( WP:NOTGOODSOURCE). In addition to all the criteria, you have to use common sense and have a good understanding of the subject. Sometimes a source seems superficially good, but when you read it, you discover that it's just bad. In such cases, you should use your WP:Editorial discretion to avoid relying on a bad source.

Here are some simple examples of how real-world bias could lead an editor to decide that a source is just bad:

  • The subject is a disease that's very common in poor countries, but your source tells you about the disease in a wealthy country.
  • You need to write something about pregnancy, and the clinical trial involves only males. (That's how the world ended up with the Thalidomide scandal, by the way.)
  • The subject is pediatrics, and the data is all from elderly people.

The main reason that we don't explain is because the potential sources of bias are enormous. Editors who are uncertain about whether their source can support the claims they are making should be discussing the specifics on the article's talk page or at Wikipedia talk:WikiProject Medicine. WhatamIdoing ( talk) 02:41, 23 February 2024 (UTC) reply

Thank you for the clarification and the examples, they definitely make this particular issue much easier for me to understand. However, I would still like to suggest some improvements to the sentence. I think here there are two separate potential issues that may make a source bad, and both are worth mentioning separately.
One is what your examples are about, and I would call this something along the lines of "subjects/subgroup analysis with low relevancy" or "irrelevant/overly restrictive subjects/subgroup analysis", since the main issue I can identify in your examples is that the source fails to take into account most of the people affected by the topic. I think this particular issue would be worth mentioning in a separate sentence because it does seem important.
The other one is about problems that are so particular about a study that it is not possible to easily fit into any well-defined criterion. I would not limit these problems to "biases", however, since not every problem possible to make a study bad is considered a bias. So maybe mentioning it in this way would be an improvement: "A source can also simply be bad, due to a bias, or other problem, too particular to be covered by a common editorial criterion. In cases like this, WP:Editorial discretion may be used."
What do you think? Bendegúz Ács ( talk) 23:30, 23 February 2024 (UTC) reply
It might be better to re-write the whole sub-section from scratch. I think it is meant to cover things like pesticide manufacturers claiming that their products are utterly harmless. WhatamIdoing ( talk) 22:55, 24 February 2024 (UTC) reply
I would support a rewrite, but I am not competent enough to come up with a suggestion for a complete new text for the sub-section. I would be happy to review it, though. Bendegúz Ács ( talk) 15:01, 25 February 2024 (UTC) reply
I've considered just blanking it. Do we truly need WP:MEDBIAS, or is it WP:CREEPY and redundant with other pages? It's only been linked in discussions about a handful of articles.
@ Bon courage, what do you think? Could we live without it? WhatamIdoing ( talk) 07:11, 7 March 2024 (UTC) reply
Hah! Have never paid any attention to this. It arrived [1] in 2016. WP:DONTSHOUTBIAS eh!? Bon courage ( talk) 07:26, 7 March 2024 (UTC) reply
Oh, that would have been around the time that we tried (and failed) to put a medicine-specific version of WP:DUE in this guideline. WhatamIdoing ( talk) 22:58, 7 March 2024 (UTC) reply
I don't see anything there that needs to be (medicine) specific and the "Personal conflicts of interest" isn't about "reliable sources" either. I'd vote for blanking both. The concerns there about biased sources or biased editors belong in general guidelines. -- Colin° Talk 08:55, 7 March 2024 (UTC) reply
Since nobody's thought this would be useful during the last ~two months, I've removed it. Removal doesn't mean that it's bad advice. It just seems unnecessary in this particular guideline, and redundant with others (e.g., the Wikipedia:Neutral point of view policy). WhatamIdoing ( talk) 03:41, 6 June 2024 (UTC) reply

MEDRS-related discussion at WP:RSN

There is a new discussion at RSN that MEDRS-interested editors may want to get involved in: Wikipedia:Reliable_sources/Noticeboard#MEDRS_sources_on_curcumin_supplementation Schazjmd  (talk) 23:53, 25 February 2024 (UTC) reply

Meta-analyses need careful scrutiny, peer-reviewed or not.

This

is a cautionary tale. 𝕁𝕄𝔽 ( talk) 19:15, 28 April 2024 (UTC) reply

Thanks, that's interesting.
I have been wondering recently why peer review doesn't involve putting each article through an evidence-based checklist (e.g., the Newcastle–Ottawa scale for non-randomized studies; the most relevant checklist for the subject matter could be agreed upon by the reviewers and the editor) and the results published alongside the article. It might make it easier to discover junk science before publication. WhatamIdoing ( talk) 21:47, 28 April 2024 (UTC) reply

Thalassophobia

I have an impression that the huge section Diagnosis and Spmothoms is written as an advertisement by snake oil peddlers, such as CBT desensitization, etc. One can find plenty of such phobia-curing webpages in the internets. In its entirety it can be cut and pasted into each and every phobia article only replacing the name of phobia. I suspect this is actually done by various headshrinkers. Can someone write a common page, like, Symptoms and treatment of specific phobias, with proper MEDRS oversight, and then simply refer to it everywhere 99.73.36.110 ( talk) 23:39, 7 June 2024 (UTC) reply

I suspect that such a page would look a lot like Thalassophobia#Treatment. It already looks similar to Specific phobia#Treatment. WhatamIdoing ( talk) 00:41, 8 June 2024 (UTC) reply

Reconsidering a blanket-ban of primary sources

The express reason given is essentially a re-hash of the "reproducibility crisis," but I have to say, should we even be trusting these "fact-checking"-like organizations for whom we essentially defer the power of keeping out bad knowledge?

Did you know, for example, the American Psychological Association, known for such works as the DSM-V and numerous textbooks likely to pass as verified work through here without a second thought, is actually a trade organization with the express intent of lobbying on behalf of practicing psychologists, in exchange extracting registration dues, and licensing fees?

Or that the American Pain Society, this time a non-profit society that nominally advocated on behalf of patients by publishing clinical practice guidelines, actually instead acted on behalf of pharmaceutical companies to propagate a treatment mandate to prescribe more products, in essence acting as a marketing channel where physicians were not primed to regard it critically?

I think it might be beneficial to re-investigate the ultimate authority of the organizations/publication guidelines we choose to let pass uncritically with more ability afforded to well-informed individuals to make their case that enough primary research exists to effectively include a consensus on matters of pathology and lines of inquiry related to potential treatment paradigms. We do ourselves a disservice when we shut out promising potential, and similarly to our readers too when academic literature acts more like a thicket than it does pathway. Additivefreesb ( talk) 17:52, 12 July 2024 (UTC) reply

Which blanket-ban on primary sources are you referring to? Firefangledfeathers ( talk / contribs) 17:56, 12 July 2024 (UTC) reply
Wikipedia:Identifying reliable sources (medicine)#Avoid primary sources isn't a blanket ban. Wikipedia:Identifying reliable sources (medicine)#Avoid over-emphasizing single studies, particularly in vitro or animal studies is just common sense.
We have had multiple problems with primary sources, including:
  • editors cherry picking the one source that says something completely different from all the others (e.g., cigarettes don't cause lung cancer)
  • editors using obviously bad primary sources (e.g., the patent claiming that colloidal silver cures HIV, but it didn't test whether people had HIV in the first place)
  • editors believing the media hype (can result in bad content [because newspapers don't always get technical details right] and unbalanced articles [because it's all about what's in today's news, and next week it'll be some other vegetable that everyone's supposed to eat to prevent cancer])
  • authors spamming their own publications into as many articles as possible (this happens much less often with review articles)
plus, of course, all the problems with the reproducibility crisis and the general difficulty of figuring out which primary source to 'believe in', if the data is conflicting.
Primary sources are more likely to be tolerated in veterinary content or for very rare conditions. WhatamIdoing ( talk) 18:38, 12 July 2024 (UTC) reply
From Wikipedia, the free encyclopedia

Simply bad

Bendegúz Ács asked a little while ago for more information about the sentence that "A source can also simply be bad, where biases in criteria make it less than ideal." Here's a short explanation, in case anyone's interested:

We have criteria that help editors identify sources that are generally better (e.g., peer-reviewed sources, secondary sources, recent sources). However, no single criterion definitely makes a source be good ( WP:NOTGOODSOURCE). In addition to all the criteria, you have to use common sense and have a good understanding of the subject. Sometimes a source seems superficially good, but when you read it, you discover that it's just bad. In such cases, you should use your WP:Editorial discretion to avoid relying on a bad source.

Here are some simple examples of how real-world bias could lead an editor to decide that a source is just bad:

  • The subject is a disease that's very common in poor countries, but your source tells you about the disease in a wealthy country.
  • You need to write something about pregnancy, and the clinical trial involves only males. (That's how the world ended up with the Thalidomide scandal, by the way.)
  • The subject is pediatrics, and the data is all from elderly people.

The main reason that we don't explain is because the potential sources of bias are enormous. Editors who are uncertain about whether their source can support the claims they are making should be discussing the specifics on the article's talk page or at Wikipedia talk:WikiProject Medicine. WhatamIdoing ( talk) 02:41, 23 February 2024 (UTC) reply

Thank you for the clarification and the examples, they definitely make this particular issue much easier for me to understand. However, I would still like to suggest some improvements to the sentence. I think here there are two separate potential issues that may make a source bad, and both are worth mentioning separately.
One is what your examples are about, and I would call this something along the lines of "subjects/subgroup analysis with low relevancy" or "irrelevant/overly restrictive subjects/subgroup analysis", since the main issue I can identify in your examples is that the source fails to take into account most of the people affected by the topic. I think this particular issue would be worth mentioning in a separate sentence because it does seem important.
The other one is about problems that are so particular about a study that it is not possible to easily fit into any well-defined criterion. I would not limit these problems to "biases", however, since not every problem possible to make a study bad is considered a bias. So maybe mentioning it in this way would be an improvement: "A source can also simply be bad, due to a bias, or other problem, too particular to be covered by a common editorial criterion. In cases like this, WP:Editorial discretion may be used."
What do you think? Bendegúz Ács ( talk) 23:30, 23 February 2024 (UTC) reply
It might be better to re-write the whole sub-section from scratch. I think it is meant to cover things like pesticide manufacturers claiming that their products are utterly harmless. WhatamIdoing ( talk) 22:55, 24 February 2024 (UTC) reply
I would support a rewrite, but I am not competent enough to come up with a suggestion for a complete new text for the sub-section. I would be happy to review it, though. Bendegúz Ács ( talk) 15:01, 25 February 2024 (UTC) reply
I've considered just blanking it. Do we truly need WP:MEDBIAS, or is it WP:CREEPY and redundant with other pages? It's only been linked in discussions about a handful of articles.
@ Bon courage, what do you think? Could we live without it? WhatamIdoing ( talk) 07:11, 7 March 2024 (UTC) reply
Hah! Have never paid any attention to this. It arrived [1] in 2016. WP:DONTSHOUTBIAS eh!? Bon courage ( talk) 07:26, 7 March 2024 (UTC) reply
Oh, that would have been around the time that we tried (and failed) to put a medicine-specific version of WP:DUE in this guideline. WhatamIdoing ( talk) 22:58, 7 March 2024 (UTC) reply
I don't see anything there that needs to be (medicine) specific and the "Personal conflicts of interest" isn't about "reliable sources" either. I'd vote for blanking both. The concerns there about biased sources or biased editors belong in general guidelines. -- Colin° Talk 08:55, 7 March 2024 (UTC) reply
Since nobody's thought this would be useful during the last ~two months, I've removed it. Removal doesn't mean that it's bad advice. It just seems unnecessary in this particular guideline, and redundant with others (e.g., the Wikipedia:Neutral point of view policy). WhatamIdoing ( talk) 03:41, 6 June 2024 (UTC) reply

MEDRS-related discussion at WP:RSN

There is a new discussion at RSN that MEDRS-interested editors may want to get involved in: Wikipedia:Reliable_sources/Noticeboard#MEDRS_sources_on_curcumin_supplementation Schazjmd  (talk) 23:53, 25 February 2024 (UTC) reply

Meta-analyses need careful scrutiny, peer-reviewed or not.

This

is a cautionary tale. 𝕁𝕄𝔽 ( talk) 19:15, 28 April 2024 (UTC) reply

Thanks, that's interesting.
I have been wondering recently why peer review doesn't involve putting each article through an evidence-based checklist (e.g., the Newcastle–Ottawa scale for non-randomized studies; the most relevant checklist for the subject matter could be agreed upon by the reviewers and the editor) and the results published alongside the article. It might make it easier to discover junk science before publication. WhatamIdoing ( talk) 21:47, 28 April 2024 (UTC) reply

Thalassophobia

I have an impression that the huge section Diagnosis and Spmothoms is written as an advertisement by snake oil peddlers, such as CBT desensitization, etc. One can find plenty of such phobia-curing webpages in the internets. In its entirety it can be cut and pasted into each and every phobia article only replacing the name of phobia. I suspect this is actually done by various headshrinkers. Can someone write a common page, like, Symptoms and treatment of specific phobias, with proper MEDRS oversight, and then simply refer to it everywhere 99.73.36.110 ( talk) 23:39, 7 June 2024 (UTC) reply

I suspect that such a page would look a lot like Thalassophobia#Treatment. It already looks similar to Specific phobia#Treatment. WhatamIdoing ( talk) 00:41, 8 June 2024 (UTC) reply

Reconsidering a blanket-ban of primary sources

The express reason given is essentially a re-hash of the "reproducibility crisis," but I have to say, should we even be trusting these "fact-checking"-like organizations for whom we essentially defer the power of keeping out bad knowledge?

Did you know, for example, the American Psychological Association, known for such works as the DSM-V and numerous textbooks likely to pass as verified work through here without a second thought, is actually a trade organization with the express intent of lobbying on behalf of practicing psychologists, in exchange extracting registration dues, and licensing fees?

Or that the American Pain Society, this time a non-profit society that nominally advocated on behalf of patients by publishing clinical practice guidelines, actually instead acted on behalf of pharmaceutical companies to propagate a treatment mandate to prescribe more products, in essence acting as a marketing channel where physicians were not primed to regard it critically?

I think it might be beneficial to re-investigate the ultimate authority of the organizations/publication guidelines we choose to let pass uncritically with more ability afforded to well-informed individuals to make their case that enough primary research exists to effectively include a consensus on matters of pathology and lines of inquiry related to potential treatment paradigms. We do ourselves a disservice when we shut out promising potential, and similarly to our readers too when academic literature acts more like a thicket than it does pathway. Additivefreesb ( talk) 17:52, 12 July 2024 (UTC) reply

Which blanket-ban on primary sources are you referring to? Firefangledfeathers ( talk / contribs) 17:56, 12 July 2024 (UTC) reply
Wikipedia:Identifying reliable sources (medicine)#Avoid primary sources isn't a blanket ban. Wikipedia:Identifying reliable sources (medicine)#Avoid over-emphasizing single studies, particularly in vitro or animal studies is just common sense.
We have had multiple problems with primary sources, including:
  • editors cherry picking the one source that says something completely different from all the others (e.g., cigarettes don't cause lung cancer)
  • editors using obviously bad primary sources (e.g., the patent claiming that colloidal silver cures HIV, but it didn't test whether people had HIV in the first place)
  • editors believing the media hype (can result in bad content [because newspapers don't always get technical details right] and unbalanced articles [because it's all about what's in today's news, and next week it'll be some other vegetable that everyone's supposed to eat to prevent cancer])
  • authors spamming their own publications into as many articles as possible (this happens much less often with review articles)
plus, of course, all the problems with the reproducibility crisis and the general difficulty of figuring out which primary source to 'believe in', if the data is conflicting.
Primary sources are more likely to be tolerated in veterinary content or for very rare conditions. WhatamIdoing ( talk) 18:38, 12 July 2024 (UTC) reply

Videos

Youtube | Vimeo | Bing

Websites

Google | Yahoo | Bing

Encyclopedia

Google | Yahoo | Bing

Facebook