Discuss this story
- I'm very dubious of the attempt to quantity someone's "intelligence". First we need to know what kind of intelligence and knowledge is important to contributing to Wikipedia. Well, I'd say the ideal first step is extremely specialised intelligence in a particular area, whether high culture or low culture—this makes it easiest to find a topic where you can substantially contribute. Note that a lot of people with specialised intelligence have worse than average general forms of intelligence. Next you need technical knowledge, otherwise learning our syntax, navigating the site and quickly writing references will be difficult. Third you need emotional intelligence, to respond well to people (sometimes lacking emotional intelligence themselves) sharply criticising your good-faith mistakes or to spring back after an article you wrote is deleted at AfD or the peanut gallery throw stones at you, or to collaborate with other editors. And fourth, probably some traits from the umbrella of critical thinking are useful for analysing sources, reviewing how content fits with our policies/guidelines/practices and learning from other editors. But a bunch of visual puzzle games in
Raven's Progressive Matrices are supposed to be a sufficient stand-in for all of this? Why not just measure people's
skull shape? Additionally, the researchers' question "[Who, when and why do] people share knowledge online, often without tangible compensation?" has a loaded premise. Do people choose hobbies because of "tangible compensation"? If you approach something from the angle of "finding the secret compensation that this person must have as a motivation" then you will of course conclude that, aha, it's because of the prestige your username gets you. I'm no scientist but I suggest that a more useful study would be to simply ask editors what their motivation is and then see if that accords with psychology or aligns with other collaborative forms of knowledge sharing. I'll give you my motivation for free: contributing to Wikipedia makes me feel like I am creating a resource which is useful to other people. (And hopefully I am.) —
Bilorv (
talk) 11:35, 1 September 2020 (UTC)
- In addition to the well-known problems with inferring anything about humanity at large based on samples of college students, I'd guess that the motivations for undergraduates to sign up for a 30-minute research study could be completely different from the motivations for picking up a hobby. I am a scientist, and I decided to try out editing Wikipedia because I wanted a downtime activity where my bookish inclinations might, in some small way, help people. XOR'easter ( talk) 19:00, 1 September 2020 (UTC)
- There exists an entire scholarly industry - going back more than a century - debating the concept of general intelligence, how or whether it can be quantified, what the biases of such tests might be etc. In the review, I linked to the articles about the journal and about g, as pathways for readers to learn more about the field's background.
- That said, the experiment found actual evidence for a relation between intelligence as measured by these tests, and article quality as rated by readers unaware of the editors' scores. In other words, the researchers' result contradicts your conclusion that these scores are as unrelated to the article-writing task as "skull shapes". (And I guess they would not dispute that there could be other relevant dimensions, such as emotional intelligence - a concept and measure that of course has attracted its own share of criticism and validity concerns, or that their experiment setup did not simulate every possible aspect of editing Wikipedia, like AfD discussions.)
- "I suggest that a more useful study would be to simply ask editors what their motivation is" - several such studies have been done, see the link in the review.
- Regards,
HaeB (
talk) 17:10, 4 September 2020 (UTC)
- I wasn't meaning to criticise your summary of the study, per se. I hope my comment is clear that I am no professional in this area but as a layperson with an interest, I am aware of the g factor and some of the other topics you mention. I think you have focused too specifically on the exact wording I used (including the mention of phrenology, which was hyperbole for comedic effect rather than a literal "conclusion") and less broadly on the idea that researchers' implicit preconceptions regarding intelligence and people acting out of self-interest influence results that are published (given such things as false positives and publication bias). I am aware of studies similar to that which I suggested but I believe, given the current replication crisis, another study in the same direction would be more useful than one that is quite far removed from the way Wikipedia works in practice. — Bilorv ( talk) 19:13, 4 September 2020 (UTC)
Discuss this story
- I'm very dubious of the attempt to quantity someone's "intelligence". First we need to know what kind of intelligence and knowledge is important to contributing to Wikipedia. Well, I'd say the ideal first step is extremely specialised intelligence in a particular area, whether high culture or low culture—this makes it easiest to find a topic where you can substantially contribute. Note that a lot of people with specialised intelligence have worse than average general forms of intelligence. Next you need technical knowledge, otherwise learning our syntax, navigating the site and quickly writing references will be difficult. Third you need emotional intelligence, to respond well to people (sometimes lacking emotional intelligence themselves) sharply criticising your good-faith mistakes or to spring back after an article you wrote is deleted at AfD or the peanut gallery throw stones at you, or to collaborate with other editors. And fourth, probably some traits from the umbrella of critical thinking are useful for analysing sources, reviewing how content fits with our policies/guidelines/practices and learning from other editors. But a bunch of visual puzzle games in
Raven's Progressive Matrices are supposed to be a sufficient stand-in for all of this? Why not just measure people's
skull shape? Additionally, the researchers' question "[Who, when and why do] people share knowledge online, often without tangible compensation?" has a loaded premise. Do people choose hobbies because of "tangible compensation"? If you approach something from the angle of "finding the secret compensation that this person must have as a motivation" then you will of course conclude that, aha, it's because of the prestige your username gets you. I'm no scientist but I suggest that a more useful study would be to simply ask editors what their motivation is and then see if that accords with psychology or aligns with other collaborative forms of knowledge sharing. I'll give you my motivation for free: contributing to Wikipedia makes me feel like I am creating a resource which is useful to other people. (And hopefully I am.) —
Bilorv (
talk) 11:35, 1 September 2020 (UTC)
- In addition to the well-known problems with inferring anything about humanity at large based on samples of college students, I'd guess that the motivations for undergraduates to sign up for a 30-minute research study could be completely different from the motivations for picking up a hobby. I am a scientist, and I decided to try out editing Wikipedia because I wanted a downtime activity where my bookish inclinations might, in some small way, help people. XOR'easter ( talk) 19:00, 1 September 2020 (UTC)
- There exists an entire scholarly industry - going back more than a century - debating the concept of general intelligence, how or whether it can be quantified, what the biases of such tests might be etc. In the review, I linked to the articles about the journal and about g, as pathways for readers to learn more about the field's background.
- That said, the experiment found actual evidence for a relation between intelligence as measured by these tests, and article quality as rated by readers unaware of the editors' scores. In other words, the researchers' result contradicts your conclusion that these scores are as unrelated to the article-writing task as "skull shapes". (And I guess they would not dispute that there could be other relevant dimensions, such as emotional intelligence - a concept and measure that of course has attracted its own share of criticism and validity concerns, or that their experiment setup did not simulate every possible aspect of editing Wikipedia, like AfD discussions.)
- "I suggest that a more useful study would be to simply ask editors what their motivation is" - several such studies have been done, see the link in the review.
- Regards,
HaeB (
talk) 17:10, 4 September 2020 (UTC)
- I wasn't meaning to criticise your summary of the study, per se. I hope my comment is clear that I am no professional in this area but as a layperson with an interest, I am aware of the g factor and some of the other topics you mention. I think you have focused too specifically on the exact wording I used (including the mention of phrenology, which was hyperbole for comedic effect rather than a literal "conclusion") and less broadly on the idea that researchers' implicit preconceptions regarding intelligence and people acting out of self-interest influence results that are published (given such things as false positives and publication bias). I am aware of studies similar to that which I suggested but I believe, given the current replication crisis, another study in the same direction would be more useful than one that is quite far removed from the way Wikipedia works in practice. — Bilorv ( talk) 19:13, 4 September 2020 (UTC)