A paper addressing gender imbalance in Wikipedia ("Gender differences in Wikipedia editing") by Judd Antin and collaborators won the "Best Short Paper" award at WikiSym. [1] This follows the awarding of "best full paper" to another study on the gender gap [2] already covered in previous editions of the research newsletter. The study by Antin and collaborators sampled 256,190 users who created a new account on the English Wikipedia between September 2010 and February 2011 and qualitatively coded their contribution by category of wiki work. The results suggest that, whereas in the lower three quartiles by activity level men and women make roughly the same contributions in each category of wiki work, in the top quartile editors behave in a significantly different way. The researchers found that among the top 25% of Wikipedians by activity level:
Another WikiSym 2011 paper by GroupLens researchers, including Summer of Research fellow Aaron Halfaker ("Don’t bite the newbies: how reverts affect the quantity and quality of Wikipedia work"), reports on the effects of reverts on the quality and quantity of Wikipedia editors, with a specific focus on newbies. [3] The study uses a number of key metrics to assess the quality of editor contributions (using reverts per revision and Persistent Word Revisions or PWR, to measure the survival across revisions of words added by an editor, other than stop-word) and changes in editor activity (using a controlled activity delta that calculates an editor's variation of activity across weeks with respect to the week preceding the revert, normalized by the editor's daily rate of activity). The results point at the same time at the important role of reverts as a learning and quality improvement process but also at their negative effects on new contributors. Below are highlights from this study:
These results are consistent with the findings by Summer of Research fellows on the effects of community interactions with new Wikipedians.
The " Wiki tools and interfaces" session at WikiSym will see the presentation of a paper titled " Autonomous link spam detection in purely collaborative environments". According to the five authors from the University of Pennsylvania, link spam is currently "an annoying, but non-pervasive issue", but could become a grave threat to Wikipedia if new spam techniques that were explored by some of them in another paper (see below) become more widespread.
Using the STiki software by one of the authors, which is already widely used as an anti-vandalism tool on the English Wikipedia, the researchers collected mainspace edits adding external links and extracted a corpus of 5,962 link additions classified as either ham or spam, using criteria such as whether the edit had been rolled back (to determine spam), or whether it had been added by a user with rollback rights (to determine ham). From this, the researchers derived numerous features that indicate link spamming behavior, in three areas: On-wiki evidence (including very simple metrics such as the URL's length – spam links tend to be shorter – or that older and more popular articles are more likely to be targeted), properties of the landing page that the link points to (these were found to be less useful), and classification from third-party sites, including Alexa and Google Safe Browsing. The backlinks data provided by Alexa proved to be most useful for the classifier that the authors went on to construct, and tested in a live implementation in the STiki tool. They conclude that "it is clear this work will benefit the Wikipedia community".
In another paper, presented earlier this month at CEAS ‘11, five authors from the same university including two of the same researchers examine the possibility of " Link spamming Wikipedia for profit". They picture spam detection on Wikipedia as a pipelined process, with the MediaWiki spam blacklist as the first stage (currently containing around 17000 regular expressions), recent changes patrollers (often aided by software tools) as the next – often reacting within seconds after an edit, watchlisters as the third (within minutes to days), and finally review by normal readers as the last stage. Based on a spam/ham corpus constructed as in the other paper, this paper contains some further analysis of the characteristics of link spam destinations and spamming accounts, and of the exposure spammed links receive before they are removed (determined by both the link's lifespan and the popularity of the spammed page). The most sensitive part of the paper then leverages these results to "describe a novel and efficient spam model we estimate can significantly outperform status quo techniques", e.g. by rapidly adding links to exploit the time lag of Wikipedia's spam removal process, or targetting popular pages. In a nod to WP:BEANS, the researchers admit that "there is the possibility that we have introduced previously unknown vectors", but the "Ethical Considerations" section emphasizes that:
The authors also point to the implementation of the spam mitigation tool described in the WikiSym article.
However, the paper fails to mention that last year, one of its authors conducted actual, extensive tests of spamming techniques on the English Wikipedia that are very similar to those outlined in the paper. The spam attacks gained the attention of several IT security news websites, and even involved setting up a fake webshop to measure how many Wikipedia readers would have carried out an actual purchase of the penis enlargement pills advertised in the links. The case led to the researcher's temporary ban as a Wikipedia user, later lifted by the arbitration committee, and informed the research guidelines drafted later that year by the Wikimedia Foundation's Research Committee. See Signpost coverage: " Large scale vandalism revealed to be 'study' by university researcher" (includes a background interview with the researcher).
A paper by three researchers from the University of the Philippines Diliman [6], presented at the International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2011) two months ago, examined statistical relations between the voting behavior in requests for adminships (RfAs) and the on-wiki social contacts of participants. The paper includes a brief review of existing literature (in particular two papers which already studied the relation with existing social networks [7] [8]). Drawing from a January 2008 dump of the English Wikipedia, they analyzed 2,587 elections conducted between 2004 and 2008 (48% of them successful, with 7,231 users voting or running in at least one RfA, and 80% of the final non-neutral votes being supportive), and "1,097,223 instances of communication between 265,155 distinct pairs of users" who had run or voted in an RfA – from user talk page messages, an undirected social graph was generated. Their results concern three areas:
A paper titled "Temporal characterization of the requests to Wikipedia" examined how search requests, read accesses and edits on Wikipedia change over time, and relate to those at the entirety of Wikimedia sites (based on squid logs for the whole year of 2009, provided by the Wikimedia Foundation). Among findings are differences between language versions of Wikipedia, such as that the "the number of edits tends to raise in weekends" for the French, Japanese, Dutch and Polish Wikipedia, but not for other languages. Another paper, titled "Circadian patterns of Wikipedia editorial activity: A demographic analysis" [9], similarly analyzed "34 Wikipedias in different languages [trying] to characterize and find the universalities and differences in temporal activity patterns of editors", with the underlying data provided by the German Wikimedia chapter from the toolserver. They found that "in contrast to diurnal [daily] pattern, which is universal to a great extent, weekly activity patterns of WPs show remarkable differences. We could, however, identify two main categories, namely 'weekends' and 'working days' active WPs." [10]
A paper addressing gender imbalance in Wikipedia ("Gender differences in Wikipedia editing") by Judd Antin and collaborators won the "Best Short Paper" award at WikiSym. [1] This follows the awarding of "best full paper" to another study on the gender gap [2] already covered in previous editions of the research newsletter. The study by Antin and collaborators sampled 256,190 users who created a new account on the English Wikipedia between September 2010 and February 2011 and qualitatively coded their contribution by category of wiki work. The results suggest that, whereas in the lower three quartiles by activity level men and women make roughly the same contributions in each category of wiki work, in the top quartile editors behave in a significantly different way. The researchers found that among the top 25% of Wikipedians by activity level:
Another WikiSym 2011 paper by GroupLens researchers, including Summer of Research fellow Aaron Halfaker ("Don’t bite the newbies: how reverts affect the quantity and quality of Wikipedia work"), reports on the effects of reverts on the quality and quantity of Wikipedia editors, with a specific focus on newbies. [3] The study uses a number of key metrics to assess the quality of editor contributions (using reverts per revision and Persistent Word Revisions or PWR, to measure the survival across revisions of words added by an editor, other than stop-word) and changes in editor activity (using a controlled activity delta that calculates an editor's variation of activity across weeks with respect to the week preceding the revert, normalized by the editor's daily rate of activity). The results point at the same time at the important role of reverts as a learning and quality improvement process but also at their negative effects on new contributors. Below are highlights from this study:
These results are consistent with the findings by Summer of Research fellows on the effects of community interactions with new Wikipedians.
The " Wiki tools and interfaces" session at WikiSym will see the presentation of a paper titled " Autonomous link spam detection in purely collaborative environments". According to the five authors from the University of Pennsylvania, link spam is currently "an annoying, but non-pervasive issue", but could become a grave threat to Wikipedia if new spam techniques that were explored by some of them in another paper (see below) become more widespread.
Using the STiki software by one of the authors, which is already widely used as an anti-vandalism tool on the English Wikipedia, the researchers collected mainspace edits adding external links and extracted a corpus of 5,962 link additions classified as either ham or spam, using criteria such as whether the edit had been rolled back (to determine spam), or whether it had been added by a user with rollback rights (to determine ham). From this, the researchers derived numerous features that indicate link spamming behavior, in three areas: On-wiki evidence (including very simple metrics such as the URL's length – spam links tend to be shorter – or that older and more popular articles are more likely to be targeted), properties of the landing page that the link points to (these were found to be less useful), and classification from third-party sites, including Alexa and Google Safe Browsing. The backlinks data provided by Alexa proved to be most useful for the classifier that the authors went on to construct, and tested in a live implementation in the STiki tool. They conclude that "it is clear this work will benefit the Wikipedia community".
In another paper, presented earlier this month at CEAS ‘11, five authors from the same university including two of the same researchers examine the possibility of " Link spamming Wikipedia for profit". They picture spam detection on Wikipedia as a pipelined process, with the MediaWiki spam blacklist as the first stage (currently containing around 17000 regular expressions), recent changes patrollers (often aided by software tools) as the next – often reacting within seconds after an edit, watchlisters as the third (within minutes to days), and finally review by normal readers as the last stage. Based on a spam/ham corpus constructed as in the other paper, this paper contains some further analysis of the characteristics of link spam destinations and spamming accounts, and of the exposure spammed links receive before they are removed (determined by both the link's lifespan and the popularity of the spammed page). The most sensitive part of the paper then leverages these results to "describe a novel and efficient spam model we estimate can significantly outperform status quo techniques", e.g. by rapidly adding links to exploit the time lag of Wikipedia's spam removal process, or targetting popular pages. In a nod to WP:BEANS, the researchers admit that "there is the possibility that we have introduced previously unknown vectors", but the "Ethical Considerations" section emphasizes that:
The authors also point to the implementation of the spam mitigation tool described in the WikiSym article.
However, the paper fails to mention that last year, one of its authors conducted actual, extensive tests of spamming techniques on the English Wikipedia that are very similar to those outlined in the paper. The spam attacks gained the attention of several IT security news websites, and even involved setting up a fake webshop to measure how many Wikipedia readers would have carried out an actual purchase of the penis enlargement pills advertised in the links. The case led to the researcher's temporary ban as a Wikipedia user, later lifted by the arbitration committee, and informed the research guidelines drafted later that year by the Wikimedia Foundation's Research Committee. See Signpost coverage: " Large scale vandalism revealed to be 'study' by university researcher" (includes a background interview with the researcher).
A paper by three researchers from the University of the Philippines Diliman [6], presented at the International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2011) two months ago, examined statistical relations between the voting behavior in requests for adminships (RfAs) and the on-wiki social contacts of participants. The paper includes a brief review of existing literature (in particular two papers which already studied the relation with existing social networks [7] [8]). Drawing from a January 2008 dump of the English Wikipedia, they analyzed 2,587 elections conducted between 2004 and 2008 (48% of them successful, with 7,231 users voting or running in at least one RfA, and 80% of the final non-neutral votes being supportive), and "1,097,223 instances of communication between 265,155 distinct pairs of users" who had run or voted in an RfA – from user talk page messages, an undirected social graph was generated. Their results concern three areas:
A paper titled "Temporal characterization of the requests to Wikipedia" examined how search requests, read accesses and edits on Wikipedia change over time, and relate to those at the entirety of Wikimedia sites (based on squid logs for the whole year of 2009, provided by the Wikimedia Foundation). Among findings are differences between language versions of Wikipedia, such as that the "the number of edits tends to raise in weekends" for the French, Japanese, Dutch and Polish Wikipedia, but not for other languages. Another paper, titled "Circadian patterns of Wikipedia editorial activity: A demographic analysis" [9], similarly analyzed "34 Wikipedias in different languages [trying] to characterize and find the universalities and differences in temporal activity patterns of editors", with the underlying data provided by the German Wikimedia chapter from the toolserver. They found that "in contrast to diurnal [daily] pattern, which is universal to a great extent, weekly activity patterns of WPs show remarkable differences. We could, however, identify two main categories, namely 'weekends' and 'working days' active WPs." [10]
Discuss this story
Less technical.
Could the standard deviation results be expressed in less technical terms so that 95% of readers know what it means? One standard deviation, BTW, is a range of about 34% on both sides of the mean, or average. Tony (talk) 03:11, 27 September 2011 (UTC) reply
Antarctica.
While Antarctica vs. country may make for a nice sound bite, it's very much apples and oranges. Whereas Antarctica's population may be just a handful, its scientific significance and the variety of wildlife are well, continent-sized. Jztinfinity ( talk) 04:54, 27 September 2011 (UTC) reply
Reverting and quality
this seems like a dubious analysis: "reverted editors are less likely to be reverted in the future (particularly in the week after the revert), whereas the probability of being reverted in the control group keeps growing every week." the second part can be rephrased as "the longer you edit the more probable you will get reverted". Insofar as EVERYONE gets reverted eventually, it doesn't mean much to me. Circéus ( talk) 19:46, 27 September 2011 (UTC) reply