|
I appreciate WMF taking a stance. Has the censored version of the logo been uploaded to Commons? I think it would be a great image to use in some of our articles on related issues. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 23:15, 16 November 2011 (UTC)
That's awkward, I think, even if the authors did use it. I'm trying to think of something idiomatic. "outwardly focused" isn't quite it. I can't find an antonym for "self". Tony (talk) 14:48, 27 December 2011 (UTC)
Hi, does ElKevbo have a point? I see that the pdf file says, inter alia, "Under review—do not cite." and "Submitted for confidential review. November 2011". Apart from other concerns, this coverage might be seen to put pressure on the peer reviewers. Tony (talk) 12:34, 28 December 2011 (UTC)
I wish I had more time right now, I am somewhat busy today. But here are short summaries of:
![]() |
The Signpost Barnstar | |
For your thoroughness in the January 30, 2012 Signpost Recent Research report. Pine talk 08:21, 31 January 2012 (UTC) |
I'll try to help and review social science papers again! -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 19:31, 22 February 2012 (UTC)
As previously, I am posting my entries here. Let me know if you'd like me to add them directly to Signpost instead in the future. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 21:26, 22 February 2012 (UTC)
Thanks for the ping.
-- Piotr Konieczny aka Prokonsul Piotrus| talk to me 17:56, 27 February 2012 (UTC)
Also, on the subject of teaching with Wikipedia, this seems relevant, so I suggest we add it:
-- Piotr Konieczny aka Prokonsul Piotrus| talk to me 19:11, 27 February 2012 (UTC)
Ok, I am done with my newest Wikipedia's article, so here are some reviews of works that I found interesting.
Moin Tbayer please take a look at this edit. Cheers Sargoth ( talk) 22:44, 29 February 2012 (UTC)
![]() |
The Teamwork Barnstar |
to all of the contributors to the April 30, 2012 Recent Research report in the Signpost for the good work there! Pine (talk) 07:51, 2 May 2012 (UTC) |
[18]: This article looks at interactions between Wikipedia editors, and the project's governance, visible in the articles on stem cells and transhumanism, and in the analysis of Wikipedia's discussion of userboxes through the prism of Jürgen Habermas universal pragmatics and Mikhail Bakhtin dialogism theories. The authors use those theories, focusing on the qualitative analysis of language used by editors, to argue that Wikipedia has elements of a democracy, and is an example of a Web 2.0 empowering (emancipating) discourse tool. They authors stress that some forms of discourse found online (and on Wikipedia) may be highly irrational, something that some previous arguments for Web 2.0 being a democratic space have often ignored, but they argue that this is in fact not as much of a hindrance as previously expected. The authors remark that on Wikipedians, discourse can develop between editors of widely differing points of view, and that some Wikipedians will engage in "repeated, strategic, and often highly manipulative attempts" to assert personal authority. Such discussions may be very lively, with "that personal, emotional, or humour-based arguments", yet the authors argue that such comments may not be a hindrance; instead, "on many occasions, there is thus a clearer exposition of views that is achieved, in spite of, or perhaps because of, these personal/sometimes vulgar methods of argumentation." In the end, the authors positively comment of the success of Wikipedia's deliberation in reaching consensus, albeit they remark that it can be "fleeting and transitory" on occasion. Unfortunately, the paper does not touch upon the existence of Wikipedia policies such as Wikipedia:Civility and Wikipedia:No personal attacks, which would certainly add to the analysis presented.
On a side note, despite the paper's claim to have received an approval for research through a "University Research Ethics Committee", the fact that the paper discusses, in occasionally critical fashion (example: "[Editor A] claim to authority and ad hominem attacks were met with derision by [Editor B]" (editor names have been replaced by anonymous pseudonyms by me), the editing of specific editors, may raise some eyebrows. As we all know, not all editors are 100% anonymous, and even those who who are have vested reputation in their identities. This raises a question if this paper has done enough to protect the identity and reputation of the editors it cites; at the very least, why weren't the editors usernames changed in the quotes? Their direct identification adds nothing to the article (what is important for the author's argument is the quote itself, not who said it), but makes it easier for others to use the paper in attacking them back.
Regarding WMC, perhaps we should link meta:Notes_on_good_practices_on_Wikipedia_research#Anonymised_re_pseudonymised. Also, I found time to review [19]. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 16:26, 29 May 2012 (UTC)
The authors collected data from Wikipedia:Request for adminship pages, and tte ties were measured through user talk interaction; core members/leaders were defined as administrators, and periphery editors as non-administrators (this operationalization may raise some doubts about the validity, as there are some very active and prominent members of the community who are not administrators, something that the authors do not address). The authors find that important ties are the early ties to the periphery, and later, ties to the leaders. They also find that overall strong ties are not as important as weak ties, althouth Simmelian ties (two leader groups) are among the most important. The authors conclude that leaders in projects like Wikipedia do not suddenly appear, instead, they evolve over time through their immersion in the project's social network. Early in their experience, those leaders get a deeper understanding of the community, and developing a network of contacts, through their connections (weak ties) on the periphery, and later, to the leaders, particularly in the form of strong connection to a leader group.
Pine ✉ 01:24, 22 June 2012 (UTC)
For [20]/ [21]. The authors develop an interesting "measure of controversiality", something that might be of interest to editors at large if it was a more widely popularized and dynamically updated statistic. They look at the patterns of conflicts (edit warring) on Wikipedia articles. The authors find that edit warriors usually are prone to reaching consensus, and the rare cases of articles with never-ending warring involve those that continously attract new editors, who have not yet joined the consensus.
Regarding methodology, the authors decision to filter out articles with under 100 edits as "evidently conflict-free" is a bit problematic, as there are articles with few than 100 edits that have been subject to clear if not overly long edit warring (a recent example: Concerns and controversies related to UEFA Euro 2012). One could also wish that the discussion of the "memory effects", a term mentioned only in the abstract and lead, which the author suggests is significant to understanding of the conflict dynamic, was given more explanation somewhere in the article (the term "memory" itself appears four times in the body and does not seem to be operationalized anywhere). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:16, 25 June 2012 (UTC)
Hi, I've just edited it, trying to fix an edit-conflict. I hope I didn't miss anything.
In my view, the piece is way too long for the Signpost genre. I wonder whether it needs to be fortnightly, in more digestible chunks. Tony (talk) 03:43, 26 June 2012 (UTC)
Two Wikipedia papers were presented during the American Sociological Association conference last week, both focusing on awards. Michael Restivo and Arnout van de Rijt presented "Experimental Study of Informal Rewards in Peer Production" (absract: We test the effects of informal rewards in peer production. Using a randomized, experimental design, we assigned editing awards or “barnstars” to a subset of the 1% most productive Wikipedia contributors. Comparison with the control group shows that receiving a barnstar increases productivity by 60% and makes contributors six times more likely to receive additional barnstars from other community members, revealing that informal rewards significantly impact individual effort.) and Benjamin Mako Hill, Aaron Shaw, and Yochai Benkler presented Status, Social Signaling, and Collective Action: A Field Study of Awards on Wikipedia (abstract: Research into collective action and the provision of public goods has primarily focused on selective incentives as solutions to free rider problems and the tragedy of the commons. By suggesting that groups can reward individual contribution to public gods with increased status, Willer has argued for a sociological mechanism for the provision of public goods through selective incentives. Willer posits a "virtuous circle" where contributors are rewarded with status by other group members and, in response, are motivated to contribute more. This "status theory of collective action" extends findings from earlier studies of awards, which have suggested that awards are an important mechanism for driving contribution. That said, many contributions to real public goods are made anonymously; and there is reason to suspect that not all individuals will be equally susceptible to status-based awards or incentives. At the very least, Willer's theory fails to take into account individual differences in the desire to signal contributions to a public good. We test whether this omission is justified and whether individuals who do not signal status in the context of collective action behave differently from those who do in the presence of a reputation-based award. We analyze evidence from a real field setting using peer-to-peer awards called "barnstars" given in Wikipedia. We show that the social signalers see a boost in their editing behavior where non-signalers do not. We conclude by considering the implications of these findings for theories of collective action.) IIRC both groups told me that their papers have been already presented before and covered by Signpost, so we can probably link this past coverage.
Assigning Students to edit Wikipedia: Four Case Studies (Carver et al.): This article presents a case study of experiences of four professors’ who participated in the Wikipedia:Wikipedia Education Program; 6 courses total (two of four instructors taught two classes each). (We could probably find and link the specific ones? They are not anonymous in the article) Important lessons from the assignments included: 1) the importance of strict deadlines, even for graduate classes; 2) having a dedicated class to editing and policies of Wikipedia, or spreading this over segments of several classes; 3) benefits from having students interact with the Campus Ambassadors and the wider Wikipedia community. Overall, the instructors saw that the student were more highly motivated than in traditional assignments, produced work of higher quality than in traditional assignments, and learned more skills (primarily, related to using Wikipedia, such as being able to judge its reliability better). Wikipedia itself benefited from several dozens created or improved articles, a number of which were featured as Did You Knows. The paper thus presents a useful addition to the emerging literature on teaching with Wikipedia, being one of the first serious and detailed discussions of specific cases of this new educational approach.
Will try to review more tomorrow. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:25, 25 August 2012 (UTC)
Wikipedia: Remembering in the digital age is a master thesis by Simin Michelle Chen. Her works theme is collective memories as represented on Wikipedia; she examins how significant events are portrayed (remembered) on this project, focuses on the Tiananmen Square Protests of 1989. She compares how this event was framed by the articles by New York Times and Xinhua News Agency, and on Wikipedia, where she focuses on the content analysis of Talk:Tiananmen Square protests of 1989 and its archives. Chen finds that the way Wikipedia frames the event is much closer to that of the New York Times than that of the sources preferred by the Chinese government, which, she notes, were "not given an equal voice" (p. 152). She later notes that this (English Wikipedia) article is of major importance to China, but it is not easily influenced by Chinese people, due language barriers, and discrimination against Chinese sources perceived by the Wikipedia community of editors as often unreliable (more subject to censorship and other forms of government manipulation than the Western sources). She notes that this leads to on-Wiki conflicts between contributors with different points of views (she refers to them as "memories" through her work), and usually the contributors who support that Chinese government POV are "silenced" (p. 152). This leads her to conclude that different memories (POVs) are attributed different weights on Wikipedia. While this finding is not revolutionary, her case study up to this point is a valuable contribution to the discussion of Wikipedia biases.
While Chen makes interesting points about the existence of different national biases, which impact editors very frames of reference, and different treatment of various sources, her subsequent critique of Wikipedia's NPOV policy is likely to raise some eyebrows (see also p. 48-50). She argues that NPOV is flawed because "it is based on the assumption that facts are irrefutable" (p.154), but those facts are based on different memories and cultural viewpoints, and thus should be treated equally, instead of some (Western) being given preference. Subsequently, she concludes that Wikipedia contributes to "the broader structures of dominance and Western hegemony in the production of knowledge" (p. 161). While she acknowledges that official Chinese sources may be biased and censored, she does not discuss this in much detail, and instead seems to be arguing that the biases affecting those sources are comparable to the biases affecting Western sources. In other words, she is saying that while some claim Chinese sources are biased, other claim that Western sources are biased, and because English Wikipedia is dominated by the Western editors their bias triumphs - whereas ideally, all sources should be allowed in order to reduce the bias. Therefore, she seems to suggest Wikipedia, in order to reduce the Western bias she perceives, should reject NPOV and accept sources currently deemed as unreliable. Her argument about the English Wikipedia having a Western bias is not very controversial, was discussed by the community before (although Chen does not seem to be aware of it, and does not use the term " systemic bias" in her thesis at all) and reducing this bias (by improving our coverage of non-Western topics) is even one of the Wikimedia Foundation goals. However, while she does not say so directly, it appears to this reviewer that her argument is: "if there are no reliable non-Western sources, we should use the unreliable ones, as this is the only way to reduce the Western bias affecting non-Western topics". Her ending comment that Wikipedia fails to leave to its potential and to deliver "postmodern approach to truth" brings to mind the community discussions about verifiability not truth (the existence of this debates she briefly acknowledges on p. 48).
Overall, Chen's discussion of biases affecting Wikipedia in general, and of Tiananmen Square Protests in particular, is certainly valuable. The thesis however suffers, in this reviewer's opinion, from two major flaws. First, the discussion of Wikipedia's policies such as reliable sources and verifiability (not truth...) seems too short, considering that their critique forms a major part of her conclusions. Second, the argumentation and accompanying value judgements that Wikipedia should stop discriminating against certain memories (POVs) seems not very convincing. Consequently, the thesis seems to spend too much time criticizing Wikipedia for its Western bias, setting it up as a major problem overshadowing all others on Wikipedia, without properly explaining the reasons for why did the Wikipedia community made those decisions (favoring verifiability and reliable sources over inclusion of all viewpoints), and without properly delving into the rich history of those debates on Wikipedia. Chen argues that Wikipedia sacrifices freedom and discriminates against some memories (contributors), which she seems to see as more of a problem that if Wikipedia was to accept unreliable sources and unverifiable claims. Therefore while she correctly points out Wikipedia is affected by a systemic (pro-Western) bias, her argument that Wikipedia should abandon its insistence of the use of reliable and verifiable sources, in order to reduce the said bias, seems much less well argued.
--
I hope this is not too much if a critique. Feel free to tone it down. I liked parts of the paper, but particularly in her conclusion, I really got rather annoyed. It's too much like an essay-rant by somebody who does not like NPOV. Btw, if you can find a link to WMF blog or site or such to back up the fact that countering systemic bias is "one of the Wikimedia Foundation goals" as I write above, it would be nice. I'd assume that others at WMF could help with finding the best source? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 20:15, 26 August 2012 (UTC)
--
Now, on to [22]. The main argument of this paper, wich the authors call the "Low-Hanging Fruit hypothesis", is as follows: "that the larger the site becomes, and the more knowledge it contains, the more difficult it becomes for editors to make novel, lasting contributions. That is, all of the easy articles have already been created, leaving only more difficult topics to write about". The authors break this hypothesis into three smaller ones, more easy to test: (1) a slowdown in edits is observed across many languages with diverse characteristics; (2) articles created earlier are more popular to edit; and (3) articles created earlier are more popular to view. They find a support for all three of the smaller hypotheses, which they use to argue supports their main Low-Hanging Fruit hypothesis. While the study seems well designed with regards to the study of the three child hypothesis, the extrapolation from them to the parent one seems problematic. The authors do not provide a proper operationalization of the terms like "novel", "lasting", "easy/difficult", making it difficult to enter into a discourse without risking miscommunication. This reviewer would point out to the following issues:
Overall, the paper presents four hypothesis, three of which seem to be well supported by data, and contribute to our understanding of Wikipedia, but their main claim seems rather controversial and poorly supported by their data and argumentation.
Hmmm, I guess it's another critique. Sorry, I'll try to find something I like next time :) -- Piotr Konieczny aka Prokonsul Piotrus| reply here 20:55, 26 August 2012 (UTC)
Couldn't really find much more to write about this. Really short for a MT... -- Piotr Konieczny aka Prokonsul Piotrus| reply here 21:16, 26 August 2012 (UTC)
[24] This is yet another paper discussing the experiences of some instructors and student involved in the recent Wikipedia:Global Education Program. Like most of the existing research, the paper is roughly positive in its description of this new educational approach, stressing the importance of deadlines, small introductory assignments familiarizing students with Wikipedia early on in the course, and the importance of good interactions with the community. A poorly justified (or not explained) deletion or removal of content can be quite a stressful experience to students (and the newbie editors are unlikely to realize an explanation may be left in an edit summary or page deletion log). A valuable suggestion in the paper encouraged the instructors (professors) to make edits themselves, so they would be able to discuss editing Wikipedia with students with some first-hand experience, instead of directing students to ambassadors and how-to manuals; and to dedicate some class time to discussing Wikipedia, the assignment, and collective editing. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 21:36, 26 August 2012 (UTC)
I suggest including the information about courts citing Wikipedia described in Wikipedia:Wikipedia Signpost/2012-08-20/In the news. Pine ✉ 09:07, 27 August 2012 (UTC)
Question, in the section:
"(a minor but annoying issue): hypothesis II in incorrectly and confusingly worded in the section dedicated to it: "Older articles (those created earlier) will be more popular to read than more newly created articles" but their study of this hypothesis (no II) is based on the number of edits to the article, not the number of page views (those are analyzed in the subsequent hypothesis no III);"
Is "no" (as in "no III") an abbreviation for number or is it saying 'there is no section III'? If it is an abbreviation for number could it be clarified either by changing to a different notation ( #, number, or such) or putting a period after it to form "no." so it looks like an abbreviation rather than a word? Thank you. RJFJR ( talk) 14:06, 27 August 2012 (UTC)
Hi Tilman, you have a bit of extra time to finish this, as we're pushing back publishing while we wait to hear back from interviewees. Can you have it ready by 18:00 UTC? Thanks very much (as always!), Ed [talk] [majestic titan] 05:10, 28 August 2012 (UTC)
Thanks for the heads up, although I don't have anything to say to the author right now (as he also does not seem to pose a specific question to me/us). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:45, 30 August 2012 (UTC)
--
First, Wikipedia is increasingly likely to reject newcomers contributions, be it in the form of reverts or deletions. Second, it is increasingly likely to meet them with depersonalized messages; the authors cite a study that shows that by mid 2008 over half of new users received their first message in a depersonalized format, usually as a warning from a bot, or an editor using a semi-automated tool. They conclude that there is a correlation between the growing use of various depersonalized tools for dealing with newcomers, and the dropping retention of newcomers. They use of those tools creates a rather negative first impression, making newcomers less likely to stay around. The authors conclude that unwanted but good faithed contributions were handled differently in the early years of the project, in a way that was more personal and less demotivating. Startlingly, the authors find that a significant number of first time editors will make an inquiry about their reverted edit on the talk page of the article they were reverted on, only to be ignored by the more experienced Wikipedians who never check up on the talk pages of the pages they have reverted (authors show in particular the users of semi-automated tools like Huggle or Twinkle are less likely to follow up and enter a discussion with such editors, compared to the editors who do not use such tools). The authors point out that the experienced Wikipedia editors are thus increasingly less likely to follow up on their own Wikipedia:Bold, revert, discuss policy, particularly when dealing with newcomers, who are increasingly assumed as unworthy of being engaged in discussion following a revert, and if engaged, they are simply treated to a depersonalized, demoralizing, and often complex templated warning message.
As a third factor, the authors note that majority of Wikipedia rules were created before 2007 and have not changed much since, and thus new editors face the environment where they have little influence on the rules creation. Further, they often have to face the rule-savvy old editors, automatically falling into the inferior position in most discussions due to their limited understanding of said policies. At the same time, authors argue that it the newcomers are more affected by those policies, compared to the established editors. The authors note that this violates Ostrom's 3rd principle for stable local common pool resource management, by effectively excluding a group that is very vulnerable to certain rules from being able to effectively influence them.
The authors recognize that automated tools and extensive rules are needed to deal with vandalism and manage a complex project, but they caution that the currently evolved costumes and procedures are not sustainable in the long term. They suggest that experienced Wikipedia editors need to be more open to personalized and friendly dealing with unwanted but good faithed contributions, and that Wikipedia needs to refocus its energies from dealing with vandals (a task that the authors conclude has already been sufficiently achieved to guarantee future stability) to mentoring newcomers (a task in which Wikipedia increasingly fails, and which threatens its future survival). Further, the recommend that the newcomers are given a larger voice when it comes to the rules creation and modification.
Overall, the authors present a series of very compelling arguments, and the only complain this reviewer has is that the authors do not discuss the fact that the Wikimedia Foundation and the wider community has recognized similar issues, and has engaged in debates, studies, pilot programs and such aimed to remedy the issue (see for example the WMF Editor Trends Study).
--
This paper, framing itself as part of the ecological psychology field, contribute to the discourse about affordances (property of an object that allows one to take a certain action). They argue that this term can be developed to further our understanding of how individuals perceive their socio-technical environment. The authors refine the term "technology affordances", which they define as "functional and relational properties of the user-technology system". Then use Wikipedia as their case study attempting to demonstrate its value, listing six affordances of Wikipedia (or in other words, they note that editors of Wikipedia can take the following six actions): contribution, control, management, collaboration, self-presentation, broadcasting.
--
The authors attempt to identify what makes Wikipedia articles with geographical coordinates different from others (besides their obvious relation to geographical locations). They rather unsurprisingly find that more developed articles are more likely to have geo-coordinates, and consequently they find that there seems to be a correlation between article quality and having geo-coordinates links. They also find that articles with geo-coordinates are more likely to be linked to, a likely function of them being above-average quality.
--
This paper contributes to the debates on Wikipedia's reliability. The authors find that density of references is correlated with the article length (the longer the article, the more references it will have per given amount of text). They also find that references attract more references (suggesting a form of a snowball mechanism at work) and that majority of references are added in short periods of time by editors who are more experienced, and who are also adding substantial content. The authors thus conclude that referencing is primarily done by a small number of experienced editors, who prefer to work on longer articles, and who drastically raise the article's quality, by both adding more content, and by adding more references.
--
This would probably look good tied to the longer review I presented above.
The authors of this paper experimented with alternative warning messages, introducing a set of shorter and more personalized warnings into those delivered by Huggle in the period of November 8 0 December 9 2011. Unfortunately, the authors are rather unclear on how exactly was the Huggle tool influenced, and whether the community was consulted on that. While in fact the community and Huggle developers have been aware of, discussed and approved of this experiment - here or here - the paper omission to clarify that this was the case can lead to some confusion with regards to research ethics, as a casual reader may assume the researchers have hijacked Huggle without consulting with the community. The wording change was good faithed (making the messages more personalized, friendly and short), and the authors conclude that the new messages they tested proved more conductive to positively influenced new editors who received Level 1 Warnings.
No idea what happened here -- thanks for fixing!
Theo
polisme 15:14, 23 September 2012 (UTC)
I know it's early, but I've been reading [29] and thought I might as well review it. I'll post something here soon. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 22:54, 9 October 2012 (UTC)
Reviewing [30]:
Reviewing [31]
Reviewing [32]:
Incidentally, this reviewer found the authors use of a Firefox add-on Wired-Maker for content analysis rather ingenious, and applauds them for mentioning such a practical methodological tip in their paper.
Brief review of [33]:
Uninspiring, although the authors did put some effort into methodology. I noted two papers I could try reviewing in epad, but don't have access to. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 22:45, 16 November 2012 (UTC)
I was considering reviewing this, March conference, but I think wasn't online till now. Should I? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 19:36, 23 November 2012 (UTC)
Tilman, nice Research report this month! Tony (talk) 10:14, 28 November 2012 (UTC)
Sorry for the late and sparse report, Holidays and New Year makes for a bit of a bad timing for me. Will try to post something very soon - just giving you the heads up! -- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:58, 31 December 2012 (UTC)
All right, Till, I know this is a bit harsh, but the thing is I have researched the SOPA protests extensively (I have a paper of my own on SOPA under review), and I think I know what I am talking about here. If I was reviewing this paper for the journal, I wouldn't accept it, and I thought FM had higher standards. It is a revise and resubmit, at best (c'mon, no methodology, no numbers, where is the author drawing all those claims from??). Sigh. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 14:08, 31 December 2012 (UTC)
I see aspersions were cast in the usual way. No thanks for fixing up the host of glitches. And you didn't even fix the incomprehensible sentence I'd pointed out in MY edit-summary. Tony (talk) 09:18, 2 January 2013 (UTC)
I saw your comment at the signpost op-ed. What about us getting multiple watchlists? Thanks. Biosthmors ( talk) 02:36, 10 January 2013 (UTC)
In other news, did someone review [39]? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:41, 28 January 2013 (UTC)
Starting early, as my next few weeks may be crazy. I wanted to review [40] but I have no access to it. If you could see about anyone being able to send it my way, I'll try to do so, but otherwise I will probably not be able to read it. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:50, 19 February 2013 (UTC)
Hey Tilman, before you have to ask me ;-), you'll have until early Wednesday morning to finish the research report—I'm waiting on email replies about Commons' PotY, so it's no trouble. Thanks for all you do! Ed [talk] [majestic titan] 09:35, 26 February 2013 (UTC)
Reviewing: Determinants of collective intelligence quality: comparison between Wiki and Q&A services in English and Korean users (I thought sb was supposed to review it last month, but since I guess it hasn't and I have the pdf, here it is). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:23, 23 March 2013 (UTC)
In South Korea, Wikipedia lags behind several other services in popularity, such as Naver's KnowledgeiN knowledge market Q&A service. This paper compares English and Korean Wikipedia to the KnowledgeiN service, and analyzes some of the factors involved in how users perceive quality in wikis and Q&A services. About 200 users of each of the three websites participated in the survey. Authors find that perceived quality is helps to determine how useful the users are going to see a given site as. Previous rsearch which suggests that community expertise, size and diversity all contribute to quality is confirmed, and those factors are recognized and valued by the general public. As might be expected, the authors find that users of Q&A sites value expertise of contributors more so than users of wikis. In turn, wikis rely on the size of their community to achieve quality. Predictably, the authors conclude that the smaller Wikipedias such as the Korean one suffer from small community size, and recommend that to improve the quality and popularity of such Wikipedians, more editors should be recruited. The study notes a number of limitations that affected it; notably it did not take into account any possible cultural differences, and does not provide any discussion of why Wikipedia's popularity in Korea is lacking compared to many other websites, such as the KnowledgeiN one.
This is a dissertation, and I don't feel likr reading it all. But the discussion of the Wikipedia student club is interesting, see p. 168+ We can briefly summarize this as (the first?) study of how a Wikipedia student club can influence the student editors, and then cite the quote above
Shane Greenstein attempted to calculatee the monetary value of consumer surplus generated by broadband Internet, focusing on how much value Internet is providing for free (that otherwise people woul be prepared to pay). Wikipedia accounted for up to $50m of that surplus - in other words, Wikipedia provides a good that otherwise people would be willing to buy, spending $50m on it that instead they get to spend on something else.
I'm sorry Tilman, I never saw this message. Ed [talk] [majestic titan] 23:45, 28 March 2013 (UTC)
I recently read a number of papers, let me know if any of those should be reviewed for the next edition: [46], [47], [48] and [49]. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 03:00, 16 April 2013 (UTC)
Reviewing: http://www.emeraldinsight.com/journals.htm?articleid=17087904&show=abstract
The paper offers a much needed comparison data from a population of editors outside English Wikipedia. Most findings related to reasons people start and continue contributing confirm previous studies - important reasons for contributing include desire to share knowledge, gaining recognition, and are reinforced by friendly interactions. Interestingly, the authors find that another significant motivation of "content production and improvement of Wikipedia in local language" also plays a major factor, something that is missing or seen as mostly irrelevant for contributors to the English Wikipedia. The authors also look at reasons for editors to be come less active, an area that is not as well understood. Their findings here again confirm some previous research - ediors may leave because they find to rules too confusing, editors too unfriendly, or have not enough time. They also list some addition reasons not mentioned significently in the existing literature, such as "issues with Persian script; sociocultural characteristics e.g. lack of research-based teaching instruction and preference for ready-to-use information; • strict rules against mass copying and copyright violation; small size of Persian Web content and a shortage of online Persian references." Unfortunately, the paper suffers from small sample (interviews with 15 editors) and does not report statistics or rankings for some of the data, for example making it difficult to conclude or verify which motivations are more or less important. (Reviewer note: the reviewed pre-print copy did not include figures, which may contain the missing data).
Should we wait for a fuller version? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:58, 20 May 2013 (UTC)
http://www.savap.org.pk/journals/ARInt./Vol.4%282%29/2013%284.2-23%29.pdf
The study looks at a tiny sample of nine undergraduates from the Sunway University in Malaysia. The students in the ENGL1050: Thoughts and writing class were assigned to discuss a topic on Wikipedia. Although the paper does not cite any specific page or account name, based on the description provided the account User:ENGL1050 can be identified. Wikipedia was used as a discussion forum, with the instructor(s) and the student using a single account, and all of their edits consisting of editing the User:ENGL1050 page. The students had generally favorite view of the assignment, with majority agreeing that it is a useful tool of learning, collaboration and improving their English skills. Nonetheless it is clear that the instructor(s) is not familiar with the basics of Wikipedia:School and university projects, nor with the basic guidelines such as WP:NOTAFORUM. The described activity had nothing to do with Wikipedia as an encyclopedia, and treated Wikipedia simply as a popular wiki host. (The instructor(s) was likely not aware of the existence of Wikiversity, where such an activity would be within the project scope). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:07, 20 May 2013 (UTC)
This paper poses an interesting question: are there differences between what is popular in different language Wikiepdias? This is measured through the comparing the highest traffic articles at different Wikipedias. The authors chose four Wikipedias: German, English, Spanish and French. The authors have used an open source software for the analysis ( [52]; the paper and the software page are not clear whether it was developed for this project). Using this software the authors obtained 65 most popular articles from 6 random months of 2009. The authors then divided the pages into categories: Entertainment (ENT), Current Issues (CUR), Politics and War (POL), and Geography (GEO), Information and Communication Technologies (ICT), Science (SCI) and Arts and Humanities (ART), Sexuality (SEX). The authors have compiled two tables, the first showing that there are some major differences between the popularity of articles on different Wikipedias. For example entertainment topics form 45% of popular articles on English Wikipedia, but only 16% on Spanish, where in turn the science articles form 24% (compared to only 3% on English). The second table compares the most contributed to content, again noting significant differences between different Wikipedias, as well as suggesting a lack of a major relation between content's area popularity and number of contributors.
Unfortunately, the paper suffers from a number of issues. The authors noted that the division of articles into categories had to be done manually, but the paper does not describe how this was accomplished (this reviewer can't but wonder how did the author deal with classification of an article that would fit more than one category, for example); nor is there any appendix which would list the articles in question. Given the rather surprising findings (which the authors themselves call "most remarkable", and this reviewer would agree that they can raise an eyebrow), this methodological omission unfortunately raises some issues about the reliability of the research done. A number of similar issues plague the paper; for example the tables also contain a "MAIN" category that is not explained anywhere in the paper. The paper does not discuss any potential biases or issues, such as how the results may not be representative of cultural traits, but of short term media news coverage; or why the data was limited only to few months in 2009 and how this could've affected our ability to generalize from it. There may be, for example, seasonal patterns of interests in certain topics; for example, one could hypothesize that science topics would receive more visits during the school year than holiday months; and if holiday months are different in sampled countries, this could be a factor in the popularity of science topics. (On a sidenote, this reviewer would also like to point out that his own paper is cited totally out of context by the authors).
Overall, such exploratory research is certainly valuable, but the authors stop short of any significant analysis of data, in fact noting themselves that the presented data would benefit from a deeper sociological or sociocultural analysis. Unfortunately, there is no indication that their data set has been made publicly available. Nonetheless, despite lack of significant analysis, and methodological issues, the authors' findings are quite intriguing, suggesting that there may be a much more significant difference in coverage of topics by different language Wikipedias than most have suspected so far. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:34, 21 May 2013 (UTC)
Hmmm, not seeing anything that is 100% up my alley but one item:
http://library.iated.org/view/AIBAR2013WIK (abstract, book chapter). Aibar, E., Lerga, M., Lladós, J., Meseguer, A., Minguillón, J. (2013). Wikipedia in Higher Education: an Empirical Study on Faculty Perceptions and Practices. In: Proceedings of the EDULEARN13 Conference. The International Association for Technology, Education and Development (IATED). ISBN. 978-84-616-3822-2
This sounds interesting, I could try to review it but where's the full text? If I get it within 24h I can try to read and review it. Are you going to be at WikiSym/Mania? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 14:02, 28 July 2013 (UTC)
Here you go, reviewing [53]. I hope I am not too harsh, feel free to moderate me down. The authors investigate why the Polish Wikipedia community of Administrators is growing slower than expected, as defined in a decrease in successful RfAs in Polish Wikipedia. The paper presents a useful lit review of related academic work on RfA, a worthy read for all interested in this topic, and is a welcome study of the under-researched population of editors at non-English Wikipedias. Unfortunately the lit review is not tied very strongly to the rest of the paper, which is tied to a major flow of the article: it would've been stronger if the authors engaged with more social science theory, such as the iron law of oligarchy. Unfortunately this conference paper seems to focus more on the computer science dimension, with a developed statistics section and little theory discussion.
The authors suggest at first such a decline may occur because administrators are chosen based on acquaintance, thus creating a closed group to which people without right connections cannot enter. Later, they conclude that this is unlikely, instead pointing to growing expectations about new candidates. Both of those would be valid hypotheses, but neither is clearly tied to any theory or previous study. (Neither does the paper contain any hypothesis descried as such). The author's analysis of the data is problematic; at one point they contradict themselves noting that "[One of the observed phenomena] could indicate, however, that the community is closing up after all" although later their conclusion states "Our conclusion is that it cannot be claimed with certainty that the Polish Wikipedia community is closing up.".
The authors also misunderstand how the WP:RFA process works on English Wikipedia, nothing that one of the key differences between Polish and English Wikipedia is voting, as in "in the case of English version of Wikipedia, new administrators are elected not by voting, but by discussion". That the authors are ready to take such policy claims at face value does cast a little doubt on the applicability of their findings.
Overall, the paper presents some interesting statistical data on trends in an understudied community, and contributes to our understanding of the governance of Wikipedia. The analysis of the received data is however rather lacking, particularly through weak ties to literature on leadership, volunteer's motivation and related social science areas. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 18:08, 28 July 2013 (UTC)
The discussion at User talk:Mdennis (WMF)#Continuing from your volunteer talk page given where we.27re going...? Apparently you're the right person to talk to about that sort of thing so I'd be interested in your thoughts. I'm especially interested to know if the WMF has done any surveys concerning what editor's views on communications are, and if so, what the WMF is doing about the results. If not, why not? It seems obvious to me that communications issues have been behind many of the problems with recent roll outs and so is something the WMF should be looking at but I don't see any sign of you doing so - although of course, and somewhat ironically, you may be but you're not communicating it well. Dpmuk ( talk) 19:28, 21 August 2013 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:43, 25 August 2013 (UTC)
re: [56]
The authors make a rather serious claim: "We find a surprisingly large number of editors who change their behavior and begin focusing more on a particular controversial topic once they are promoted to administrator status." This reviewer does not find it shocking, as he has written about this problem years ago. The authors note that those editors are difficult to understand based on their pattern of edits, but are more easily spotted by analyzing the pattern of votes at RfA, through they also suggest that a relatively simple fix may be helpful - simply increase the threshold of success votes required for a successful RfA may increase the quality of the Wikipedia admin corps.
One may however quibble with "enforcement of neutrality is in the hands of comparatively few, powerful administrators", another attention-drawing claim in the abstract, that however finds little discussion or support in the body. Discussions about NPOV topics are hardly limited to mop'n'bucket wielders, and thus this claim, and the article abstract, may be exaggerating the importance of the findings. Some admins wait until getting the nearly-impossible-to-remove mop before becoming, well, regular editors. As long as they are not abusing their powers - this reviewer is not sure why should we care. What is more relevant, certainly, is how this entire process shows the inefficiency of the RfA, which forces people to hide behind false "I am perfect" personas, as any sign of being a real person (i.e. making errors, being human, etc.) is often enough to threaten to derail that process. Still, this review is not a place for beating that nearly dead horse - but those interested in the RfA reform process should likely read this article in more detail.
This is a new paper in the emerging subfield of "academics and educators attitudes on Wikipedia", which we have covered before (links). This paper benefits from a respectable sample (about 800 respondents from the faculty body of the Open University of Catalonia). The paper confirms a number of previous finding, namely the importance of one's perception of Wikipedia usefulness and quality, which is significantly and positively correlated to whether one will consider using it as a teaching resource. Correspondingly, poor knowledge about Wikipedia in particular and open access and collaborative knowledge creation models in general are negatively correlated with the views on Wikipedia. Having an respected figure (role model) using Wikipedia in teaching is also likely to influence others, through the usual informal peer networks. Individual characteristics (academic rank, teaching experience, age or gender) are not seen as significant. As the authors conclude, there is much work to be done in educating the worlds of education and academia about the basics of Wikipedia - something we should never take for granted.
Still no access, so not much I can do. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 05:40, 23 September 2013 (UTC)
Done at User:Piotrus/Sandbox/Notes#.C5.BBycie_Wirtualnych_Dzikich but I'd suggest waiting with the publication until the English version is at least available for pre-orders at Amazon. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:53, 5 October 2013 (UTC)
Left some other comments in the pad; not seeing any paper I can review more extensively this month. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:49, 21 October 2013 (UTC)
"Google loves Wikipedia". Why? And how about other search engines? This is the question posed by this study, an updated version of research covered in Singpost previously ( Wikipedia:Wikipedia Signpost/2013-08-28/Recent research). Back then, only an abstract of the paper was available; now a draft paper can be accessed. Han-Teng Liao presents interesting data backing up his claim that neither Google nor Wikipedia are unique, rather we are seeing a more generic rule that "search engines favor user-generated encyclopedias". His study's valuable contribution, beside methodology, is the data from the Chinese Internet, through as he notes we need further research on "the cases of Russia (where Yandex dominates) and South Korea (where Naver and other dominate)".
Not seeing anything else that jumps right out at me... -- Piotr Konieczny aka Prokonsul Piotrus| reply here 03:14, 19 November 2013 (UTC)
In light of the recent increase in for-hire editing on Wikipedia, often carried out by PR professionals (link related Signpost article), comes another (link previous related research reported here https://meta.wikimedia.org/wiki/Research:Newsletter/2012-04-30#Wikipedia_in_the_eyes_of_PR_professionals?) timely study. This works examines how familiar the PR professionals (working not only for for-protit organizations,but also for non-profits, educational institutions, government institutions, and others)) are with Wikipedia rules, based on two surveys (from 2012 and 2013). 74% of respondents noted that their institution had a Wikipedia article, a significant (5%) increase over the 2012 survey, through over 50% of the PR professionals do not monitor those articles more often than on quarterly basis. The study confirms that there is a steady but slow increase in direct edits to Wikipedia by PR professionals; 40% of the 2013 survey respondents hadengaged with Wikipedia through editing (with about a quarter of the respondents editing talk pages, and the reminder, directly editing the main space content), compared to 35% of the 2012 survey respondents. Over 60% agree that "editing Wikipedia for a client or company is a common practice". While "posing as someone else to make changes in Wikipedia" is not seen a a common practice, it is nonetheless supported by ~15% of respondents in the USA, and almost 30% elsewhere (through the latter number should be taken tentatively, as 97% of the survey respondents came from the US).
At the same time, approximately two thirds of the respondents do not know of or understand Wikipedia rules on COI/PR and related topics (defined in this study as Wales' 2012 "Bright Line" policy proposal, linked to his comment in a Corporate Representatives for Ethical Wikipedia Engagement (CREWE) from January 10, 2012, 5:56 am, 2012 (editorial note: can we get a permlink to a Facebook comment?)). Of those who had experience editing Wikipedia directly, thus breaking the rule, over a third (36%) did so knowing about it, thus knowingly violating the site's policy.
The significant breadth of ignorance about Wikipedia rules reinforces the point that even a decade after Wikipedia's creation, most of its users do not even realize that it is a project "anyone can edit", much less what it means: 71% respondents replied that they simply "don't know" "How Wikipedia articles about their clients or companies are started" question, which presumably indicates that they do not understand the basic function and capabilities of the article history function. Majority of other respondents (24% total) admit to writing it themselves; 3% hired a PR firm specializing in this task, 1% hired a "Wikipedia firm" (a concept unfortunately not defined in the article), and only 2% note that they "made a request through Request Article Page"). When it comes to existing articles, only 21% of the respondents wait for the public; vast majority of the rest makes edits themselves, with 5% outsourcing this to a specialized PR or "Wikipedia firm".
Respondent s who had directly edited Wikipedia for their company or client said their edits typically “stick” most of the time. Over three quarters noted that their changes stick half the time of more often; only 8% said they never stick, always being reverted. This raises the question about the efficiency of Wikipedia COI-detection practices, as well as of their desirability (are we not reverting those changes because we don't realize they are COI-based, or are they reviewed and left alone as net-positive edits?).
60% of the respondents note that the articles about their clients or companies have factual errors they would like to correct; many observed that potentially reputation-harming errors last for many months, or even years. This statistic poses an interesting question about Wikipedia responsibility to the world: by denying PR people the ability to correct such errors, aren't we hurting our own mission?
Majority of respondents were not satisfied with existing Wikikipedia rules, feeling that the community treats PR professionals unfairly, denying them equal rights in participation; even out of the respondents who tried to follow Wikipedia policies and who raised concerns on article's talk page rather than directly editing them, 10% percents noted that they had to wait weeks to get any response, and 13% said they never received a response.
With regards to the new editors experience, it is also interesting to note that only a quarter of PR professionals felt that making edits was easy, majority complained that editing Wikipedia is time consuming or even "nearly impossible".
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:49, 25 November 2013 (UTC)
The study, based on Wikia wikis, rather than WMF projects (but with significant implication for them as well) found support for the claims that the iron law of oligarchy holds in wikis; i.e. that the wiki transparent and egalitarian models does not prevent the most active contributors from obtaining significant and disproportionate control over those projects. In particular, the study found that as wiki communities grow 1) theys are less likely to add new administrators; 2) the number of edits made by administrators to administrative “project” pages will increase and 3) the number of edits made by experienced contributors that are reverted by administrators also grows. The authors also note that while there are some interesting exceptions to this rule, proving that wikis can, on occasion, function and egalitarian, democratic public spaces, on average "as wikis become larger and more complex, a small group – present at the beginning – will restrict entry into positions of formal authority in the community and account for more administrative activity while using their authority to restrict contributions from experienced community members".
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:05, 25 November 2013 (UTC)
Tilman, nice coverage in the Signpost. Tony (talk) 13:00, 6 December 2013 (UTC)
No idea if regular editors can edit the Signpost but shouldn't "cought" be "caught"? -- NeilN talk to me 05:34, 28 December 2013 (UTC)
My paper on teaching with Wikipedia and academia's attitude to that idea was just published at [64]. I obviously cannot review it myself... :) Cheers, -- Piotr Konieczny aka Prokonsul Piotrus| reply here 15:47, 19 February 2014 (UTC)
Overall, the authors find that the overall quality of the discussions is high, as most of the participants display knowlededge of Wikipedia's policy, particularly on the notability and credibility (or what we would more likely refer to as reliability) of the articles whose deletion is considered. In re, notability far outweighs the second most frequent rationale, credibility (reliability). They confirm that the deletion system works as intended, with decisions made by majority voters.
Interestingly, the authors find that Wcertain topics did tend to trigger more deletion outcomes, said topics being articles about people, for-profit organizations, and definitions. The authors in turn observe that "locations or events are more likely to be kept than expected, and articles about nonprofit organizations and media are more likely to be suggested for other options (e.g., merge, redirect, etc.) than expected". Discussions about people and for-profit organizations were more likely to be unanimous than expected, whereas articles about nonprofit organizations, certain locations, or events were more likely to lead to a nonunanimous discussion. Regarding the SOPA protests influence on deletion debates, the authors find a small and short-lived increase in keep decisions, and tentatively attribute this to editors being impacted by the idea of internet freedom and consequently allowing free Internet publishing.
The authors sum up those observation noting that "the community members of Wikipedia have clear standards for judging the acceptability of a biography or commercial organization article; and such standards are missing or less clear when it comes to the topics on location, event, or nonprofit organization... Thus, one suggestion to the Wikipedia community is to make the criteria of judging these topics more clear or specific with examples, so it will alleviate the ambiguity of the situation.". This reviewer, as a participant of a not insignificant number of deletion discussions as well as those about the associated policies, agrees with said statement. With regards to the wider scheme, the authors conclude that the AfD process is an example of "a democratic deliberation process interested in maintaining information quality in Wikipedia".
Note to TB: it would be nice to link Wikipedia:Arguments to avoid in deletion discussions somewhere, perhaps? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:13, 26 February 2014 (UTC)
The author reviews the linguistic conceptualisation of feminity on (English) Wikipedia, with regards to whether language used to refer to women differs depending on the type of articles it is used in. Specifically, the author analyzed the use of five lexemes (a term which in the context of this study means words): ladylike, girly, girlish, feminine and womanly. The findings confirm that the usage of those terms in non-accidental. The word feminine, most commonly used of of the studied five, correlates primarily to the topics of fashion, sexuality, and to a lesser extent, culture, society and female historical biographies. The second most popular is the word womanly, which in turn correlates with topics of female artists, religion and history. Girlish, the fourth most popular world, correlates most strongly with the biographies of males, as well as with the articles on movies & TV, female entertainers, literature and music. Finally, girly and ladylike, respectively 3rd and 5th in terms of popularity, cluster together and correlate to topics such as movies & TV (animated), Japanese culture, art, tobacco and female athletes. Later, the author also suggests that there is a not insignificant overlap in usage between cluster for girlish and the combined cluster for giry and ladylike. The author concludes that there are three or four different conceptualisations of feminity on Wikipedia, which in more simple terms means, to quote the author, that "people do indeed represent women in different ways when talking about different things [on Wikipedia]", with "girly and girlish having a somewhat frivolous undertone and womanly, feminine and ladylike being of a more serious and reserved nature".
The study does suffer from a few issues: a literature review could be more comprehensive (the paper cites only six works, and not a single one of them from the field of Wikipedia studies), and this reviewer did not find sufficient justification for why the author limited himself to the analysis of only 500 occurrences (total) of the five lexemes analyzed. A futher discussion of how the said 500 cases were selected would likely strengthen the paper.
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 09:17, 27 February 2014 (UTC)
Ed [talk] [majestic titan] 01:11, 24 March 2014 (UTC)
This study examined academics’ awareness of and attitudes towards Wikipedia and open-access journals for academic publishing through a survey of 120 academics carried out in late 2011 and early 2012. This study comes from the same authors who published a similar paper in 2012, reviewed here, which suffered from the a major basic fallacy: Wikipedia is not the place to publish original research academic work. The authors, unfortunately, seem to seem to ignore no original research policy when they write: "There are in general three models in the current movement towards open-access academic publishing: pushing traditional journals towards open access by changing policies; creating open-access journals; and using existing online open-access venue Wikipedia" and "we surveyed academics to understand their perspectives on using Wikipedia for academic publishing in comparison with open-access journals". In the final discussion segment, the authors do acknowledge the existence of the OR policy, where they suggest that certain types or academic papers (reviews) are similar enough to Wikipedia articles that integration of such articles into Wikipedia could be feasible. The authors do provide a valuable literature review noting prior works which analyze the peer review system in Wikipedia, perceptions of Wikipedia in academia, and related issues (through said review is partially split between the introduction and discussion section).
The study provide some interesting findings regarding academics view of benefits of Wikipedia-style peer review and publishing. Most respondents (77 percent) reported reading Wikipedia, and a rather high number (43 percent) reported having made at least one edit, with 15 percent having written an article. Interestingly, as many as four respondents stated that they were "credited for time spent reviewing Wikipedia articles related to their academic careers" in their professional workplaces. The more experience one had with Wikipedia, the more likely one would see advantages in the wiki publishing model. Most common advantages listed were "cost reductions (40 percent), timely review (19 percent), post-publication corrections (52 percent), making articles available before validation (27 percent) and reaching a wider audience (8 percent)." Disadvantages included questionable stability (86 percent), absence of integration with libraries and scholarly search engines (55 percent), lower quality (43 percent), less credibility (57 percent), less academic acceptance (78 percent) and less impact on academia (56 percent).
With regards to Wikipedia peer review, 54 percent of respondents were aware that Wikipedia had a peer-review process and about third of these considered it to be less rigorous than that of scholarly journals; none of the respondents demonstrated any significant experience with the specifics of how Wikipedia articles are reviewed, suggesting that their involvement with the Wikipedia is rather limited. 75% of the survey respondents did not feel comfortable having others edit their papers-in-progress, and over 25% expressed concern about the lack of control over changes made post-publications. Majority of respondents did not also feel comfortable with their work being reviewed by Wikipedians, with the most common concern being unknown qualifications of Wikipedia editors and reviewers.
Perhaps of most value to the Wikipedia community is the analysis of suggestions made by the respondents with regards to making Wikipedia more accepted at the universities. Here, the most common suggestion was “making the promoted peer-reviewed articles searchable from university libraries” and in general, making it more easy to find and identify high quality articles (some functionality as displaying the quality assessment of an article in mainspace already exists in MediaWiki but is implemented as opt-in feature only).
The authors conclude that the academic researchers’ increased familiarity with either open access publishing or wiki publishing is associated with increased comfort with these models; and the academic researchers’ attitudes towards these models are associated with their familiarity, academic environment and professional status. Overall, this study seems like a major improvement over the authors prior 2012 paper, and a valuable paper addressing the topics of the place of Wikipedia in the open publishing movement and the relationship between Wikipedia and academia.
Done with this one, feel free to copyedit. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:36, 29 April 2014 (UTC)
Brief review of [69]
The authors find that readers of English and Chinese Wikipedia from time zones of high Chinese activity browse different categories of pages. Chinese readers visit English Wikipedia about Asian culture (in particular, Japanese and Korean pop culture), as well as about mobile communications and networking technologies. The authors also find that English pages are almost ten times as popular as Chinese pages (through their results are not identifying users by nationality directly, rather focusing on time zone analysis).
In this reviewers opinion, the study suffers from major methodological problems that are serious enough to cast all the findings in doubt. The authors find that only 7603 pages were eligible to be analyzed (had both English and Chinese version), however the Chinese Wikipedia in the studied period had approximately half a million articles; and while many don't have English equivalents yet, to expect that less than 2% did seems rather dubious. Similarly, our own WikiProject China estimates that English Wikipedia has at almost 50,000 China-related articles. That, given that WikiProject assessments are often underestimating the number of relevant topics, and usually don't cover many core topics, suggests that the study missed a vast majority of articles that exist in both languages. It is further unclear how "an English speaking or Chinese speaking time-zone" were operationalized. The authors do not reveal how, if at all, they controlled for the fact that readers of English Wikipedia can also come from countries were English is not a native language, and that there are hundreds of millions of people outside China who live in the five time zones that span China, which overlap with India, half of Russia, Korea and major parts of Southeast Asia. As such, the findings of that study can be more broadly interpreted as "readership patterns of English and Chinese Wikipedia in Asia and the the world, with regards to a small subset of pages that exist both on English and Chinese Wikipedia."
--not done, but not sure if I can continue today... the article is not written very clearly, I am still surprised what get published. Can you figure out if this article studied people from non-Chinese time zones and how did the authors try to control for people such as those from other East Asian countries? --
Piotr Konieczny aka Prokonsul Piotrus|
reply here 05:59, 1 May 2014 (UTC)
A survey of research skills of a group of students at an Australian institution showed that purposeful engaging with Wikipedia, including contributing to it, improved their academic skillset.
[short entry, but the topic is pretty straightforward]
This article describes IntelWiki, a set of MediaWiki tools designed to facilitate new editor's engagement by making research easier. The tool "automatically generates resource recommendations, ranks the references based on the occurrence of salient keywords, and allows users to interact with the recommended references within the Wikipedia editor." The researchers find that volunteers using this tool were more productive, contributing more high quality text. The studied group was composed of sixteen editors with no prior Wikipedia editing experience, who completed two editing tasks in a sandbox wiki, one using a mockup Wikipedia editing interface and Google search engine, and using the IntelWiki interface and reference search engine. The reference suggestion tool developed by the author seems valuable, unfortunately this reviewer was unable to locate any proof that the developer engaged Wikipedia community, or made his code or the tool publicly available for further testing. Further, the research and the thesis does not discuss the differences between their MediaWiki clone and Wikipedia in any significant details. Based on the limited description available, the study's overall conclusions may not be very reliable, as the mockup Wikipedia interface used for comparison seems to be a default MediaWiki clone, lacking many Wikipedia-specific tools; therefore the theme of comparing IntelWiki to Wikipedia is a bit misleading. Overall, while the study is interesting, it is disappointing that the main purpose of this appears to be completing a thesis, with little thought given to actually improving Wikipedia (by developing public tools and/or releasing open code).
Till: please check Wikipedia:Village_pump_(miscellaneous)#Was_.22Intel_wiki.22_tool_discussed_by_the_community_before.3F for any developments, I may have limited net access next week. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:13, 22 June 2014 (UTC)
This paper explores the use of Chinese Wikipedia and Baidu Baike encyclopedia by Chinese microblog ( Twitter, Sina Weibo) users through qualitative and quantitative analyses of Chinese microblog postings. Both encyclopedias are often cited by microblog users, and are highly popular in China to the extent that the words wiki band baidu became verbs meaning to look up content on respective websites, serving as the verb "to google" in contemporary English. One of the study's major focus is the impact of Internet censorship in China; particularly as Wikipedia is not censored - but access to it, and it's discussion in most Chinese websites may be, and in turn Baidu Baike is both censored and more likely to host copyright violating content (thus making it more informative to the vast majority of Internet users not caring about legal technicalities). Despite Baidu Baike copyright violating content, many users still prefer uncensored and more reliable Chinese Wikipedia, through can get frustrated by not being able to access it due to censorship. Whether some Wikipedia content is censored or not is seen by some as a measure of the topic's political sensitivity. The author also suggests that there certain distinguishing characteristic can be observed between groups which prefer one encyclopedia over the other, but does not discuss this in detail, suggesting a very interesting further research avenue. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:46, 22 June 2014 (UTC)
[saving draft]
There are few more works I am willing to review but I don't have access to them. If you can get me copies of them, I can work on them.
Oh, feel free to add "review by Piotrus" or such to each and any of my reviews from now on, I support non-anonymous reviewing.
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:07, 29 July 2014 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 09:19, 26 August 2014 (UTC)
The authors find that the AfC review process is not subject to insurmountable delays; they conclude that "most drafts will be submitted for review quickly and that reviews will happen in a timely manner.". In fact, 2/3 or reviews take place within a day of submission (a figure that positively surprised this reviewer, through a current AfC status report suggests a situation has worsened since: "Severe backlog: 2599 pending submissions"). In either case, the authors find that about a third of so of newcomers using AfC system fail to understand the fact that they need to finalize the process by submitting their drafts to the review at all - a likely indication that the AfC instructions need revising, and that the AfC regulars may want to implement a system of identifying stalled drafts, which in some cases may be ready for mainspace despite having never been officially "submitted" (due to their newbie creator not knowing about this step or carrying it properly). The authors do however stand by their second hypothesis, i.e. they conclude that the AfC articles suffer from not receiving collaborative help they would if they were mainspaced. They discuss the a specific case of an article ( Wikipedia_talk:Articles_for_creation/Dwight_K._Shellman,_Jr/ Wikipedia_talk:Articles_for_creation/Dwight_Shellman). This article has been tagged potentially rescuable ( Wikipedia:WikiProject_Articles_for_creation/G13_rescue) and has been languishing in that state for years, hidden in the AfC namespace, together with many other similarly backlogged articles ( Category:AfC pending submissions by age/Very old), all stuck in a low visibility limbo, prevented from receiving proper Wikipedia-style collaboration-driven improvements (or deletion discussions) as an article in the mainspace would receive.
The authors also identify a number of other reasons that reduce the functionality of the AfC process. As in many other aspects of Wikipedia, negative feedback dominates. Reviewers are rarely thanked for anything, but are more likely to be criticized for passing an article deemed problematic by another editor; thus leading to the mentality that "rejecting articles is safest" (as newbies are less likely to complain about their article's rejection than experienced editors about passing one). AfC also suffers from the same "one reviewer" problem as GA - the reviewer may not always be qualified to carry the review, yet the newbies have little knowledge how to ask for a second opinion. The authors specifically discuss a case of reviewers not familiar with the specific notability criteria: "[despite being notable] an article about an Emmy-award winning TV show from the 1980's was twice declined at AfC, before finally being published 15 months after the draft was started". Presumably if this article was not submitted to a review it would never be deleted from the mainspace. The authors are also critical of the interface of the AfC process, concluding that it is too unfriendly to newbies, instruction wise: "Newcomers do not understand the review process, including how to submit articles for review and the expected timeframe for reviews" and "Newcomers cannot always find the articles they created. They may recreate drafts, so that the same con-tent is created and reviewed multiple times. This is worsened by having multiple article creation spaces(Main, userspace, Wikipediatalk, and the recently-created Draft namespace".
The authors conclude that the AfC works well as a filtering process for the encyclopedia, however "for helping and training newcomers [it] seems inadequate". The AfC succeeds in protecting content under (recently established) G13, in theory allowing newbies to keep fixing it - but many do not take this opportunity. Neither is the community able to deal with this, and thus the authors call for a creation of "a mechanism for editors to find interesting drafts". That said, this reviewers wants to point out that the G13 backlog, while quite interesting (thousands of articles almost ready for main space...), is not the only backlog Wikipedia has to deal with - something that the authors overlook. The G13 backlog is likely partially a result of imperfect AfC design that could be improved, but all such backlogs are also an artifact of the lack of active editors affecting Wikipedia projects on many levels. In either case, AfC regulars should carefully examine the authors suggestions. This reviewer in particular finds the following ideas worth pursuing. 1) Determine which drafts need collaboration and make them more visible to potential editors. Here the authors suggest use of a recent academic model that should help automatically identify valuable articles, and then feeding those articles to SuggestBot. 2) Support newcomers’ first contributions - almost a dead horse at this point, but we know we are not doing enough to be friendly to newcomers. In particular, the authors note that we need to create better mechanisms for newcomers to get help on their draft, and to improve the article creation advice, in particular - the Article Wizard. (As a teacher who introduced hundreds of newcomers to Wikipedia, I can attest myself that the current outreach to newbies on those levels is grossly inadequate).
A final comment, to the community in general: was AfC intended to help newcomers, or was it intended from the start to reduce the strain on new page patrollers by sandboxing the drafts in the first place? One of the roles of AfC is to prevent problematic articles from appearing in the mainspace, and it does seem that in this role it is succeeding quite well. English Wikipedia community has rejected the Wikipedia:Flagged revisions-like tool, but allowed implementation of it on a voluntary basis for newcomers, who in turn may not often realize that by choosing the AfC process, friendly on a surface, they are in fact slow-tracking themselves, and inviting extra-ordinary scrutiny. This leads to a larger question that is worth considering: we, the Wikipedia community of active editors, have declined to have our edits classified as second tier and hidden from public until reviewed, but we are fine pushing this on to the newbies. To what degree this is contributing to the general trend of Wikipedia being less and less friendly to newcomers? Is the resulting quality control worth turning away potential newbies? Would we be here if years ago our first experience with Wikipedia was through AfC?
I was sollicited to provide some feedback on this thread. Tbayer (WMF): I observe that your chronology of events (Siegenthaler, Jimbo's Fiat, the evolution of AFC, the request by members of enWP (including AFC) to have the Draft namespace) is so far out of sync with reality that I can only assume you are fabricating a desired timeline to support your viewpoint. Early on in the "research" that this was based on false premises... The consideration of how many submissions come in daily versus the amount of pages that are promoted is drastically imballanced. The consideration of pages that start out as a ward of AFC that survive 30 days is not a accurate judgement either as some pages may lie unviewed for months only to be deleted. Furthermore your argument with Dodger67 about who came up with the idea for the draft namespace (and demands to disprove his view) shows either your willful ignorance or deliberate incompetence as it took only 2 pages to get to the original RFC where the idea was proposed and accepted including the WikiMedia Bugzilla ticket ID ( WP:DRAFTS which describes the namespace which links to Wikipedia:Village pump (proposals)/Archive 107#Proposed new Draft namespace which is the RFC on Village pump proposals and also includes the Bugzilla ticket ID). In the spirit of open and even handedness that the foundation (and it's employees) are supposed to be espousing, I strongly suggest you apologize to Rodger. CCing Mdennis (WMF) as the community advocate to take this suboptimal interaction back to her supervisors for further action. Hasteur ( talk) 15:26, 19 September 2014 (UTC)
Hi, it was delayed by a week this time: no matter, but should we presume the next RR will be on 24 September or 1 October? Tony (talk) 01:19, 7 September 2014 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:31, 23 September 2014 (UTC)
Update: either etherpad is down or my 'net is borked and I cannot access it anymore. Will try again tomorrow :(
This study analyzed the development of the referencing of 45 articles over nine topic groups related to health and nutrition over a period of five years (2007-2011) (unfortunately, the authors are not very clear on which particular articles were analyzed, and tend to use the concepts of an article and topic group in a rather confusing manner). Authors coded for references (3,029 total), information on editing history, and search ranking in Google, Bing and Yahoo! search engines. The study confirmed that Wikipedia articles are highly ranked by all search engines, with Yahoo! actually being even more "Wikipedia-friendly" than Google. The author shows that (as expected) the articles improve in quality (or at least, number and density of references) over time. Crucially, the authors show that the overall percentage of mainstream news media references has decreased, while references to academic publications increased over that time. By the end of the study period, only article on (or topic group of?) trans fat contained more references to news sources than to academic publications. The authors overall support the description of Wikipedia as a source aiming for reliablity, through are hesitant to call it reliable, pointing out that for example 15% of analyzed references were coded as "outside the main reference type categories or... not be clearly determined". The authors conclude, commendably, that "Wikipedia needs to be high on the agenda for health communication researchers and practitioners." and that "communications professionals in the health field need to be much more actively involved in ensuring that the content on Wikipedia is reliable and well sourced with reliable references". -- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:08, 25 November 2014 (UTC)
This article contributes to the discussion on gender inequalities on Wikipedia. The authors take a novel approach of looking for answers outside the Wikipedia community, thus also tying their reseearch into the analysis of new editors recruitment, motivations, and barriers to contribute. The authors focus their analysis on the role of Internet experiences and skills, and their lack among certain groups. The authors study whether the level of one's skills in digital literacy is related to their chance of becoming a Wikipedia editor, by surveying 547 young adults (aged 21-22) - students at a (presumably American) university, the most convenience sample in academia. The survey was carried in 2009, with a follow-up wave in 2012. The students were asked about their socioeconomic and demographic background, as well as about their level of digital literacy skills. The authors report that "the average respondent's confidence in editing Wikipedia is relatively low" but that "about one in eight students had been given an assignment in class at some point either to edit or create a new entry on Wikipedia" - which likely suggests that the (undisclosed by authors) university was one where at least one member of the faculty participated in the Wikipedia:Education Program. The vast majority (99%) of respondents reported having read an entry on Wikipedia, and over a quarter (28%) have had some experience editing it (interestingly, even when controlling for students who were assigned to edit Wikipedia, the former number still is as high as 20%).
With regards to the gender gap issues, women are much less likely to have contributed to Wikipedia than men (21% to 38%), and that becomes even more divergent when controlling for student assignments (13% to 32%). The authors find indication of gender gap affecting the likelyhood of Wikipedia's contributions: students who are white, economically affluent, male and Internet-experienced are more likely to edit than others. The strongest and statistically significant predictor variables, however, are Internet skills and gender, and regression models show that variables such as race, ethnicity, socioeconomic status, availability, Internet experience, and confidence in editing Wikipedia are not significant. The authors find that the gender becomes more significant as one's digital literacy increases. At low level of Internet skills, the likelyhood of one's contribution to Wikipedia is low, regardless of gender. As one's skills increase, males became much more likely to contribute, but women fall behind. The authors find that women tend to have lower Internet skills than men, which helps explain a part of the Wikipedia gender gap: to contribute to Wikipedia, one needs to have a certain level of digital literacy, and the digital gap is reducing the number of women who have required level of skills. The authors crucially admit that "why women, on average, report lower level understanding of Internet-related terms remains a puzzle. Studies with detailed data about actual skills based on performance tests suggest no gender differences in the observed skills research that looks at self-rated know-how consistently finds gender variation with real consequences for online behavior". This suggests that while men and women have, in reality, similar skills, women are much less confident about them, which in turns makes them much less confident about contributing to (or trying to conribute to) Wikipedia. This, however, is a hypothesis to be confirmed by future research. In the end, the authors do feel confident enough to conclude that "gender and Internet skills likely have a relatively mild interaction with each other, reinforcing the gender gap at the high end of the Internet skills spectrum." In the end, this reviwer finds this study to be a highly valuable one, both for the literature on gender gap and online communities, and for the Wikipedia community and WMF efforts to reduce this gap in our environment.
This book chapter looks at the Wikimedia community as a social movement. The book chapter is interesting as in clearly placing itself in the relatively small body of literature that describe Wikipedia/Wikimedia as a social movement, unfortunately it is primarily a descriptive, rather than an analytical piece, and does not provide any significant theoretical justification for calling the Wikimedia movement a social movement, a weakness amplified by the fact that this work fails to engage with prior relevant body of Wikipedia research, and is only very loosely connected to the literature on social movements.
I think that's all I have time for today, -- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:01, 25 November 2014 (UTC)
This article proposes a way for combining Wikipedia and Online Books Page data, for the purpose of identifying most notable (important, popular, read) authors whose work is about to enter the public domain, in order to facilitate and prioritize digitization of their works. The proposed algorithm may be of interest to members of Wikipedia:WikiProject Books, Wikipedia:WikiProject Libraries, Wikipedia:WikiProject Open, and related ones, as a means generating a Wikipedia:Assessment importance rating and selecting underdeveloped articles for development.
TB, I think we should have one of our librarians develop this further. Ping User:Phoebe? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 10:16, 26 November 2014 (UTC)
|
I appreciate WMF taking a stance. Has the censored version of the logo been uploaded to Commons? I think it would be a great image to use in some of our articles on related issues. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 23:15, 16 November 2011 (UTC)
That's awkward, I think, even if the authors did use it. I'm trying to think of something idiomatic. "outwardly focused" isn't quite it. I can't find an antonym for "self". Tony (talk) 14:48, 27 December 2011 (UTC)
Hi, does ElKevbo have a point? I see that the pdf file says, inter alia, "Under review—do not cite." and "Submitted for confidential review. November 2011". Apart from other concerns, this coverage might be seen to put pressure on the peer reviewers. Tony (talk) 12:34, 28 December 2011 (UTC)
I wish I had more time right now, I am somewhat busy today. But here are short summaries of:
![]() |
The Signpost Barnstar | |
For your thoroughness in the January 30, 2012 Signpost Recent Research report. Pine talk 08:21, 31 January 2012 (UTC) |
I'll try to help and review social science papers again! -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 19:31, 22 February 2012 (UTC)
As previously, I am posting my entries here. Let me know if you'd like me to add them directly to Signpost instead in the future. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 21:26, 22 February 2012 (UTC)
Thanks for the ping.
-- Piotr Konieczny aka Prokonsul Piotrus| talk to me 17:56, 27 February 2012 (UTC)
Also, on the subject of teaching with Wikipedia, this seems relevant, so I suggest we add it:
-- Piotr Konieczny aka Prokonsul Piotrus| talk to me 19:11, 27 February 2012 (UTC)
Ok, I am done with my newest Wikipedia's article, so here are some reviews of works that I found interesting.
Moin Tbayer please take a look at this edit. Cheers Sargoth ( talk) 22:44, 29 February 2012 (UTC)
![]() |
The Teamwork Barnstar |
to all of the contributors to the April 30, 2012 Recent Research report in the Signpost for the good work there! Pine (talk) 07:51, 2 May 2012 (UTC) |
[18]: This article looks at interactions between Wikipedia editors, and the project's governance, visible in the articles on stem cells and transhumanism, and in the analysis of Wikipedia's discussion of userboxes through the prism of Jürgen Habermas universal pragmatics and Mikhail Bakhtin dialogism theories. The authors use those theories, focusing on the qualitative analysis of language used by editors, to argue that Wikipedia has elements of a democracy, and is an example of a Web 2.0 empowering (emancipating) discourse tool. They authors stress that some forms of discourse found online (and on Wikipedia) may be highly irrational, something that some previous arguments for Web 2.0 being a democratic space have often ignored, but they argue that this is in fact not as much of a hindrance as previously expected. The authors remark that on Wikipedians, discourse can develop between editors of widely differing points of view, and that some Wikipedians will engage in "repeated, strategic, and often highly manipulative attempts" to assert personal authority. Such discussions may be very lively, with "that personal, emotional, or humour-based arguments", yet the authors argue that such comments may not be a hindrance; instead, "on many occasions, there is thus a clearer exposition of views that is achieved, in spite of, or perhaps because of, these personal/sometimes vulgar methods of argumentation." In the end, the authors positively comment of the success of Wikipedia's deliberation in reaching consensus, albeit they remark that it can be "fleeting and transitory" on occasion. Unfortunately, the paper does not touch upon the existence of Wikipedia policies such as Wikipedia:Civility and Wikipedia:No personal attacks, which would certainly add to the analysis presented.
On a side note, despite the paper's claim to have received an approval for research through a "University Research Ethics Committee", the fact that the paper discusses, in occasionally critical fashion (example: "[Editor A] claim to authority and ad hominem attacks were met with derision by [Editor B]" (editor names have been replaced by anonymous pseudonyms by me), the editing of specific editors, may raise some eyebrows. As we all know, not all editors are 100% anonymous, and even those who who are have vested reputation in their identities. This raises a question if this paper has done enough to protect the identity and reputation of the editors it cites; at the very least, why weren't the editors usernames changed in the quotes? Their direct identification adds nothing to the article (what is important for the author's argument is the quote itself, not who said it), but makes it easier for others to use the paper in attacking them back.
Regarding WMC, perhaps we should link meta:Notes_on_good_practices_on_Wikipedia_research#Anonymised_re_pseudonymised. Also, I found time to review [19]. -- Piotr Konieczny aka Prokonsul Piotrus| talk to me 16:26, 29 May 2012 (UTC)
The authors collected data from Wikipedia:Request for adminship pages, and tte ties were measured through user talk interaction; core members/leaders were defined as administrators, and periphery editors as non-administrators (this operationalization may raise some doubts about the validity, as there are some very active and prominent members of the community who are not administrators, something that the authors do not address). The authors find that important ties are the early ties to the periphery, and later, ties to the leaders. They also find that overall strong ties are not as important as weak ties, althouth Simmelian ties (two leader groups) are among the most important. The authors conclude that leaders in projects like Wikipedia do not suddenly appear, instead, they evolve over time through their immersion in the project's social network. Early in their experience, those leaders get a deeper understanding of the community, and developing a network of contacts, through their connections (weak ties) on the periphery, and later, to the leaders, particularly in the form of strong connection to a leader group.
Pine ✉ 01:24, 22 June 2012 (UTC)
For [20]/ [21]. The authors develop an interesting "measure of controversiality", something that might be of interest to editors at large if it was a more widely popularized and dynamically updated statistic. They look at the patterns of conflicts (edit warring) on Wikipedia articles. The authors find that edit warriors usually are prone to reaching consensus, and the rare cases of articles with never-ending warring involve those that continously attract new editors, who have not yet joined the consensus.
Regarding methodology, the authors decision to filter out articles with under 100 edits as "evidently conflict-free" is a bit problematic, as there are articles with few than 100 edits that have been subject to clear if not overly long edit warring (a recent example: Concerns and controversies related to UEFA Euro 2012). One could also wish that the discussion of the "memory effects", a term mentioned only in the abstract and lead, which the author suggests is significant to understanding of the conflict dynamic, was given more explanation somewhere in the article (the term "memory" itself appears four times in the body and does not seem to be operationalized anywhere). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:16, 25 June 2012 (UTC)
Hi, I've just edited it, trying to fix an edit-conflict. I hope I didn't miss anything.
In my view, the piece is way too long for the Signpost genre. I wonder whether it needs to be fortnightly, in more digestible chunks. Tony (talk) 03:43, 26 June 2012 (UTC)
Two Wikipedia papers were presented during the American Sociological Association conference last week, both focusing on awards. Michael Restivo and Arnout van de Rijt presented "Experimental Study of Informal Rewards in Peer Production" (absract: We test the effects of informal rewards in peer production. Using a randomized, experimental design, we assigned editing awards or “barnstars” to a subset of the 1% most productive Wikipedia contributors. Comparison with the control group shows that receiving a barnstar increases productivity by 60% and makes contributors six times more likely to receive additional barnstars from other community members, revealing that informal rewards significantly impact individual effort.) and Benjamin Mako Hill, Aaron Shaw, and Yochai Benkler presented Status, Social Signaling, and Collective Action: A Field Study of Awards on Wikipedia (abstract: Research into collective action and the provision of public goods has primarily focused on selective incentives as solutions to free rider problems and the tragedy of the commons. By suggesting that groups can reward individual contribution to public gods with increased status, Willer has argued for a sociological mechanism for the provision of public goods through selective incentives. Willer posits a "virtuous circle" where contributors are rewarded with status by other group members and, in response, are motivated to contribute more. This "status theory of collective action" extends findings from earlier studies of awards, which have suggested that awards are an important mechanism for driving contribution. That said, many contributions to real public goods are made anonymously; and there is reason to suspect that not all individuals will be equally susceptible to status-based awards or incentives. At the very least, Willer's theory fails to take into account individual differences in the desire to signal contributions to a public good. We test whether this omission is justified and whether individuals who do not signal status in the context of collective action behave differently from those who do in the presence of a reputation-based award. We analyze evidence from a real field setting using peer-to-peer awards called "barnstars" given in Wikipedia. We show that the social signalers see a boost in their editing behavior where non-signalers do not. We conclude by considering the implications of these findings for theories of collective action.) IIRC both groups told me that their papers have been already presented before and covered by Signpost, so we can probably link this past coverage.
Assigning Students to edit Wikipedia: Four Case Studies (Carver et al.): This article presents a case study of experiences of four professors’ who participated in the Wikipedia:Wikipedia Education Program; 6 courses total (two of four instructors taught two classes each). (We could probably find and link the specific ones? They are not anonymous in the article) Important lessons from the assignments included: 1) the importance of strict deadlines, even for graduate classes; 2) having a dedicated class to editing and policies of Wikipedia, or spreading this over segments of several classes; 3) benefits from having students interact with the Campus Ambassadors and the wider Wikipedia community. Overall, the instructors saw that the student were more highly motivated than in traditional assignments, produced work of higher quality than in traditional assignments, and learned more skills (primarily, related to using Wikipedia, such as being able to judge its reliability better). Wikipedia itself benefited from several dozens created or improved articles, a number of which were featured as Did You Knows. The paper thus presents a useful addition to the emerging literature on teaching with Wikipedia, being one of the first serious and detailed discussions of specific cases of this new educational approach.
Will try to review more tomorrow. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:25, 25 August 2012 (UTC)
Wikipedia: Remembering in the digital age is a master thesis by Simin Michelle Chen. Her works theme is collective memories as represented on Wikipedia; she examins how significant events are portrayed (remembered) on this project, focuses on the Tiananmen Square Protests of 1989. She compares how this event was framed by the articles by New York Times and Xinhua News Agency, and on Wikipedia, where she focuses on the content analysis of Talk:Tiananmen Square protests of 1989 and its archives. Chen finds that the way Wikipedia frames the event is much closer to that of the New York Times than that of the sources preferred by the Chinese government, which, she notes, were "not given an equal voice" (p. 152). She later notes that this (English Wikipedia) article is of major importance to China, but it is not easily influenced by Chinese people, due language barriers, and discrimination against Chinese sources perceived by the Wikipedia community of editors as often unreliable (more subject to censorship and other forms of government manipulation than the Western sources). She notes that this leads to on-Wiki conflicts between contributors with different points of views (she refers to them as "memories" through her work), and usually the contributors who support that Chinese government POV are "silenced" (p. 152). This leads her to conclude that different memories (POVs) are attributed different weights on Wikipedia. While this finding is not revolutionary, her case study up to this point is a valuable contribution to the discussion of Wikipedia biases.
While Chen makes interesting points about the existence of different national biases, which impact editors very frames of reference, and different treatment of various sources, her subsequent critique of Wikipedia's NPOV policy is likely to raise some eyebrows (see also p. 48-50). She argues that NPOV is flawed because "it is based on the assumption that facts are irrefutable" (p.154), but those facts are based on different memories and cultural viewpoints, and thus should be treated equally, instead of some (Western) being given preference. Subsequently, she concludes that Wikipedia contributes to "the broader structures of dominance and Western hegemony in the production of knowledge" (p. 161). While she acknowledges that official Chinese sources may be biased and censored, she does not discuss this in much detail, and instead seems to be arguing that the biases affecting those sources are comparable to the biases affecting Western sources. In other words, she is saying that while some claim Chinese sources are biased, other claim that Western sources are biased, and because English Wikipedia is dominated by the Western editors their bias triumphs - whereas ideally, all sources should be allowed in order to reduce the bias. Therefore, she seems to suggest Wikipedia, in order to reduce the Western bias she perceives, should reject NPOV and accept sources currently deemed as unreliable. Her argument about the English Wikipedia having a Western bias is not very controversial, was discussed by the community before (although Chen does not seem to be aware of it, and does not use the term " systemic bias" in her thesis at all) and reducing this bias (by improving our coverage of non-Western topics) is even one of the Wikimedia Foundation goals. However, while she does not say so directly, it appears to this reviewer that her argument is: "if there are no reliable non-Western sources, we should use the unreliable ones, as this is the only way to reduce the Western bias affecting non-Western topics". Her ending comment that Wikipedia fails to leave to its potential and to deliver "postmodern approach to truth" brings to mind the community discussions about verifiability not truth (the existence of this debates she briefly acknowledges on p. 48).
Overall, Chen's discussion of biases affecting Wikipedia in general, and of Tiananmen Square Protests in particular, is certainly valuable. The thesis however suffers, in this reviewer's opinion, from two major flaws. First, the discussion of Wikipedia's policies such as reliable sources and verifiability (not truth...) seems too short, considering that their critique forms a major part of her conclusions. Second, the argumentation and accompanying value judgements that Wikipedia should stop discriminating against certain memories (POVs) seems not very convincing. Consequently, the thesis seems to spend too much time criticizing Wikipedia for its Western bias, setting it up as a major problem overshadowing all others on Wikipedia, without properly explaining the reasons for why did the Wikipedia community made those decisions (favoring verifiability and reliable sources over inclusion of all viewpoints), and without properly delving into the rich history of those debates on Wikipedia. Chen argues that Wikipedia sacrifices freedom and discriminates against some memories (contributors), which she seems to see as more of a problem that if Wikipedia was to accept unreliable sources and unverifiable claims. Therefore while she correctly points out Wikipedia is affected by a systemic (pro-Western) bias, her argument that Wikipedia should abandon its insistence of the use of reliable and verifiable sources, in order to reduce the said bias, seems much less well argued.
--
I hope this is not too much if a critique. Feel free to tone it down. I liked parts of the paper, but particularly in her conclusion, I really got rather annoyed. It's too much like an essay-rant by somebody who does not like NPOV. Btw, if you can find a link to WMF blog or site or such to back up the fact that countering systemic bias is "one of the Wikimedia Foundation goals" as I write above, it would be nice. I'd assume that others at WMF could help with finding the best source? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 20:15, 26 August 2012 (UTC)
--
Now, on to [22]. The main argument of this paper, wich the authors call the "Low-Hanging Fruit hypothesis", is as follows: "that the larger the site becomes, and the more knowledge it contains, the more difficult it becomes for editors to make novel, lasting contributions. That is, all of the easy articles have already been created, leaving only more difficult topics to write about". The authors break this hypothesis into three smaller ones, more easy to test: (1) a slowdown in edits is observed across many languages with diverse characteristics; (2) articles created earlier are more popular to edit; and (3) articles created earlier are more popular to view. They find a support for all three of the smaller hypotheses, which they use to argue supports their main Low-Hanging Fruit hypothesis. While the study seems well designed with regards to the study of the three child hypothesis, the extrapolation from them to the parent one seems problematic. The authors do not provide a proper operationalization of the terms like "novel", "lasting", "easy/difficult", making it difficult to enter into a discourse without risking miscommunication. This reviewer would point out to the following issues:
Overall, the paper presents four hypothesis, three of which seem to be well supported by data, and contribute to our understanding of Wikipedia, but their main claim seems rather controversial and poorly supported by their data and argumentation.
Hmmm, I guess it's another critique. Sorry, I'll try to find something I like next time :) -- Piotr Konieczny aka Prokonsul Piotrus| reply here 20:55, 26 August 2012 (UTC)
Couldn't really find much more to write about this. Really short for a MT... -- Piotr Konieczny aka Prokonsul Piotrus| reply here 21:16, 26 August 2012 (UTC)
[24] This is yet another paper discussing the experiences of some instructors and student involved in the recent Wikipedia:Global Education Program. Like most of the existing research, the paper is roughly positive in its description of this new educational approach, stressing the importance of deadlines, small introductory assignments familiarizing students with Wikipedia early on in the course, and the importance of good interactions with the community. A poorly justified (or not explained) deletion or removal of content can be quite a stressful experience to students (and the newbie editors are unlikely to realize an explanation may be left in an edit summary or page deletion log). A valuable suggestion in the paper encouraged the instructors (professors) to make edits themselves, so they would be able to discuss editing Wikipedia with students with some first-hand experience, instead of directing students to ambassadors and how-to manuals; and to dedicate some class time to discussing Wikipedia, the assignment, and collective editing. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 21:36, 26 August 2012 (UTC)
I suggest including the information about courts citing Wikipedia described in Wikipedia:Wikipedia Signpost/2012-08-20/In the news. Pine ✉ 09:07, 27 August 2012 (UTC)
Question, in the section:
"(a minor but annoying issue): hypothesis II in incorrectly and confusingly worded in the section dedicated to it: "Older articles (those created earlier) will be more popular to read than more newly created articles" but their study of this hypothesis (no II) is based on the number of edits to the article, not the number of page views (those are analyzed in the subsequent hypothesis no III);"
Is "no" (as in "no III") an abbreviation for number or is it saying 'there is no section III'? If it is an abbreviation for number could it be clarified either by changing to a different notation ( #, number, or such) or putting a period after it to form "no." so it looks like an abbreviation rather than a word? Thank you. RJFJR ( talk) 14:06, 27 August 2012 (UTC)
Hi Tilman, you have a bit of extra time to finish this, as we're pushing back publishing while we wait to hear back from interviewees. Can you have it ready by 18:00 UTC? Thanks very much (as always!), Ed [talk] [majestic titan] 05:10, 28 August 2012 (UTC)
Thanks for the heads up, although I don't have anything to say to the author right now (as he also does not seem to pose a specific question to me/us). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:45, 30 August 2012 (UTC)
--
First, Wikipedia is increasingly likely to reject newcomers contributions, be it in the form of reverts or deletions. Second, it is increasingly likely to meet them with depersonalized messages; the authors cite a study that shows that by mid 2008 over half of new users received their first message in a depersonalized format, usually as a warning from a bot, or an editor using a semi-automated tool. They conclude that there is a correlation between the growing use of various depersonalized tools for dealing with newcomers, and the dropping retention of newcomers. They use of those tools creates a rather negative first impression, making newcomers less likely to stay around. The authors conclude that unwanted but good faithed contributions were handled differently in the early years of the project, in a way that was more personal and less demotivating. Startlingly, the authors find that a significant number of first time editors will make an inquiry about their reverted edit on the talk page of the article they were reverted on, only to be ignored by the more experienced Wikipedians who never check up on the talk pages of the pages they have reverted (authors show in particular the users of semi-automated tools like Huggle or Twinkle are less likely to follow up and enter a discussion with such editors, compared to the editors who do not use such tools). The authors point out that the experienced Wikipedia editors are thus increasingly less likely to follow up on their own Wikipedia:Bold, revert, discuss policy, particularly when dealing with newcomers, who are increasingly assumed as unworthy of being engaged in discussion following a revert, and if engaged, they are simply treated to a depersonalized, demoralizing, and often complex templated warning message.
As a third factor, the authors note that majority of Wikipedia rules were created before 2007 and have not changed much since, and thus new editors face the environment where they have little influence on the rules creation. Further, they often have to face the rule-savvy old editors, automatically falling into the inferior position in most discussions due to their limited understanding of said policies. At the same time, authors argue that it the newcomers are more affected by those policies, compared to the established editors. The authors note that this violates Ostrom's 3rd principle for stable local common pool resource management, by effectively excluding a group that is very vulnerable to certain rules from being able to effectively influence them.
The authors recognize that automated tools and extensive rules are needed to deal with vandalism and manage a complex project, but they caution that the currently evolved costumes and procedures are not sustainable in the long term. They suggest that experienced Wikipedia editors need to be more open to personalized and friendly dealing with unwanted but good faithed contributions, and that Wikipedia needs to refocus its energies from dealing with vandals (a task that the authors conclude has already been sufficiently achieved to guarantee future stability) to mentoring newcomers (a task in which Wikipedia increasingly fails, and which threatens its future survival). Further, the recommend that the newcomers are given a larger voice when it comes to the rules creation and modification.
Overall, the authors present a series of very compelling arguments, and the only complain this reviewer has is that the authors do not discuss the fact that the Wikimedia Foundation and the wider community has recognized similar issues, and has engaged in debates, studies, pilot programs and such aimed to remedy the issue (see for example the WMF Editor Trends Study).
--
This paper, framing itself as part of the ecological psychology field, contribute to the discourse about affordances (property of an object that allows one to take a certain action). They argue that this term can be developed to further our understanding of how individuals perceive their socio-technical environment. The authors refine the term "technology affordances", which they define as "functional and relational properties of the user-technology system". Then use Wikipedia as their case study attempting to demonstrate its value, listing six affordances of Wikipedia (or in other words, they note that editors of Wikipedia can take the following six actions): contribution, control, management, collaboration, self-presentation, broadcasting.
--
The authors attempt to identify what makes Wikipedia articles with geographical coordinates different from others (besides their obvious relation to geographical locations). They rather unsurprisingly find that more developed articles are more likely to have geo-coordinates, and consequently they find that there seems to be a correlation between article quality and having geo-coordinates links. They also find that articles with geo-coordinates are more likely to be linked to, a likely function of them being above-average quality.
--
This paper contributes to the debates on Wikipedia's reliability. The authors find that density of references is correlated with the article length (the longer the article, the more references it will have per given amount of text). They also find that references attract more references (suggesting a form of a snowball mechanism at work) and that majority of references are added in short periods of time by editors who are more experienced, and who are also adding substantial content. The authors thus conclude that referencing is primarily done by a small number of experienced editors, who prefer to work on longer articles, and who drastically raise the article's quality, by both adding more content, and by adding more references.
--
This would probably look good tied to the longer review I presented above.
The authors of this paper experimented with alternative warning messages, introducing a set of shorter and more personalized warnings into those delivered by Huggle in the period of November 8 0 December 9 2011. Unfortunately, the authors are rather unclear on how exactly was the Huggle tool influenced, and whether the community was consulted on that. While in fact the community and Huggle developers have been aware of, discussed and approved of this experiment - here or here - the paper omission to clarify that this was the case can lead to some confusion with regards to research ethics, as a casual reader may assume the researchers have hijacked Huggle without consulting with the community. The wording change was good faithed (making the messages more personalized, friendly and short), and the authors conclude that the new messages they tested proved more conductive to positively influenced new editors who received Level 1 Warnings.
No idea what happened here -- thanks for fixing!
Theo
polisme 15:14, 23 September 2012 (UTC)
I know it's early, but I've been reading [29] and thought I might as well review it. I'll post something here soon. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 22:54, 9 October 2012 (UTC)
Reviewing [30]:
Reviewing [31]
Reviewing [32]:
Incidentally, this reviewer found the authors use of a Firefox add-on Wired-Maker for content analysis rather ingenious, and applauds them for mentioning such a practical methodological tip in their paper.
Brief review of [33]:
Uninspiring, although the authors did put some effort into methodology. I noted two papers I could try reviewing in epad, but don't have access to. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 22:45, 16 November 2012 (UTC)
I was considering reviewing this, March conference, but I think wasn't online till now. Should I? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 19:36, 23 November 2012 (UTC)
Tilman, nice Research report this month! Tony (talk) 10:14, 28 November 2012 (UTC)
Sorry for the late and sparse report, Holidays and New Year makes for a bit of a bad timing for me. Will try to post something very soon - just giving you the heads up! -- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:58, 31 December 2012 (UTC)
All right, Till, I know this is a bit harsh, but the thing is I have researched the SOPA protests extensively (I have a paper of my own on SOPA under review), and I think I know what I am talking about here. If I was reviewing this paper for the journal, I wouldn't accept it, and I thought FM had higher standards. It is a revise and resubmit, at best (c'mon, no methodology, no numbers, where is the author drawing all those claims from??). Sigh. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 14:08, 31 December 2012 (UTC)
I see aspersions were cast in the usual way. No thanks for fixing up the host of glitches. And you didn't even fix the incomprehensible sentence I'd pointed out in MY edit-summary. Tony (talk) 09:18, 2 January 2013 (UTC)
I saw your comment at the signpost op-ed. What about us getting multiple watchlists? Thanks. Biosthmors ( talk) 02:36, 10 January 2013 (UTC)
In other news, did someone review [39]? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:41, 28 January 2013 (UTC)
Starting early, as my next few weeks may be crazy. I wanted to review [40] but I have no access to it. If you could see about anyone being able to send it my way, I'll try to do so, but otherwise I will probably not be able to read it. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 16:50, 19 February 2013 (UTC)
Hey Tilman, before you have to ask me ;-), you'll have until early Wednesday morning to finish the research report—I'm waiting on email replies about Commons' PotY, so it's no trouble. Thanks for all you do! Ed [talk] [majestic titan] 09:35, 26 February 2013 (UTC)
Reviewing: Determinants of collective intelligence quality: comparison between Wiki and Q&A services in English and Korean users (I thought sb was supposed to review it last month, but since I guess it hasn't and I have the pdf, here it is). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:23, 23 March 2013 (UTC)
In South Korea, Wikipedia lags behind several other services in popularity, such as Naver's KnowledgeiN knowledge market Q&A service. This paper compares English and Korean Wikipedia to the KnowledgeiN service, and analyzes some of the factors involved in how users perceive quality in wikis and Q&A services. About 200 users of each of the three websites participated in the survey. Authors find that perceived quality is helps to determine how useful the users are going to see a given site as. Previous rsearch which suggests that community expertise, size and diversity all contribute to quality is confirmed, and those factors are recognized and valued by the general public. As might be expected, the authors find that users of Q&A sites value expertise of contributors more so than users of wikis. In turn, wikis rely on the size of their community to achieve quality. Predictably, the authors conclude that the smaller Wikipedias such as the Korean one suffer from small community size, and recommend that to improve the quality and popularity of such Wikipedians, more editors should be recruited. The study notes a number of limitations that affected it; notably it did not take into account any possible cultural differences, and does not provide any discussion of why Wikipedia's popularity in Korea is lacking compared to many other websites, such as the KnowledgeiN one.
This is a dissertation, and I don't feel likr reading it all. But the discussion of the Wikipedia student club is interesting, see p. 168+ We can briefly summarize this as (the first?) study of how a Wikipedia student club can influence the student editors, and then cite the quote above
Shane Greenstein attempted to calculatee the monetary value of consumer surplus generated by broadband Internet, focusing on how much value Internet is providing for free (that otherwise people woul be prepared to pay). Wikipedia accounted for up to $50m of that surplus - in other words, Wikipedia provides a good that otherwise people would be willing to buy, spending $50m on it that instead they get to spend on something else.
I'm sorry Tilman, I never saw this message. Ed [talk] [majestic titan] 23:45, 28 March 2013 (UTC)
I recently read a number of papers, let me know if any of those should be reviewed for the next edition: [46], [47], [48] and [49]. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 03:00, 16 April 2013 (UTC)
Reviewing: http://www.emeraldinsight.com/journals.htm?articleid=17087904&show=abstract
The paper offers a much needed comparison data from a population of editors outside English Wikipedia. Most findings related to reasons people start and continue contributing confirm previous studies - important reasons for contributing include desire to share knowledge, gaining recognition, and are reinforced by friendly interactions. Interestingly, the authors find that another significant motivation of "content production and improvement of Wikipedia in local language" also plays a major factor, something that is missing or seen as mostly irrelevant for contributors to the English Wikipedia. The authors also look at reasons for editors to be come less active, an area that is not as well understood. Their findings here again confirm some previous research - ediors may leave because they find to rules too confusing, editors too unfriendly, or have not enough time. They also list some addition reasons not mentioned significently in the existing literature, such as "issues with Persian script; sociocultural characteristics e.g. lack of research-based teaching instruction and preference for ready-to-use information; • strict rules against mass copying and copyright violation; small size of Persian Web content and a shortage of online Persian references." Unfortunately, the paper suffers from small sample (interviews with 15 editors) and does not report statistics or rankings for some of the data, for example making it difficult to conclude or verify which motivations are more or less important. (Reviewer note: the reviewed pre-print copy did not include figures, which may contain the missing data).
Should we wait for a fuller version? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:58, 20 May 2013 (UTC)
http://www.savap.org.pk/journals/ARInt./Vol.4%282%29/2013%284.2-23%29.pdf
The study looks at a tiny sample of nine undergraduates from the Sunway University in Malaysia. The students in the ENGL1050: Thoughts and writing class were assigned to discuss a topic on Wikipedia. Although the paper does not cite any specific page or account name, based on the description provided the account User:ENGL1050 can be identified. Wikipedia was used as a discussion forum, with the instructor(s) and the student using a single account, and all of their edits consisting of editing the User:ENGL1050 page. The students had generally favorite view of the assignment, with majority agreeing that it is a useful tool of learning, collaboration and improving their English skills. Nonetheless it is clear that the instructor(s) is not familiar with the basics of Wikipedia:School and university projects, nor with the basic guidelines such as WP:NOTAFORUM. The described activity had nothing to do with Wikipedia as an encyclopedia, and treated Wikipedia simply as a popular wiki host. (The instructor(s) was likely not aware of the existence of Wikiversity, where such an activity would be within the project scope). -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:07, 20 May 2013 (UTC)
This paper poses an interesting question: are there differences between what is popular in different language Wikiepdias? This is measured through the comparing the highest traffic articles at different Wikipedias. The authors chose four Wikipedias: German, English, Spanish and French. The authors have used an open source software for the analysis ( [52]; the paper and the software page are not clear whether it was developed for this project). Using this software the authors obtained 65 most popular articles from 6 random months of 2009. The authors then divided the pages into categories: Entertainment (ENT), Current Issues (CUR), Politics and War (POL), and Geography (GEO), Information and Communication Technologies (ICT), Science (SCI) and Arts and Humanities (ART), Sexuality (SEX). The authors have compiled two tables, the first showing that there are some major differences between the popularity of articles on different Wikipedias. For example entertainment topics form 45% of popular articles on English Wikipedia, but only 16% on Spanish, where in turn the science articles form 24% (compared to only 3% on English). The second table compares the most contributed to content, again noting significant differences between different Wikipedias, as well as suggesting a lack of a major relation between content's area popularity and number of contributors.
Unfortunately, the paper suffers from a number of issues. The authors noted that the division of articles into categories had to be done manually, but the paper does not describe how this was accomplished (this reviewer can't but wonder how did the author deal with classification of an article that would fit more than one category, for example); nor is there any appendix which would list the articles in question. Given the rather surprising findings (which the authors themselves call "most remarkable", and this reviewer would agree that they can raise an eyebrow), this methodological omission unfortunately raises some issues about the reliability of the research done. A number of similar issues plague the paper; for example the tables also contain a "MAIN" category that is not explained anywhere in the paper. The paper does not discuss any potential biases or issues, such as how the results may not be representative of cultural traits, but of short term media news coverage; or why the data was limited only to few months in 2009 and how this could've affected our ability to generalize from it. There may be, for example, seasonal patterns of interests in certain topics; for example, one could hypothesize that science topics would receive more visits during the school year than holiday months; and if holiday months are different in sampled countries, this could be a factor in the popularity of science topics. (On a sidenote, this reviewer would also like to point out that his own paper is cited totally out of context by the authors).
Overall, such exploratory research is certainly valuable, but the authors stop short of any significant analysis of data, in fact noting themselves that the presented data would benefit from a deeper sociological or sociocultural analysis. Unfortunately, there is no indication that their data set has been made publicly available. Nonetheless, despite lack of significant analysis, and methodological issues, the authors' findings are quite intriguing, suggesting that there may be a much more significant difference in coverage of topics by different language Wikipedias than most have suspected so far. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:34, 21 May 2013 (UTC)
Hmmm, not seeing anything that is 100% up my alley but one item:
http://library.iated.org/view/AIBAR2013WIK (abstract, book chapter). Aibar, E., Lerga, M., Lladós, J., Meseguer, A., Minguillón, J. (2013). Wikipedia in Higher Education: an Empirical Study on Faculty Perceptions and Practices. In: Proceedings of the EDULEARN13 Conference. The International Association for Technology, Education and Development (IATED). ISBN. 978-84-616-3822-2
This sounds interesting, I could try to review it but where's the full text? If I get it within 24h I can try to read and review it. Are you going to be at WikiSym/Mania? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 14:02, 28 July 2013 (UTC)
Here you go, reviewing [53]. I hope I am not too harsh, feel free to moderate me down. The authors investigate why the Polish Wikipedia community of Administrators is growing slower than expected, as defined in a decrease in successful RfAs in Polish Wikipedia. The paper presents a useful lit review of related academic work on RfA, a worthy read for all interested in this topic, and is a welcome study of the under-researched population of editors at non-English Wikipedias. Unfortunately the lit review is not tied very strongly to the rest of the paper, which is tied to a major flow of the article: it would've been stronger if the authors engaged with more social science theory, such as the iron law of oligarchy. Unfortunately this conference paper seems to focus more on the computer science dimension, with a developed statistics section and little theory discussion.
The authors suggest at first such a decline may occur because administrators are chosen based on acquaintance, thus creating a closed group to which people without right connections cannot enter. Later, they conclude that this is unlikely, instead pointing to growing expectations about new candidates. Both of those would be valid hypotheses, but neither is clearly tied to any theory or previous study. (Neither does the paper contain any hypothesis descried as such). The author's analysis of the data is problematic; at one point they contradict themselves noting that "[One of the observed phenomena] could indicate, however, that the community is closing up after all" although later their conclusion states "Our conclusion is that it cannot be claimed with certainty that the Polish Wikipedia community is closing up.".
The authors also misunderstand how the WP:RFA process works on English Wikipedia, nothing that one of the key differences between Polish and English Wikipedia is voting, as in "in the case of English version of Wikipedia, new administrators are elected not by voting, but by discussion". That the authors are ready to take such policy claims at face value does cast a little doubt on the applicability of their findings.
Overall, the paper presents some interesting statistical data on trends in an understudied community, and contributes to our understanding of the governance of Wikipedia. The analysis of the received data is however rather lacking, particularly through weak ties to literature on leadership, volunteer's motivation and related social science areas. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 18:08, 28 July 2013 (UTC)
The discussion at User talk:Mdennis (WMF)#Continuing from your volunteer talk page given where we.27re going...? Apparently you're the right person to talk to about that sort of thing so I'd be interested in your thoughts. I'm especially interested to know if the WMF has done any surveys concerning what editor's views on communications are, and if so, what the WMF is doing about the results. If not, why not? It seems obvious to me that communications issues have been behind many of the problems with recent roll outs and so is something the WMF should be looking at but I don't see any sign of you doing so - although of course, and somewhat ironically, you may be but you're not communicating it well. Dpmuk ( talk) 19:28, 21 August 2013 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:43, 25 August 2013 (UTC)
re: [56]
The authors make a rather serious claim: "We find a surprisingly large number of editors who change their behavior and begin focusing more on a particular controversial topic once they are promoted to administrator status." This reviewer does not find it shocking, as he has written about this problem years ago. The authors note that those editors are difficult to understand based on their pattern of edits, but are more easily spotted by analyzing the pattern of votes at RfA, through they also suggest that a relatively simple fix may be helpful - simply increase the threshold of success votes required for a successful RfA may increase the quality of the Wikipedia admin corps.
One may however quibble with "enforcement of neutrality is in the hands of comparatively few, powerful administrators", another attention-drawing claim in the abstract, that however finds little discussion or support in the body. Discussions about NPOV topics are hardly limited to mop'n'bucket wielders, and thus this claim, and the article abstract, may be exaggerating the importance of the findings. Some admins wait until getting the nearly-impossible-to-remove mop before becoming, well, regular editors. As long as they are not abusing their powers - this reviewer is not sure why should we care. What is more relevant, certainly, is how this entire process shows the inefficiency of the RfA, which forces people to hide behind false "I am perfect" personas, as any sign of being a real person (i.e. making errors, being human, etc.) is often enough to threaten to derail that process. Still, this review is not a place for beating that nearly dead horse - but those interested in the RfA reform process should likely read this article in more detail.
This is a new paper in the emerging subfield of "academics and educators attitudes on Wikipedia", which we have covered before (links). This paper benefits from a respectable sample (about 800 respondents from the faculty body of the Open University of Catalonia). The paper confirms a number of previous finding, namely the importance of one's perception of Wikipedia usefulness and quality, which is significantly and positively correlated to whether one will consider using it as a teaching resource. Correspondingly, poor knowledge about Wikipedia in particular and open access and collaborative knowledge creation models in general are negatively correlated with the views on Wikipedia. Having an respected figure (role model) using Wikipedia in teaching is also likely to influence others, through the usual informal peer networks. Individual characteristics (academic rank, teaching experience, age or gender) are not seen as significant. As the authors conclude, there is much work to be done in educating the worlds of education and academia about the basics of Wikipedia - something we should never take for granted.
Still no access, so not much I can do. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 05:40, 23 September 2013 (UTC)
Done at User:Piotrus/Sandbox/Notes#.C5.BBycie_Wirtualnych_Dzikich but I'd suggest waiting with the publication until the English version is at least available for pre-orders at Amazon. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 13:53, 5 October 2013 (UTC)
Left some other comments in the pad; not seeing any paper I can review more extensively this month. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:49, 21 October 2013 (UTC)
"Google loves Wikipedia". Why? And how about other search engines? This is the question posed by this study, an updated version of research covered in Singpost previously ( Wikipedia:Wikipedia Signpost/2013-08-28/Recent research). Back then, only an abstract of the paper was available; now a draft paper can be accessed. Han-Teng Liao presents interesting data backing up his claim that neither Google nor Wikipedia are unique, rather we are seeing a more generic rule that "search engines favor user-generated encyclopedias". His study's valuable contribution, beside methodology, is the data from the Chinese Internet, through as he notes we need further research on "the cases of Russia (where Yandex dominates) and South Korea (where Naver and other dominate)".
Not seeing anything else that jumps right out at me... -- Piotr Konieczny aka Prokonsul Piotrus| reply here 03:14, 19 November 2013 (UTC)
In light of the recent increase in for-hire editing on Wikipedia, often carried out by PR professionals (link related Signpost article), comes another (link previous related research reported here https://meta.wikimedia.org/wiki/Research:Newsletter/2012-04-30#Wikipedia_in_the_eyes_of_PR_professionals?) timely study. This works examines how familiar the PR professionals (working not only for for-protit organizations,but also for non-profits, educational institutions, government institutions, and others)) are with Wikipedia rules, based on two surveys (from 2012 and 2013). 74% of respondents noted that their institution had a Wikipedia article, a significant (5%) increase over the 2012 survey, through over 50% of the PR professionals do not monitor those articles more often than on quarterly basis. The study confirms that there is a steady but slow increase in direct edits to Wikipedia by PR professionals; 40% of the 2013 survey respondents hadengaged with Wikipedia through editing (with about a quarter of the respondents editing talk pages, and the reminder, directly editing the main space content), compared to 35% of the 2012 survey respondents. Over 60% agree that "editing Wikipedia for a client or company is a common practice". While "posing as someone else to make changes in Wikipedia" is not seen a a common practice, it is nonetheless supported by ~15% of respondents in the USA, and almost 30% elsewhere (through the latter number should be taken tentatively, as 97% of the survey respondents came from the US).
At the same time, approximately two thirds of the respondents do not know of or understand Wikipedia rules on COI/PR and related topics (defined in this study as Wales' 2012 "Bright Line" policy proposal, linked to his comment in a Corporate Representatives for Ethical Wikipedia Engagement (CREWE) from January 10, 2012, 5:56 am, 2012 (editorial note: can we get a permlink to a Facebook comment?)). Of those who had experience editing Wikipedia directly, thus breaking the rule, over a third (36%) did so knowing about it, thus knowingly violating the site's policy.
The significant breadth of ignorance about Wikipedia rules reinforces the point that even a decade after Wikipedia's creation, most of its users do not even realize that it is a project "anyone can edit", much less what it means: 71% respondents replied that they simply "don't know" "How Wikipedia articles about their clients or companies are started" question, which presumably indicates that they do not understand the basic function and capabilities of the article history function. Majority of other respondents (24% total) admit to writing it themselves; 3% hired a PR firm specializing in this task, 1% hired a "Wikipedia firm" (a concept unfortunately not defined in the article), and only 2% note that they "made a request through Request Article Page"). When it comes to existing articles, only 21% of the respondents wait for the public; vast majority of the rest makes edits themselves, with 5% outsourcing this to a specialized PR or "Wikipedia firm".
Respondent s who had directly edited Wikipedia for their company or client said their edits typically “stick” most of the time. Over three quarters noted that their changes stick half the time of more often; only 8% said they never stick, always being reverted. This raises the question about the efficiency of Wikipedia COI-detection practices, as well as of their desirability (are we not reverting those changes because we don't realize they are COI-based, or are they reviewed and left alone as net-positive edits?).
60% of the respondents note that the articles about their clients or companies have factual errors they would like to correct; many observed that potentially reputation-harming errors last for many months, or even years. This statistic poses an interesting question about Wikipedia responsibility to the world: by denying PR people the ability to correct such errors, aren't we hurting our own mission?
Majority of respondents were not satisfied with existing Wikikipedia rules, feeling that the community treats PR professionals unfairly, denying them equal rights in participation; even out of the respondents who tried to follow Wikipedia policies and who raised concerns on article's talk page rather than directly editing them, 10% percents noted that they had to wait weeks to get any response, and 13% said they never received a response.
With regards to the new editors experience, it is also interesting to note that only a quarter of PR professionals felt that making edits was easy, majority complained that editing Wikipedia is time consuming or even "nearly impossible".
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:49, 25 November 2013 (UTC)
The study, based on Wikia wikis, rather than WMF projects (but with significant implication for them as well) found support for the claims that the iron law of oligarchy holds in wikis; i.e. that the wiki transparent and egalitarian models does not prevent the most active contributors from obtaining significant and disproportionate control over those projects. In particular, the study found that as wiki communities grow 1) theys are less likely to add new administrators; 2) the number of edits made by administrators to administrative “project” pages will increase and 3) the number of edits made by experienced contributors that are reverted by administrators also grows. The authors also note that while there are some interesting exceptions to this rule, proving that wikis can, on occasion, function and egalitarian, democratic public spaces, on average "as wikis become larger and more complex, a small group – present at the beginning – will restrict entry into positions of formal authority in the community and account for more administrative activity while using their authority to restrict contributions from experienced community members".
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:05, 25 November 2013 (UTC)
Tilman, nice coverage in the Signpost. Tony (talk) 13:00, 6 December 2013 (UTC)
No idea if regular editors can edit the Signpost but shouldn't "cought" be "caught"? -- NeilN talk to me 05:34, 28 December 2013 (UTC)
My paper on teaching with Wikipedia and academia's attitude to that idea was just published at [64]. I obviously cannot review it myself... :) Cheers, -- Piotr Konieczny aka Prokonsul Piotrus| reply here 15:47, 19 February 2014 (UTC)
Overall, the authors find that the overall quality of the discussions is high, as most of the participants display knowlededge of Wikipedia's policy, particularly on the notability and credibility (or what we would more likely refer to as reliability) of the articles whose deletion is considered. In re, notability far outweighs the second most frequent rationale, credibility (reliability). They confirm that the deletion system works as intended, with decisions made by majority voters.
Interestingly, the authors find that Wcertain topics did tend to trigger more deletion outcomes, said topics being articles about people, for-profit organizations, and definitions. The authors in turn observe that "locations or events are more likely to be kept than expected, and articles about nonprofit organizations and media are more likely to be suggested for other options (e.g., merge, redirect, etc.) than expected". Discussions about people and for-profit organizations were more likely to be unanimous than expected, whereas articles about nonprofit organizations, certain locations, or events were more likely to lead to a nonunanimous discussion. Regarding the SOPA protests influence on deletion debates, the authors find a small and short-lived increase in keep decisions, and tentatively attribute this to editors being impacted by the idea of internet freedom and consequently allowing free Internet publishing.
The authors sum up those observation noting that "the community members of Wikipedia have clear standards for judging the acceptability of a biography or commercial organization article; and such standards are missing or less clear when it comes to the topics on location, event, or nonprofit organization... Thus, one suggestion to the Wikipedia community is to make the criteria of judging these topics more clear or specific with examples, so it will alleviate the ambiguity of the situation.". This reviewer, as a participant of a not insignificant number of deletion discussions as well as those about the associated policies, agrees with said statement. With regards to the wider scheme, the authors conclude that the AfD process is an example of "a democratic deliberation process interested in maintaining information quality in Wikipedia".
Note to TB: it would be nice to link Wikipedia:Arguments to avoid in deletion discussions somewhere, perhaps? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:13, 26 February 2014 (UTC)
The author reviews the linguistic conceptualisation of feminity on (English) Wikipedia, with regards to whether language used to refer to women differs depending on the type of articles it is used in. Specifically, the author analyzed the use of five lexemes (a term which in the context of this study means words): ladylike, girly, girlish, feminine and womanly. The findings confirm that the usage of those terms in non-accidental. The word feminine, most commonly used of of the studied five, correlates primarily to the topics of fashion, sexuality, and to a lesser extent, culture, society and female historical biographies. The second most popular is the word womanly, which in turn correlates with topics of female artists, religion and history. Girlish, the fourth most popular world, correlates most strongly with the biographies of males, as well as with the articles on movies & TV, female entertainers, literature and music. Finally, girly and ladylike, respectively 3rd and 5th in terms of popularity, cluster together and correlate to topics such as movies & TV (animated), Japanese culture, art, tobacco and female athletes. Later, the author also suggests that there is a not insignificant overlap in usage between cluster for girlish and the combined cluster for giry and ladylike. The author concludes that there are three or four different conceptualisations of feminity on Wikipedia, which in more simple terms means, to quote the author, that "people do indeed represent women in different ways when talking about different things [on Wikipedia]", with "girly and girlish having a somewhat frivolous undertone and womanly, feminine and ladylike being of a more serious and reserved nature".
The study does suffer from a few issues: a literature review could be more comprehensive (the paper cites only six works, and not a single one of them from the field of Wikipedia studies), and this reviewer did not find sufficient justification for why the author limited himself to the analysis of only 500 occurrences (total) of the five lexemes analyzed. A futher discussion of how the said 500 cases were selected would likely strengthen the paper.
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 09:17, 27 February 2014 (UTC)
Ed [talk] [majestic titan] 01:11, 24 March 2014 (UTC)
This study examined academics’ awareness of and attitudes towards Wikipedia and open-access journals for academic publishing through a survey of 120 academics carried out in late 2011 and early 2012. This study comes from the same authors who published a similar paper in 2012, reviewed here, which suffered from the a major basic fallacy: Wikipedia is not the place to publish original research academic work. The authors, unfortunately, seem to seem to ignore no original research policy when they write: "There are in general three models in the current movement towards open-access academic publishing: pushing traditional journals towards open access by changing policies; creating open-access journals; and using existing online open-access venue Wikipedia" and "we surveyed academics to understand their perspectives on using Wikipedia for academic publishing in comparison with open-access journals". In the final discussion segment, the authors do acknowledge the existence of the OR policy, where they suggest that certain types or academic papers (reviews) are similar enough to Wikipedia articles that integration of such articles into Wikipedia could be feasible. The authors do provide a valuable literature review noting prior works which analyze the peer review system in Wikipedia, perceptions of Wikipedia in academia, and related issues (through said review is partially split between the introduction and discussion section).
The study provide some interesting findings regarding academics view of benefits of Wikipedia-style peer review and publishing. Most respondents (77 percent) reported reading Wikipedia, and a rather high number (43 percent) reported having made at least one edit, with 15 percent having written an article. Interestingly, as many as four respondents stated that they were "credited for time spent reviewing Wikipedia articles related to their academic careers" in their professional workplaces. The more experience one had with Wikipedia, the more likely one would see advantages in the wiki publishing model. Most common advantages listed were "cost reductions (40 percent), timely review (19 percent), post-publication corrections (52 percent), making articles available before validation (27 percent) and reaching a wider audience (8 percent)." Disadvantages included questionable stability (86 percent), absence of integration with libraries and scholarly search engines (55 percent), lower quality (43 percent), less credibility (57 percent), less academic acceptance (78 percent) and less impact on academia (56 percent).
With regards to Wikipedia peer review, 54 percent of respondents were aware that Wikipedia had a peer-review process and about third of these considered it to be less rigorous than that of scholarly journals; none of the respondents demonstrated any significant experience with the specifics of how Wikipedia articles are reviewed, suggesting that their involvement with the Wikipedia is rather limited. 75% of the survey respondents did not feel comfortable having others edit their papers-in-progress, and over 25% expressed concern about the lack of control over changes made post-publications. Majority of respondents did not also feel comfortable with their work being reviewed by Wikipedians, with the most common concern being unknown qualifications of Wikipedia editors and reviewers.
Perhaps of most value to the Wikipedia community is the analysis of suggestions made by the respondents with regards to making Wikipedia more accepted at the universities. Here, the most common suggestion was “making the promoted peer-reviewed articles searchable from university libraries” and in general, making it more easy to find and identify high quality articles (some functionality as displaying the quality assessment of an article in mainspace already exists in MediaWiki but is implemented as opt-in feature only).
The authors conclude that the academic researchers’ increased familiarity with either open access publishing or wiki publishing is associated with increased comfort with these models; and the academic researchers’ attitudes towards these models are associated with their familiarity, academic environment and professional status. Overall, this study seems like a major improvement over the authors prior 2012 paper, and a valuable paper addressing the topics of the place of Wikipedia in the open publishing movement and the relationship between Wikipedia and academia.
Done with this one, feel free to copyedit. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:36, 29 April 2014 (UTC)
Brief review of [69]
The authors find that readers of English and Chinese Wikipedia from time zones of high Chinese activity browse different categories of pages. Chinese readers visit English Wikipedia about Asian culture (in particular, Japanese and Korean pop culture), as well as about mobile communications and networking technologies. The authors also find that English pages are almost ten times as popular as Chinese pages (through their results are not identifying users by nationality directly, rather focusing on time zone analysis).
In this reviewers opinion, the study suffers from major methodological problems that are serious enough to cast all the findings in doubt. The authors find that only 7603 pages were eligible to be analyzed (had both English and Chinese version), however the Chinese Wikipedia in the studied period had approximately half a million articles; and while many don't have English equivalents yet, to expect that less than 2% did seems rather dubious. Similarly, our own WikiProject China estimates that English Wikipedia has at almost 50,000 China-related articles. That, given that WikiProject assessments are often underestimating the number of relevant topics, and usually don't cover many core topics, suggests that the study missed a vast majority of articles that exist in both languages. It is further unclear how "an English speaking or Chinese speaking time-zone" were operationalized. The authors do not reveal how, if at all, they controlled for the fact that readers of English Wikipedia can also come from countries were English is not a native language, and that there are hundreds of millions of people outside China who live in the five time zones that span China, which overlap with India, half of Russia, Korea and major parts of Southeast Asia. As such, the findings of that study can be more broadly interpreted as "readership patterns of English and Chinese Wikipedia in Asia and the the world, with regards to a small subset of pages that exist both on English and Chinese Wikipedia."
--not done, but not sure if I can continue today... the article is not written very clearly, I am still surprised what get published. Can you figure out if this article studied people from non-Chinese time zones and how did the authors try to control for people such as those from other East Asian countries? --
Piotr Konieczny aka Prokonsul Piotrus|
reply here 05:59, 1 May 2014 (UTC)
A survey of research skills of a group of students at an Australian institution showed that purposeful engaging with Wikipedia, including contributing to it, improved their academic skillset.
[short entry, but the topic is pretty straightforward]
This article describes IntelWiki, a set of MediaWiki tools designed to facilitate new editor's engagement by making research easier. The tool "automatically generates resource recommendations, ranks the references based on the occurrence of salient keywords, and allows users to interact with the recommended references within the Wikipedia editor." The researchers find that volunteers using this tool were more productive, contributing more high quality text. The studied group was composed of sixteen editors with no prior Wikipedia editing experience, who completed two editing tasks in a sandbox wiki, one using a mockup Wikipedia editing interface and Google search engine, and using the IntelWiki interface and reference search engine. The reference suggestion tool developed by the author seems valuable, unfortunately this reviewer was unable to locate any proof that the developer engaged Wikipedia community, or made his code or the tool publicly available for further testing. Further, the research and the thesis does not discuss the differences between their MediaWiki clone and Wikipedia in any significant details. Based on the limited description available, the study's overall conclusions may not be very reliable, as the mockup Wikipedia interface used for comparison seems to be a default MediaWiki clone, lacking many Wikipedia-specific tools; therefore the theme of comparing IntelWiki to Wikipedia is a bit misleading. Overall, while the study is interesting, it is disappointing that the main purpose of this appears to be completing a thesis, with little thought given to actually improving Wikipedia (by developing public tools and/or releasing open code).
Till: please check Wikipedia:Village_pump_(miscellaneous)#Was_.22Intel_wiki.22_tool_discussed_by_the_community_before.3F for any developments, I may have limited net access next week. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:13, 22 June 2014 (UTC)
This paper explores the use of Chinese Wikipedia and Baidu Baike encyclopedia by Chinese microblog ( Twitter, Sina Weibo) users through qualitative and quantitative analyses of Chinese microblog postings. Both encyclopedias are often cited by microblog users, and are highly popular in China to the extent that the words wiki band baidu became verbs meaning to look up content on respective websites, serving as the verb "to google" in contemporary English. One of the study's major focus is the impact of Internet censorship in China; particularly as Wikipedia is not censored - but access to it, and it's discussion in most Chinese websites may be, and in turn Baidu Baike is both censored and more likely to host copyright violating content (thus making it more informative to the vast majority of Internet users not caring about legal technicalities). Despite Baidu Baike copyright violating content, many users still prefer uncensored and more reliable Chinese Wikipedia, through can get frustrated by not being able to access it due to censorship. Whether some Wikipedia content is censored or not is seen by some as a measure of the topic's political sensitivity. The author also suggests that there certain distinguishing characteristic can be observed between groups which prefer one encyclopedia over the other, but does not discuss this in detail, suggesting a very interesting further research avenue. -- Piotr Konieczny aka Prokonsul Piotrus| reply here 11:46, 22 June 2014 (UTC)
[saving draft]
There are few more works I am willing to review but I don't have access to them. If you can get me copies of them, I can work on them.
Oh, feel free to add "review by Piotrus" or such to each and any of my reviews from now on, I support non-anonymous reviewing.
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 06:07, 29 July 2014 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 09:19, 26 August 2014 (UTC)
The authors find that the AfC review process is not subject to insurmountable delays; they conclude that "most drafts will be submitted for review quickly and that reviews will happen in a timely manner.". In fact, 2/3 or reviews take place within a day of submission (a figure that positively surprised this reviewer, through a current AfC status report suggests a situation has worsened since: "Severe backlog: 2599 pending submissions"). In either case, the authors find that about a third of so of newcomers using AfC system fail to understand the fact that they need to finalize the process by submitting their drafts to the review at all - a likely indication that the AfC instructions need revising, and that the AfC regulars may want to implement a system of identifying stalled drafts, which in some cases may be ready for mainspace despite having never been officially "submitted" (due to their newbie creator not knowing about this step or carrying it properly). The authors do however stand by their second hypothesis, i.e. they conclude that the AfC articles suffer from not receiving collaborative help they would if they were mainspaced. They discuss the a specific case of an article ( Wikipedia_talk:Articles_for_creation/Dwight_K._Shellman,_Jr/ Wikipedia_talk:Articles_for_creation/Dwight_Shellman). This article has been tagged potentially rescuable ( Wikipedia:WikiProject_Articles_for_creation/G13_rescue) and has been languishing in that state for years, hidden in the AfC namespace, together with many other similarly backlogged articles ( Category:AfC pending submissions by age/Very old), all stuck in a low visibility limbo, prevented from receiving proper Wikipedia-style collaboration-driven improvements (or deletion discussions) as an article in the mainspace would receive.
The authors also identify a number of other reasons that reduce the functionality of the AfC process. As in many other aspects of Wikipedia, negative feedback dominates. Reviewers are rarely thanked for anything, but are more likely to be criticized for passing an article deemed problematic by another editor; thus leading to the mentality that "rejecting articles is safest" (as newbies are less likely to complain about their article's rejection than experienced editors about passing one). AfC also suffers from the same "one reviewer" problem as GA - the reviewer may not always be qualified to carry the review, yet the newbies have little knowledge how to ask for a second opinion. The authors specifically discuss a case of reviewers not familiar with the specific notability criteria: "[despite being notable] an article about an Emmy-award winning TV show from the 1980's was twice declined at AfC, before finally being published 15 months after the draft was started". Presumably if this article was not submitted to a review it would never be deleted from the mainspace. The authors are also critical of the interface of the AfC process, concluding that it is too unfriendly to newbies, instruction wise: "Newcomers do not understand the review process, including how to submit articles for review and the expected timeframe for reviews" and "Newcomers cannot always find the articles they created. They may recreate drafts, so that the same con-tent is created and reviewed multiple times. This is worsened by having multiple article creation spaces(Main, userspace, Wikipediatalk, and the recently-created Draft namespace".
The authors conclude that the AfC works well as a filtering process for the encyclopedia, however "for helping and training newcomers [it] seems inadequate". The AfC succeeds in protecting content under (recently established) G13, in theory allowing newbies to keep fixing it - but many do not take this opportunity. Neither is the community able to deal with this, and thus the authors call for a creation of "a mechanism for editors to find interesting drafts". That said, this reviewers wants to point out that the G13 backlog, while quite interesting (thousands of articles almost ready for main space...), is not the only backlog Wikipedia has to deal with - something that the authors overlook. The G13 backlog is likely partially a result of imperfect AfC design that could be improved, but all such backlogs are also an artifact of the lack of active editors affecting Wikipedia projects on many levels. In either case, AfC regulars should carefully examine the authors suggestions. This reviewer in particular finds the following ideas worth pursuing. 1) Determine which drafts need collaboration and make them more visible to potential editors. Here the authors suggest use of a recent academic model that should help automatically identify valuable articles, and then feeding those articles to SuggestBot. 2) Support newcomers’ first contributions - almost a dead horse at this point, but we know we are not doing enough to be friendly to newcomers. In particular, the authors note that we need to create better mechanisms for newcomers to get help on their draft, and to improve the article creation advice, in particular - the Article Wizard. (As a teacher who introduced hundreds of newcomers to Wikipedia, I can attest myself that the current outreach to newbies on those levels is grossly inadequate).
A final comment, to the community in general: was AfC intended to help newcomers, or was it intended from the start to reduce the strain on new page patrollers by sandboxing the drafts in the first place? One of the roles of AfC is to prevent problematic articles from appearing in the mainspace, and it does seem that in this role it is succeeding quite well. English Wikipedia community has rejected the Wikipedia:Flagged revisions-like tool, but allowed implementation of it on a voluntary basis for newcomers, who in turn may not often realize that by choosing the AfC process, friendly on a surface, they are in fact slow-tracking themselves, and inviting extra-ordinary scrutiny. This leads to a larger question that is worth considering: we, the Wikipedia community of active editors, have declined to have our edits classified as second tier and hidden from public until reviewed, but we are fine pushing this on to the newbies. To what degree this is contributing to the general trend of Wikipedia being less and less friendly to newcomers? Is the resulting quality control worth turning away potential newbies? Would we be here if years ago our first experience with Wikipedia was through AfC?
I was sollicited to provide some feedback on this thread. Tbayer (WMF): I observe that your chronology of events (Siegenthaler, Jimbo's Fiat, the evolution of AFC, the request by members of enWP (including AFC) to have the Draft namespace) is so far out of sync with reality that I can only assume you are fabricating a desired timeline to support your viewpoint. Early on in the "research" that this was based on false premises... The consideration of how many submissions come in daily versus the amount of pages that are promoted is drastically imballanced. The consideration of pages that start out as a ward of AFC that survive 30 days is not a accurate judgement either as some pages may lie unviewed for months only to be deleted. Furthermore your argument with Dodger67 about who came up with the idea for the draft namespace (and demands to disprove his view) shows either your willful ignorance or deliberate incompetence as it took only 2 pages to get to the original RFC where the idea was proposed and accepted including the WikiMedia Bugzilla ticket ID ( WP:DRAFTS which describes the namespace which links to Wikipedia:Village pump (proposals)/Archive 107#Proposed new Draft namespace which is the RFC on Village pump proposals and also includes the Bugzilla ticket ID). In the spirit of open and even handedness that the foundation (and it's employees) are supposed to be espousing, I strongly suggest you apologize to Rodger. CCing Mdennis (WMF) as the community advocate to take this suboptimal interaction back to her supervisors for further action. Hasteur ( talk) 15:26, 19 September 2014 (UTC)
Hi, it was delayed by a week this time: no matter, but should we presume the next RR will be on 24 September or 1 October? Tony (talk) 01:19, 7 September 2014 (UTC)
-- Piotr Konieczny aka Prokonsul Piotrus| reply here 12:31, 23 September 2014 (UTC)
Update: either etherpad is down or my 'net is borked and I cannot access it anymore. Will try again tomorrow :(
This study analyzed the development of the referencing of 45 articles over nine topic groups related to health and nutrition over a period of five years (2007-2011) (unfortunately, the authors are not very clear on which particular articles were analyzed, and tend to use the concepts of an article and topic group in a rather confusing manner). Authors coded for references (3,029 total), information on editing history, and search ranking in Google, Bing and Yahoo! search engines. The study confirmed that Wikipedia articles are highly ranked by all search engines, with Yahoo! actually being even more "Wikipedia-friendly" than Google. The author shows that (as expected) the articles improve in quality (or at least, number and density of references) over time. Crucially, the authors show that the overall percentage of mainstream news media references has decreased, while references to academic publications increased over that time. By the end of the study period, only article on (or topic group of?) trans fat contained more references to news sources than to academic publications. The authors overall support the description of Wikipedia as a source aiming for reliablity, through are hesitant to call it reliable, pointing out that for example 15% of analyzed references were coded as "outside the main reference type categories or... not be clearly determined". The authors conclude, commendably, that "Wikipedia needs to be high on the agenda for health communication researchers and practitioners." and that "communications professionals in the health field need to be much more actively involved in ensuring that the content on Wikipedia is reliable and well sourced with reliable references". -- Piotr Konieczny aka Prokonsul Piotrus| reply here 07:08, 25 November 2014 (UTC)
This article contributes to the discussion on gender inequalities on Wikipedia. The authors take a novel approach of looking for answers outside the Wikipedia community, thus also tying their reseearch into the analysis of new editors recruitment, motivations, and barriers to contribute. The authors focus their analysis on the role of Internet experiences and skills, and their lack among certain groups. The authors study whether the level of one's skills in digital literacy is related to their chance of becoming a Wikipedia editor, by surveying 547 young adults (aged 21-22) - students at a (presumably American) university, the most convenience sample in academia. The survey was carried in 2009, with a follow-up wave in 2012. The students were asked about their socioeconomic and demographic background, as well as about their level of digital literacy skills. The authors report that "the average respondent's confidence in editing Wikipedia is relatively low" but that "about one in eight students had been given an assignment in class at some point either to edit or create a new entry on Wikipedia" - which likely suggests that the (undisclosed by authors) university was one where at least one member of the faculty participated in the Wikipedia:Education Program. The vast majority (99%) of respondents reported having read an entry on Wikipedia, and over a quarter (28%) have had some experience editing it (interestingly, even when controlling for students who were assigned to edit Wikipedia, the former number still is as high as 20%).
With regards to the gender gap issues, women are much less likely to have contributed to Wikipedia than men (21% to 38%), and that becomes even more divergent when controlling for student assignments (13% to 32%). The authors find indication of gender gap affecting the likelyhood of Wikipedia's contributions: students who are white, economically affluent, male and Internet-experienced are more likely to edit than others. The strongest and statistically significant predictor variables, however, are Internet skills and gender, and regression models show that variables such as race, ethnicity, socioeconomic status, availability, Internet experience, and confidence in editing Wikipedia are not significant. The authors find that the gender becomes more significant as one's digital literacy increases. At low level of Internet skills, the likelyhood of one's contribution to Wikipedia is low, regardless of gender. As one's skills increase, males became much more likely to contribute, but women fall behind. The authors find that women tend to have lower Internet skills than men, which helps explain a part of the Wikipedia gender gap: to contribute to Wikipedia, one needs to have a certain level of digital literacy, and the digital gap is reducing the number of women who have required level of skills. The authors crucially admit that "why women, on average, report lower level understanding of Internet-related terms remains a puzzle. Studies with detailed data about actual skills based on performance tests suggest no gender differences in the observed skills research that looks at self-rated know-how consistently finds gender variation with real consequences for online behavior". This suggests that while men and women have, in reality, similar skills, women are much less confident about them, which in turns makes them much less confident about contributing to (or trying to conribute to) Wikipedia. This, however, is a hypothesis to be confirmed by future research. In the end, the authors do feel confident enough to conclude that "gender and Internet skills likely have a relatively mild interaction with each other, reinforcing the gender gap at the high end of the Internet skills spectrum." In the end, this reviwer finds this study to be a highly valuable one, both for the literature on gender gap and online communities, and for the Wikipedia community and WMF efforts to reduce this gap in our environment.
This book chapter looks at the Wikimedia community as a social movement. The book chapter is interesting as in clearly placing itself in the relatively small body of literature that describe Wikipedia/Wikimedia as a social movement, unfortunately it is primarily a descriptive, rather than an analytical piece, and does not provide any significant theoretical justification for calling the Wikimedia movement a social movement, a weakness amplified by the fact that this work fails to engage with prior relevant body of Wikipedia research, and is only very loosely connected to the literature on social movements.
I think that's all I have time for today, -- Piotr Konieczny aka Prokonsul Piotrus| reply here 08:01, 25 November 2014 (UTC)
This article proposes a way for combining Wikipedia and Online Books Page data, for the purpose of identifying most notable (important, popular, read) authors whose work is about to enter the public domain, in order to facilitate and prioritize digitization of their works. The proposed algorithm may be of interest to members of Wikipedia:WikiProject Books, Wikipedia:WikiProject Libraries, Wikipedia:WikiProject Open, and related ones, as a means generating a Wikipedia:Assessment importance rating and selecting underdeveloped articles for development.
TB, I think we should have one of our librarians develop this further. Ping User:Phoebe? -- Piotr Konieczny aka Prokonsul Piotrus| reply here 10:16, 26 November 2014 (UTC)