A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.
This paper [1] is a good example of how to write articles for the "teaching with Wikipedia" field. The authors report their positive experiences with several under- and postgraduate classes at the University of Sydney, developing articles such as pregnancy vegetarianism, Cleo (magazine) or Slave Labour (mural). They describe in relative detail a number of assignments and assessment criteria, and discuss benefits that their Wikipedia assignments have for the community (improving valuable and underrepresented content) and for the students themselves (improving their writing, research and collaborative skills). The paper could benefit from a more comprehensive literature review, however: while it describes a useful set of educational activities, and rather well at that, these are not groundbreaking—practically all activities discussed in this paper have been discussed in peer reviewed literature by others. Unfortunately, the authors fail to cite many of related works (I count only about five citations to the other peer-reviewed works from the much larger field of teaching with Wikipedia). Furthermore, the authors seem unaware of the Wikipedia:Education Program. It does not appear that any of their courses so far have been registered on Wikipedia; sadly they have no on-wiki homepage allowing identification of all edited articles or participating students; it is also unclear if the instructors themselves have a Wikipedia account. This suggests a failing both on the part of the researchers (they spent years reading about, researching and engaging with the teaching with Wikipedia approach without realizing there is a major support infrastructure in place to assist them), as well as on the part of the Wikipedia community and the Education Program itself, which is clearly still not being visible enough, nor active enough to identify and reach out to such educators who have been engaged in several years of ongoing teaching on Wikipedia. Hopefully in the future we can integrate those and other educators into our framework better.
Researchers from the University of Regensburg in Germany have used eyetracking methods to find out which article elements readers focus on while searching for information on Wikipedia, depending on the nature of the search task (factual information lookup, learning, or casual reading—a classification taken from a 2006 article [supp 1] about exploratory search in general).
In two 2012 articles [2] [3] the researchers summarized the methodology and results of one of their lab experiments with 28 participants, which besides eyetracking also incorporated data from survey questionnaires, browser logs and electromyography for two facial muscles that indicate emotional reactions (the corrugator and the zygomaticus major). Among the results of this first study (see also a related paper in English with illustrations explaining the various article elements [4]):
A subsequent German-language PhD thesis [5] (see also 2012 conference poster) contains much more detail, e.g. reporting that in "lookup" tasks, readers spend >45% of their time on scanning the table of content and lists in the article, in "learn" tasks these only amount to <10% of the time.
A second PhD thesis, covered in a brief paper [6] last year, examined for example which elements readers look at first within an article (from an experiment involving 163 German Wikipedia articles and 90 participants who were asked to prepare themselves for an course on the history of Bavaria in the 20th century, i.e. a "learning" overview task): The table of contents was the most frequent entry point (36%) followed by the lead section (31%) and the text body itself. The author observes further that "the article heading and images serve less often as entry point. The text heading [presumably the first section heading after the lead] and image captions very rarely occurs as points of first contact". Another publication [7] by the same author focused on "users' interaction with pictorial and textual contents ...[ The spread] of information within the articles and the relation between text and images are analyzed. ... By now 30 articles have been analyzed according to this scheme. [Within these, there] are 639 contact points leading to images. Results show that 39% of all contact points lead from image to image, in mutual directions (previous or next). All text contact points [e.g. citations] sum up to a total of 37%. In 5% of all cases, an introduction triggers a saccade to an image. The remaining types of contact points occur rather rarely."
A later overview article [8] summarizes other aspects in less detail, e.g:
(For an overview over other new data sources shedding light on how readers navigate within articles, see also this reviewer's recent tech talk at the Wikimedia Foundation, and a research overview page on Meta about the question "Which parts of an article do readers read?)
A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.
A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.
This paper [1] is a good example of how to write articles for the "teaching with Wikipedia" field. The authors report their positive experiences with several under- and postgraduate classes at the University of Sydney, developing articles such as pregnancy vegetarianism, Cleo (magazine) or Slave Labour (mural). They describe in relative detail a number of assignments and assessment criteria, and discuss benefits that their Wikipedia assignments have for the community (improving valuable and underrepresented content) and for the students themselves (improving their writing, research and collaborative skills). The paper could benefit from a more comprehensive literature review, however: while it describes a useful set of educational activities, and rather well at that, these are not groundbreaking—practically all activities discussed in this paper have been discussed in peer reviewed literature by others. Unfortunately, the authors fail to cite many of related works (I count only about five citations to the other peer-reviewed works from the much larger field of teaching with Wikipedia). Furthermore, the authors seem unaware of the Wikipedia:Education Program. It does not appear that any of their courses so far have been registered on Wikipedia; sadly they have no on-wiki homepage allowing identification of all edited articles or participating students; it is also unclear if the instructors themselves have a Wikipedia account. This suggests a failing both on the part of the researchers (they spent years reading about, researching and engaging with the teaching with Wikipedia approach without realizing there is a major support infrastructure in place to assist them), as well as on the part of the Wikipedia community and the Education Program itself, which is clearly still not being visible enough, nor active enough to identify and reach out to such educators who have been engaged in several years of ongoing teaching on Wikipedia. Hopefully in the future we can integrate those and other educators into our framework better.
Researchers from the University of Regensburg in Germany have used eyetracking methods to find out which article elements readers focus on while searching for information on Wikipedia, depending on the nature of the search task (factual information lookup, learning, or casual reading—a classification taken from a 2006 article [supp 1] about exploratory search in general).
In two 2012 articles [2] [3] the researchers summarized the methodology and results of one of their lab experiments with 28 participants, which besides eyetracking also incorporated data from survey questionnaires, browser logs and electromyography for two facial muscles that indicate emotional reactions (the corrugator and the zygomaticus major). Among the results of this first study (see also a related paper in English with illustrations explaining the various article elements [4]):
A subsequent German-language PhD thesis [5] (see also 2012 conference poster) contains much more detail, e.g. reporting that in "lookup" tasks, readers spend >45% of their time on scanning the table of content and lists in the article, in "learn" tasks these only amount to <10% of the time.
A second PhD thesis, covered in a brief paper [6] last year, examined for example which elements readers look at first within an article (from an experiment involving 163 German Wikipedia articles and 90 participants who were asked to prepare themselves for an course on the history of Bavaria in the 20th century, i.e. a "learning" overview task): The table of contents was the most frequent entry point (36%) followed by the lead section (31%) and the text body itself. The author observes further that "the article heading and images serve less often as entry point. The text heading [presumably the first section heading after the lead] and image captions very rarely occurs as points of first contact". Another publication [7] by the same author focused on "users' interaction with pictorial and textual contents ...[ The spread] of information within the articles and the relation between text and images are analyzed. ... By now 30 articles have been analyzed according to this scheme. [Within these, there] are 639 contact points leading to images. Results show that 39% of all contact points lead from image to image, in mutual directions (previous or next). All text contact points [e.g. citations] sum up to a total of 37%. In 5% of all cases, an introduction triggers a saccade to an image. The remaining types of contact points occur rather rarely."
A later overview article [8] summarizes other aspects in less detail, e.g:
(For an overview over other new data sources shedding light on how readers navigate within articles, see also this reviewer's recent tech talk at the Wikimedia Foundation, and a research overview page on Meta about the question "Which parts of an article do readers read?)
A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.
Discuss this story
I also had two of my own academic articles on Wikipedia published in March ( [3], [4]), but for obvious COI reasons I am not reviewing them. If anyone enjoying this newsletter would, however, like to return the favor and review my works (feel free to be critical), I'd appreciate it :) -- Piotr Konieczny aka Prokonsul Piotrus| reply here 04:29, 4 April 2016 (UTC) reply
I hope that the following responses address some of your concerns, which centre on the inclusion of only five works from the larger corpus of “teaching with Wikipedia” literature, the fact that the paper does not report on any “groundbreaking” activities, and that the authors don’t seem to have connections with major support structures, don’t seem to have a Wikipedia account, and so forth.
As our purpose was to promote the efficacy of using Wikipedia to assess students’ performance, we wrote our article for scholars interested in higher education assessment pedagogy and so our focus was quite narrow. The article passed peer review and was accepted by a respected higher education journal that has only every published one article on the use of Wikipedia in Education: ours. A keyword search shows that of the 6 others, 3 mentioned Wikipedia twice, 3 mentioned it 4 times and only one of those did not reinforce common negative perceptions about Wikipedia.
Perhaps not groundbreaking, it was nonetheless a breakthrough to have such a high-ranking journal in the field accept our article for publication, particularly as we were advocating something that is not appealing to many conservative academics. In order to be accepted by such a high ranking journal, it was essential for us to show the ways our practices hinge on the theories of scholars renown in the field of higher education assessment pedagogy. We were also limited by a word count that included the bibliography and therefore could not afford to expand our literature review to cover scholarship in the broader field that sits outside our area of focus.
Lastly, I have been a Wikipedia editor since 2012, and listed my courses on the Wikipedia:Education noticeboard [5] in May 2013, and in September and October 2013 (See for example [6]). I am listed as one of the University of Sydney Contacts on the Wikipedia Education Program’s page for Australia [7] and have worked on initiatives with Wikimedia AU, and received support from Wikipedia volunteers and Wikipedians at my institution. My coauthor, Rebecca Johinke, developed an interest in teaching with Wikipedia in 2013 and has used it in teaching since 2014. To be fair to the Wikimedia Foundation, their support has been invaluable, as has the support of our local chapter, Wikimedia AU. Frances Di Lauro 08:54, 4 April 2016 (UTC)