This page is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page. |
The past six days of brainstorming here (yes, this discussion has been going on for almost a week) have focused on "da board". I think it would be nice to discuss a little about the actual reviews for a change. My proposal: we pick a guinea pig article and try to carry out a review process in vitro. Karol 22:15, 15 March 2006 (UTC)
Alright: Science.-- HereToHelp ( talk • contribs) 23:30, 15 March 2006 (UTC)
OK, well, there's also a glaring error pertaining to physics in the subsection titled "philosophy of science". It’s exactly the kind of error a college kid might make; it implies that atoms are "mostly empty space"; they most certainly are not. Now what? I can fix this error, but so what? How does the board help here? Or let me rephrase: bduke says "It hardly contains any science itself, so it does not need experts to review it." -- yet it contains an error that non-physicists will almost certainly overlook. ... which is part of the problem with WP articles. linas 00:09, 16 March 2006 (UTC)
If you want to wet your teeth on something more technical, but not generally controversial, how about Structure of the Earth. It is a substantially technical article with only a small number of references, mostly to other websites, and few interesting problems like the percent compositions not equally 100%. Dragons flight 01:47, 16 March 2006 (UTC)
I was wondering about a trial of supergravity, not a trial of a review but more a trial of whether we could find a reviewer. Looking at the article raised some new issues. JarahE, who has nominated for the Board, could, from his nomination, do the review. However he has edited the article considerably in the last three days. Charles Mathews had an edit on it earlier, although it is only a modification to a category. Linas, who raised this article, has extensively edited it. Surely it would not do to have, as a reviewer, someone who had helped to write the article. So it might be difficult to find someone. This raises another issue. It might be best if the nomination of an article was only put on the Project page first, leaving the tag on the article's talk page to be added when we know we can review it. Maybe it does not matter. I do not know.
I think the process would be this. The Board or perhaps one person (this role might rotate) concludes that the expertise to review this article is not available among the Board members and reviewers list. S/he puts on this talk page:-
If nobody responds after a couple of days, they add:-
There might be a gap of a day or so between the two points above being added. If this gets a nominee to review the article, the Board consults and if they find the reviewer is expert enough, they ask them to do the review. Later the Board looks at the review and helps to put together a suitable report.
I think this is where the Board is useful. There is someone with a responsibility to try to find a reviewer. Without this, it is much less likely that someone will try to find one. -- Bduke 21:48, 16 March 2006 (UTC)
While I understand Samsara's point, could I suggest that we leave comments until there is an actual call to comment and vote? This candidate might withdraw. I think general comment on this talk page is OK, but let's leave the Project page for now. -- Bduke 00:29, 16 March 2006 (UTC)
HereToHelp has just altered the recommended headers for sub pages with this comment - (If we're not templating the subpages but rather linking them (sounds okay to me) we should use bigger headers.). The original proposal did have the sub page "templated" i.e in {{ }} brackets, but this was changed to [[ ]]. I let it go at the time, because I thought we could come back to it later. Now it seems is later. I think we should template in the sub-pages, in which case we should put the headers back to something like they were before. I grant you that this is a small technical point, but we need to address it sometime. Any thoughts? -- Bduke 03:32, 16 March 2006 (UTC)
Just wondering...if SPR stands for "scientific" peer review, should it then be reflexive and apply scientific method (aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.) to itself? Or does SPR have immunity from scientific method, and if so then how is the word "scientific" defined in the SPR acronym? Is there a "science" of SPR? Should SPR demonstrate the property "science" which the peer reviewer lays claims to?
Let's try an example - Samsara's current SPR of the Science article.
Title
A general model of scientific peer review in the open source user-defined environment.
Abstract
Open source user-defined scientific peer review (SPR) is an emergent property of the Wikipedia internet technology. Initial steps are taken to produce a model of SPR with the aim of SPR formalization. Two definitions of 'science' were identified as needing analysis and universally agreed definition. Results are pending.
To define the term "science" in such a way that it will be achieve maximal agreement among the world's population of scientists.
Hypothesis
<Not sure about this...>
Scientific method says
<I'm not sure what the mathematical model here is. Being mathematical it would suggest that my/our results would have to be in some way quantitative. What would be the quantities>
Hypothesis says
<Ok so the scientific community could prove the aim false by disagreeing with the definition? This leads me to think that each definition & article needs a means of a user formally registering and displaying (a counter) ascend to the definition.>
Samsara's SPR of the Science article was analyzed through the Wikipedia collaborative online method. In particular, two definitions of science were analyzed.
Wikipedia portal, computer, internet access.
Analysis
Two definitions of science have been obtained throughout the process of SPR.
"science is a system of knowledge acquisition"
"science is an exercise in model selection"
Results
Pending...
Time: ?!
$: ?!
Time/$ efficiency = ?!
Energy: ?! joules
Exergy: ?!
Embodied energy: ?! embodied joules
EROEI: ?1
Conclusions
Recommendations
Bibliography
Sholto Maud 04:45, 16 March 2006 (UTC)
I was going to nominate Classical theories of gravitation but it currently sits in a category with 100's of members. The category:pages needing expert attention does not currently separate out the Natural science articles. But there is currently only 'Template:expert' and 'Template:Technical (expert)' for the remediation markup. So, what would everyone like to see:
perhaps? That would invite expert attention to the article, before or perhaps in the process of a review. What other templates might come to mind? -- Ancheta Wis 11:30, 16 March 2006 (UTC)
I propose that we create 1 template for each named science (Biology for example), and invoke it with {{subst:Expert|Biology}} for example -- Ancheta Wis 13:29, 16 March 2006 (UTC)
Samsara ( talk • contribs) 13:32, 16 March 2006 (UTC)
OK, here is the usage.
One of these notices, when placed in an article, ought to feed right to the cleanup team who can then route the names of the respective articles to an appropriate Peer review pool.
It is possible, and in my opinion preferable, as it makes for a more pleasant reading experience, to place this notice on the Talk page of the article, which will then appear properly alphabetized in the category listings. -- Ancheta Wis 13:58, 16 March 2006 (UTC)
Splash, I have also created the analogous expertScience expertTechnology and expertHistory templates. Others may wish to create Med. and Law templates, but those would need disclaimers. --14:07, 16 March 2006 (UTC)
It should be possible to coordinate this project with the entries in Wikipedia:Pages needing attention/Mathematical and Natural Sciences
The templates can have an embedded category in them to facilitate searches. Example: category:Biology page needing Scientific peer review -- Ancheta Wis 15:46, 16 March 2006 (UTC)
Would a single template do? Called like:-
We would then not have to keep writing new ones. -- Bduke 21:07, 16 March 2006 (UTC)
I think that the idea of a scientific board is great (one of the best idea so far), but I disagree with one point : it seems to me that creating a board with people of diverse origins is a bad idea. I would prefer several boards specialized on one topic (astronomy, biology, chemistry). I think that the community is large enough so that we can find seven high-level scientist in every field. The problem is see right now is that I don't know how an astronom will be able to make any correction on the content of a biology article (for example). So, the goal of the committee would fail. What do you think about creating several committees ? Poppypetty 13:11, 16 March 2006 (UTC)
Let's not overstate the value of multiple expert input into each article; that's what the contributors should supply. If at least one expert in each area is on the board, that's fine: the rest will all be scientists, and so much of critiquing a scientific text is concerned with scientific method and scientific language, in which all board members should have expertise.
Having said that, I think your idea may be prophetic. If this board works well, perhaps it will evolve to subdivide itself naturally. Let's try to get the process right with this overarching board first, don't you think? Tony 14:11, 16 March 2006 (UTC)
Being an expert in the Wikipedia Manual of Style or in editing articles does NOT make anyone an expert in any scientific fields. I fully realize that my next statement may offend some people, but I would be remiss not to say it. Anyone that is still an undergraduate, a post-graduate or a post-doc in any university is NOT per se a scientific expert no matter how brilliant a student he or she may be. Anyone without at least 10 years of real world experience in a specific field is NOT a scientific expert. mbeychok 23:33, 16 March 2006 (UTC)
If my nomination is going to cause a problem, then I shall withdraw. I agree, being a student does not make me a expert but the idea behind my nomination was to be a 2nd opinion, not an expert but someone who can understand what your talking about. I think its important to have a 2nd party (in all forms of review) but physics being physics is not understandable by all, so someone who can understand physics, but who is not an expert, ie, a physics student would be an idea. Above was by User:Eevo 11:53, 17 March 2006.
As the nominee who meets mbeychok's criteria in spades, it being 42 years since I did my D Phil and I worked in academia until 2002 with an Honorary Reserarch position since then, I want to say that I think his comment is quite silly in the Wikipedia context and I do not appreciate being shouted at (Capital letter, NOT). To Eevo, I say that being a student is not the issue. Your lack of experience on Wikipedia may be. -- Bduke 01:09, 17 March 2006 (UTC)
Lack of experience would not be a problem, under the name Evo, I have about 10 edits, under no name, i have about 800 edits (about 200 in physics/astro).Evo
To Turmosis. Nobody is suggesting that you need 10 years of experience to edit Wikipedia. The question is about expert reviews of articles about science. First year students make great Wikipedians. My experience suggests they would get the science wrong quite often. To Linas. I think we have to cover both basic science and advanced science. If we only do the former, we do not need this project. We just need a mechanism on the various WikiProject Science pages to send people over to WP:PR to look at the science and you can put the articles there that need attention. Of course in one sense the whole discussion is not resolvable. I suspect the WP:SPR will try to deal with any article that gets nominated. That might be a weakness, but whatever guidelines we agree on now are likely to be ignored. -- Bduke 01:27, 17 March 2006 (UTC)
I just realized that there really isn’t a point to this discussion. All the people are still in the nomination stage and I am sure the community will make the right decision for who to vote for. I don’t think mbeychok has anything to worry about since I am sure everyone understands that only people with scientific knowledge are suitable for this. Tutmosis 01:46, 17 March 2006 (UTC)
PLEASE let's not invest too much in this election. It's the process of conducting an experiment in a new structure on WP for improving articles that is much more important than any individual. Whoever misses out this time can stand again, and who knows, by then, there might be more 'boards' with narrower specialisations.
I can see that nominees who miss out this time may drop the bucket and be inclined not to participate at all (that would be my immediate impulse if I were standing and lost). For this reason, I wonder whether people might consider that all nominees who miss out be automatically offered the position of 'Special science contributor', or an equivalent. They represent talent that we cannot afford to lose. Tony 02:41, 17 March 2006 (UTC)
On second thought, perhaps not 'all', which might encourage nominations solely for that purpose; but the new board might give serious consideration to motivating those who didn't succeed. Tony 02:46, 17 March 2006 (UTC)
I think what makes someone good for reviewing articles is:
I think the credentials matter-- but they only go so far. [1] Everyone that has spent some time reading peer-reviewed scientific literature has come across papers that have mistakes and others that are completely wrong.
What really matters, in my opinion, is the above three points and a track record that demonstrates that. You can have people that do things for a hobby and are really darn good at 'em. Einstein did a lot of his best work when he was working at the patent office. Srinivasa Ramanujan basically had no credentials whatsoever... but made great contributions to his area.
While I agree that a lot of crap comes from people with a bit of knowledge and a big ego, we should think about how we can design a process that lets the talented contribute, revise and review. The key, I think, is not selecting the people with the best credentials-- it is selecting the best people. Nephron 23:40, 18 March 2006 (UTC)
A wiki depends on a set of close-knit relationships (meaning concord among its members) in support of some project, such as an article, or an intellectual achievement. A scientific community is knit from similar interests. A group of scientific peers, by definition, is a group of equals (in some sense). When a peer review occurs, the participants coöperate (as defined among themselves). The coöperation is the project.
But principled actions can be respected.
It is not cause for celebration when we lose a contributing/producing member of any community. I would like to voice my regret at the circumstances which have prompted Samsara's resignation from Wikipedia and furthermore thank him for his contributions in our behalf. -- Ancheta Wis 09:35, 17 March 2006 (UTC)
Indeed, it is not cause for any kind of celebration. I have no idea however why Samsara has done this. We had a reasonable exchange of views on our talk pages not too long before he left. He expressed a bit of frustration but nothing more. I hope someone can get him back. -- Bduke 11:14, 17 March 2006 (UTC)
This kind of situation is why I (and also people above) suggested a formal 'expert/authority' weighting system. It is also why I have tried to talk about the model and science of SPR, else we seem to leave the scientific community open to internal emotional turmoil, which is not scientific... or is it? I think it is worth repeating my question: Does SPR need to be scientific in the sense that it should follow scientific method? (or am I talking to myself here? :) ) Sholto Maud 12:45, 17 March 2006 (UTC)
I think using the scientific method here is overly bureaucratic. I have no idea why Samsara left, and I too regret the loss. If, at the very minimum, we knew why...-- HereToHelp ( talk • contribs) 12:48, 17 March 2006 (UTC)
I believe the discussion and the experiment has identified several issues that are at this moment unresolved. I'll try to recap here. I'd like discussion to work around getting consensus on these.
What type of article should this project focus on? Generalist and overview articles, such as Science, etc. or narrow, specialist topics? Almost all of the rest of the debate on this issue hinges on the answer to this question, including what the structure and organization will be, who the participants will be, what their qualifications might be, etc.
Generalist articles:
Specialist articles:
Set up two very distinct systems to deal with these two very different classes of articles.
For the Generalist articles, the current suggested board structure might work. The board would be responsible for reviewing content, as well as interfacing with the general WP peer review process. In particular, we've identified a "bug" in he current WP peer review process: it does not recruit or solicit help from qualified Wikiprojects when conducting peer review. There is no mechanism to, for example, contact the Biology Wikiproject when an article about biology is nominated for good article status. This is wrong, and needs fixing.
For the Specialist articles, I suggest setting up a decentralized network of specialist boards: e.g. a board of string theorists who would be responsible for reviewing the content of string theory articles, etc. Each such board would be "licensed", and issued an imprimatur which they could stamp on reviewed articles: for example, "This article has been reviewed for scientific accuracy by the Ichthyologists Association of Wikipedia". These smaller groups would hash out their own procedures for vetting an article, which might vary from discipline to discipline. The pedigree of participants would also vary from discipline to discipline: there are 500 articles in Category:Elementary mathematics which could all be reviewed by the "Middle and High School Teachers Association of Wikipedia": clearly this does not require a PhD in physics, but does require experience in pedagogy. The only real question is whether we have enough participants and enough interest to get the specialist boards going. Its not clear that we do.
Can this work, then? linas 23:09, 17 March 2006 (UTC)
On general issues, there is really so much new material here while I have been watching a Rowing Regatta all day, that I think I have to sleep on it all before commenting. However, I will say one thing. I think Linas' division between generalist and specialist is too sharp. We do not jump from science to supergravity in one leap. There are rather a lot of articles that school students and undergraduates think they know enough about to edit, sometime with good results and sometimes with very bad results. In chemistry, for example, there are rather narrow articles such as Redox or VSEPR theory, which are 1st year undergraduate topics. These often contain errors and they need a reasonable expert to fix at least some of them. We really have a gradual change from general to specialist here. -- Bduke 07:51, 18 March 2006 (UTC)
Many of the issues surrounding the board / reviewers, specialist v. generalist, and other concerns are rooted in a framework of peer review that I think the community might well be better off without. So let me propose an alternative that could address many of the same issues regarding article improvement and maintenance of scientific topics. Technically, what follows could be created in addition to SPR, but I am inclined to believe it might be a better/more effective first step.
My radical suggestion is as follows: How about forming a "Scientific Contributors Noticeboard", where people could come to ask for assistance in the maintenance and improvement of technical topics. It could be broken out into disciplines, and those with appropriate technical training could be encouraged to watch the pages and contribute. To first order, what we need is a way to get scientific expertise to those articles that need it, and I believe that creating such a noticeboard would do so while being less bogged down in the formality of peer review.
This proposal obviously doesn't address the concern that our articles are never checked by what the outside world might view as authority, and so we might still need something like SPR, but I think the Nature study shows our content is already doing pretty well without it. As a first step, I think it makes more sense to try and build the scientific community within Wikipedia by providing a common forum for discussion of the scientific work within Wikipedia. Dragons flight 00:22, 18 March 2006 (UTC)
I've contributed to a fairly wide variety of Wikipedia articles, so I've had some experience with the wide variety of levels of expertise that may be applied to controversial articles. When I'm dealing with writing of people who are writing at or below my own level of expertise, it's usually fairly easy for me to see where there are obvious mistakes or problems, and it is usually fairly easy to find definitive sources that can be cited to establish whatever the current "wisdom of the field" may be. When I am dealing with articles that are out of my field and even a little over my head, then it may be perfectly obvious to me when the article is poorly written. When I have attempted to get these articles straightened out to the point that they at least contain comprehensible sentences I have encountered difficulties because when I have sought help in making clarifications from people who claim expertise (1) many of them have been satisfied with the existing formulations because (it seems to me) they believe that they understand what the formulations were intended to convey, (2) sometimes two "experts" will support opposing positions, and (3) I have been unable to determine who to believe because they stand on the authority of their own superior qualifications and offer no citations or even explications.
Some of the articles in Wikipedia, e.g., storm cellars could be badly written and inaccurate and it probably would not make a great deal of difference. Other articles are on the cutting edge of science, discuss topics where it would be a great service (especially to young people) to be both correct and also very clear, and most likely are controversial in the sense that scientific development has not ironed out all the problems and contending explanations exist.
In the generation of Bohr, Heisenberg, Schrödinger, Dirac, et al., who could have refereed a Wikipedia article on quantum theory? What would have been the correct way out if a physics professor Wikipedian from North Dakota had written an article on Dirac's theory and another physics professor from New Delhi had ripped it up? Who among us would know whether the Indian physicist was to be believed or whether the U.S. physicist had it right? If neither of their versions of such an article had any obvious flaws of logic or English-language presentation, would we go to a Venezuelan physicist to decide between the two of them? Or would we try to get some assessment of each writer/editor's merit in the field and then, if still necessary, find somebody whose qualifications were so much better than theirs that it would be reasonable to expect that s/he could sort things out and put matters that are in doubt in their proper perspective?
In the early 1900s we could have probably asked any of the several physicists mentioned above to "vet" an article, and any one of them would have treated the contributions made by "rivals" to the field in a fair and objective way. In the area of spiders I know of a few people who do not ordinarily write articles but who have international reputations in the field and who are willing to answer the occasional "dumb" question. I'm not sure whether somebody like George Greenstein would do that kind of thing for articles on astrophysics, or whether Brian Greene would straighten us out on quantum physics, but what we really need are people of that caliber if we really want an "imprimatur" on an advanced article. P0M 02:51, 18 March 2006 (UTC)
I decided to do some digging, and I want to share my finds. I really wanted to know how and why this Wikipedian could turn into this Wikipedian. I found that he was discontent with the running of the peer review and verification process: see this and this. Ultimately—and you don't know how bad I feel about this—I think he felt that we didn't want him here: I told him he was welcome to withdraw if he so desired. Sure enough, he did with the edit summary: don't expect me to support this project in any way in future. After saying a few other goodbyes, he simply left. It would be one thing to withdraw his nomination—though his peer review of Science looked great—but I never said I would support him leaving. I would never fully support anyone leaving besides vandals. I don't know whether this is cumulative or some sudden epiphany, or even a misunderstanding, but give Wikipedia another chance. Samsara, please forgive this samsara and join us.-- HereToHelp ( talk • contribs) 02:52, 18 March 2006 (UTC)
I have been following things from the sideline in the last days, and I have not read everything (especially earlier stuff), so maybe I am going to say some very stupid things.
I have the feeling there is a kind of mixing going on. Normally, you have the editorial board, and the outside reviewers. The editorial board can make decisions, but they are at the level of suitability for the journal, or they turn down articles that are blatantly incorrect. In general, they do not do the peer reviews themselves (although depends on the field).
So, why not set it up with this in mind. Have a small editorial board (2, 3 people to start with ), and a longer list of people who list themselves with their credentials and field of expertise. The editorial board should consist of people who can do the initial reading (it is at least written well, and understandable (at least the basic section) to lay people?) and can request people from the list to do the review. They can (in special cases) also go outside Wikipedia, and ask specialists for their comments (aka Nature).
Ok, enough rambling, shoot it down... :-) KimvdLinde 03:00, 18 March 2006 (UTC)
The "editorial board" concept was discussed up top, and many/most academics seemed to cotton on to it, as that is the structure they're used to. But then the discussion sank on whether one even needed to have expert credentials for reviewing certain types of articles. I'm now of the opinion that a light-weight, minimal process "noticeboard" might be the way to start. linas 17:21, 19 March 2006 (UTC)
Chris Hillman brought this up to me. There are some templates (for example Template:expertBiology) which are intended to tag an article as being in category:Articles needing review by a biology expert. The template will include a category. Might we at least settle on a phrase for the category, which can then be placed in the templates? I moot ..."Articles needing review by"... . Suggestions, please? These categories could then serve as organizing lists of Biology articles needing review, under review, already reviewed, Good articles in Biology, etc.
As currently implemented, the generated text can then be modified to suit the article. If an article had been reviewed, then the category could be updated as well, to reflect status. For example, ..."Article reviewed by"... on the Talk page. Thus, after a review of an article, WMC might see fit alter its category to category:Articles reviewed by William M. Connolley. Or perhaps category:Very Good articles reviewed by William M. Connolley.
If a reviewer were to amass enough of these reviews, as tagged in a category of his/her own, this might even constitute an objective list of the reviews by the reviewer on Wikipedia, and possibly even worthy of the notice of a Vice Chancellor. -- Ancheta Wis 12:03, 18 March 2006 (UTC)
I admit this requires administration. For example, there would need to be a page with a list of all the Science peers, containing links of the form category:Articles reviewed by William M. Connolley. But that would be a Good Thing if there were thousands of these reviews to administer. 12:03, 18 March 2006 (UTC)
Maybe we should start to make a list of people who are wiling to do peer reviews Wikipedia:Peer reviewers, and list them selves to a list with the short credentials, along the line of:
This list can be made regardless of the outcome of this discussion here. KimvdLinde 17:44, 18 March 2006 (UTC)
I created the page Wikipedia:Peer reviewers, so if you are available to do peer reviews at request, list yourself. -- KimvdLinde 18:59, 18 March 2006 (UTC)
So many new ideas and I do not have time to address them. I think we need someone, preferably Adam who started this all off, to try to write a summary of the current consensus with space for people to agree or disagree that each item is consensus.
I am about to strike out (note strike out, not delete) my nomination for membership of the Board. I recommend that all other nominees do likewise. It is not clear whether there is going to be a Board. If there is, it is not clear what it would do. It is not clear what the Project is going to do. For example, the above section on Wikipedia:Peer reviewers is possibly a great idea, but it seems to me to be setting up an alternative peer review of scientific articles system before we have decided whether we want this one. I am not clear any more that I want to be on any Board that arises. It all depends on what it is supposed to do and in what sort of project. -- Bduke 22:44, 18 March 2006 (UTC)
Ok, I am confused also. What is the task of a board member? KimvdLinde 23:46, 18 March 2006 (UTC)
Please feel free to edit, add to, delete or rearrange this summary if I have misspoken, especially other Users whom I may have neglected, please. -- Ancheta Wis 01:15, 19 March 2006 (UTC)
Dear contributors
I wonder whether user Tufkat's recent innovation, the Wikipedia:Featured_Music_Project, provides ideas that might be useful in achieving our goal of raising the standards of scientific articles. Tony 06:55, 19 March 2006 (UTC)
This might be interesting, but sorry, No, I for one do not want you to continue and do not think it helps matters.
Review for science in Wikipedia is exactly the same as for other areas. Are we reporting correctly what scientists are saying? Is what we write verified from sources or at the very least verifiable? Is it NPOV? The only difference is that it might need an expert to understand what the scientists are saying. There is no need for philosophy of any kind about the editing process or the review process. We are just writing an encyclopedia. -- Bduke 09:33, 19 March 2006 (UTC)
You are fundamentally misunderstanding the wikipedia process. Verifiability is about whether there is a reputable source for the information. See Wikipedia:Verifiability. It does mean that the information is true in some absolute philosophical sense, although of course we should use sources that we have reason to believe are reliable. We are not "judging sentences true, false and meaningless" in the sense that you use it. Look at Wikipedia:Neutral point of view and Wikipedia:Scientific point of view.
I would add that your suggestions elsewhere on this page that the review process should follow the scientific method are also misplaced.
I am reviewing an article right now for a Royal Society of Chemistry journal. I am not asked to follow the scientific method. I am asked to address a series of questions. There is no scientific way to review. It is a human process with flaws. An attempt here to develop one would be original research and that is not allowed on Wikipedia.
See Wikipedia:No original research. I note also that you defined scientific method as "aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.". It is not. That is just one way (very common, yes, but not the only way) to write up scientific reports and academic papers.
I thought at the time that you were attempting to add a light note to the debate, but it seems you were serious. -- Bduke 21:34, 19 March 2006 (UTC)
You and I (with I think wikipedia on my side) are at loggerheads. Do not confuse peer review of academic journal articles with the review of science articles proposed here. Read how WP uses the term "verify". It is about sources (not OR) and not about repeatability. Your search for a measurement procedure for articles is misplaced. Actually neither review process uses such a thing. Understand the difference between WP:NPOV and WP:SPOV. Any experienced scientist can judge whether an article in their own field is a good summary of what the standard texts and review articles are saying. You are trying to make it more difficult than it is. If there is a disagreement in the texts or articles, then all points of view should be given in a NPOV. We are not trying to resolve truth. This is an encyclopedia. -- Bduke 03:07, 20 March 2006 (UTC)
Ok Bduke lets see where we get with this (maybe I need to be beaten into submission):
Take an example. The scientific article, maximum_power_theorem, has one cited source: H.W. Jackson (1959) Introduction to Electronic Circuits, Prentice-Hall. Which I put there so it does not count. What is left are six external links.
I.e. how do I verify these sources? Sholto Maud 04:20, 20 March 2006 (UTC)
There is a tendency, and a bad one, to think that since WP is an internet encyclopedia that all references should be internet references. It may well be that this article is not well referenced. However, we also look at verifiable. Is, for example, the material in the article well covered by the book you added as a reference? If so you confirmed, perhaps indirectly, that the material is verified. Do the external links actually support what is said? What kind of links are they? Some web links are more plausable than others. If they do support it are they saying the same thing as standard texts. If the facts are not verifiable, then someone is likely to say so. This is a wiki afterall. -- Bduke 05:26, 20 March 2006 (UTC)
Ahem. Now lets take another example. Let's look at the foundational concept of most of natural science; i.e. power. The scientific article, Power_(physics). No sources.... Sholto Maud 05:29, 20 March 2006 (UTC)
To Sholto Maud Several points:-
But wait a second,
"Verified" in this context means that reader2 and/or other editor2, can repeat the process of going to a source to check the accuracy/reliability of editor1's works. Thus repeatability is inherent in verify. In the power article I am unable to repeat the process of going to a source. Thus the article is unverifiable and fails to comply to Wikipedia rules. This, by the way, is the same meaning of "unverifiable" as used in logical positivism. Sholto Maud 06:06, 20 March 2006 (UTC)
The repeatablility of going to a source is a bit different from repeatability of an experiment. However, what you says does mean it is unverifiable (unless the stuff is in the sources I mentioned - the book you say you added and the external links). However, it does not mean it is unveferifiable. Just go to a standard text and see whether the article is supported by the text. Add that text as a reference and the article is verified. Simple. Lots of articles do not comply with Wikipedia rules, but most of them are easily fixed. That is my last post for 15 hours or so. -- Bduke 06:15, 20 March 2006 (UTC)
Thanks User:Bduke! Have a well-earned break. Sholto Maud 06:49, 20 March 2006 (UTC)
The concept of verification appears to be a fundamental feature of the imagined SPR process. Representing the popular opinion of Wikipedia users, User:Bduke seems to agree that the Wikipedia concept of verification is the same as that used by logical positivists. A difficulty arises here because the logical positivist definition of verifiability defers to the correspondence theory of truth: " The correspondence theory of truth states that something is rendered true by the existence of a fact with corresponding elements and a similar structure". And this would appear to be what we do when conducting a scientific peer review. That is, we check whether a Wikipedia article is true to the facts of the reliable sources. But then this makes the Wikipedia philosophy of verification non-standard, because it is convcerned with " verifiability, not truth". Hence the Wikipeda theory of verification is Original Research. This would appear to qualify as a --FATAL ERROR--
Furthermore the Wikipedia theory of verifiability does not cite any reliable sources in defining the concept, thereby failing it's own conditions --FATAL ERROR-- Sholto Maud 09:50, 20 March 2006 (UTC)
I feel motivated to make a statement of the SPR process as given by User:Bduke above.
---!!!SEMANTIC ERROR: "Plausability" variable Y is undefined!!!---
---!!!SEMANTIC ERROR: "standard texts" variable is undefined!!!---
SPRbot demands the article is supported by standard texts or will be excluded in 7 days."
---!!!SEMANTIC ERROR: "article type" variable is undefined!!!---
---!!!SEMANTIC ERROR: "article type requirement" variable is undefined!!!---
---!!!SEMANTIC ERROR: "article content" variable is undefined!!!--- ---!!!SEMANTIC ERROR: "text book content" variable is undefined!!!---
Not so difficult. However this hasn't formalised the criteria by which to evaluate the reliability of a source. Sholto Maud 06:49, 20 March 2006 (UTC)
OK. So perhaps I'm revealing that I work in something more approximating the 'social' sciences, but ``it's the vibe" ( The Castle (film)).-- Limegreen 11:07, 20 March 2006 (UTC)
This page is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page. |
The past six days of brainstorming here (yes, this discussion has been going on for almost a week) have focused on "da board". I think it would be nice to discuss a little about the actual reviews for a change. My proposal: we pick a guinea pig article and try to carry out a review process in vitro. Karol 22:15, 15 March 2006 (UTC)
Alright: Science.-- HereToHelp ( talk • contribs) 23:30, 15 March 2006 (UTC)
OK, well, there's also a glaring error pertaining to physics in the subsection titled "philosophy of science". It’s exactly the kind of error a college kid might make; it implies that atoms are "mostly empty space"; they most certainly are not. Now what? I can fix this error, but so what? How does the board help here? Or let me rephrase: bduke says "It hardly contains any science itself, so it does not need experts to review it." -- yet it contains an error that non-physicists will almost certainly overlook. ... which is part of the problem with WP articles. linas 00:09, 16 March 2006 (UTC)
If you want to wet your teeth on something more technical, but not generally controversial, how about Structure of the Earth. It is a substantially technical article with only a small number of references, mostly to other websites, and few interesting problems like the percent compositions not equally 100%. Dragons flight 01:47, 16 March 2006 (UTC)
I was wondering about a trial of supergravity, not a trial of a review but more a trial of whether we could find a reviewer. Looking at the article raised some new issues. JarahE, who has nominated for the Board, could, from his nomination, do the review. However he has edited the article considerably in the last three days. Charles Mathews had an edit on it earlier, although it is only a modification to a category. Linas, who raised this article, has extensively edited it. Surely it would not do to have, as a reviewer, someone who had helped to write the article. So it might be difficult to find someone. This raises another issue. It might be best if the nomination of an article was only put on the Project page first, leaving the tag on the article's talk page to be added when we know we can review it. Maybe it does not matter. I do not know.
I think the process would be this. The Board or perhaps one person (this role might rotate) concludes that the expertise to review this article is not available among the Board members and reviewers list. S/he puts on this talk page:-
If nobody responds after a couple of days, they add:-
There might be a gap of a day or so between the two points above being added. If this gets a nominee to review the article, the Board consults and if they find the reviewer is expert enough, they ask them to do the review. Later the Board looks at the review and helps to put together a suitable report.
I think this is where the Board is useful. There is someone with a responsibility to try to find a reviewer. Without this, it is much less likely that someone will try to find one. -- Bduke 21:48, 16 March 2006 (UTC)
While I understand Samsara's point, could I suggest that we leave comments until there is an actual call to comment and vote? This candidate might withdraw. I think general comment on this talk page is OK, but let's leave the Project page for now. -- Bduke 00:29, 16 March 2006 (UTC)
HereToHelp has just altered the recommended headers for sub pages with this comment - (If we're not templating the subpages but rather linking them (sounds okay to me) we should use bigger headers.). The original proposal did have the sub page "templated" i.e in {{ }} brackets, but this was changed to [[ ]]. I let it go at the time, because I thought we could come back to it later. Now it seems is later. I think we should template in the sub-pages, in which case we should put the headers back to something like they were before. I grant you that this is a small technical point, but we need to address it sometime. Any thoughts? -- Bduke 03:32, 16 March 2006 (UTC)
Just wondering...if SPR stands for "scientific" peer review, should it then be reflexive and apply scientific method (aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.) to itself? Or does SPR have immunity from scientific method, and if so then how is the word "scientific" defined in the SPR acronym? Is there a "science" of SPR? Should SPR demonstrate the property "science" which the peer reviewer lays claims to?
Let's try an example - Samsara's current SPR of the Science article.
Title
A general model of scientific peer review in the open source user-defined environment.
Abstract
Open source user-defined scientific peer review (SPR) is an emergent property of the Wikipedia internet technology. Initial steps are taken to produce a model of SPR with the aim of SPR formalization. Two definitions of 'science' were identified as needing analysis and universally agreed definition. Results are pending.
To define the term "science" in such a way that it will be achieve maximal agreement among the world's population of scientists.
Hypothesis
<Not sure about this...>
Scientific method says
<I'm not sure what the mathematical model here is. Being mathematical it would suggest that my/our results would have to be in some way quantitative. What would be the quantities>
Hypothesis says
<Ok so the scientific community could prove the aim false by disagreeing with the definition? This leads me to think that each definition & article needs a means of a user formally registering and displaying (a counter) ascend to the definition.>
Samsara's SPR of the Science article was analyzed through the Wikipedia collaborative online method. In particular, two definitions of science were analyzed.
Wikipedia portal, computer, internet access.
Analysis
Two definitions of science have been obtained throughout the process of SPR.
"science is a system of knowledge acquisition"
"science is an exercise in model selection"
Results
Pending...
Time: ?!
$: ?!
Time/$ efficiency = ?!
Energy: ?! joules
Exergy: ?!
Embodied energy: ?! embodied joules
EROEI: ?1
Conclusions
Recommendations
Bibliography
Sholto Maud 04:45, 16 March 2006 (UTC)
I was going to nominate Classical theories of gravitation but it currently sits in a category with 100's of members. The category:pages needing expert attention does not currently separate out the Natural science articles. But there is currently only 'Template:expert' and 'Template:Technical (expert)' for the remediation markup. So, what would everyone like to see:
perhaps? That would invite expert attention to the article, before or perhaps in the process of a review. What other templates might come to mind? -- Ancheta Wis 11:30, 16 March 2006 (UTC)
I propose that we create 1 template for each named science (Biology for example), and invoke it with {{subst:Expert|Biology}} for example -- Ancheta Wis 13:29, 16 March 2006 (UTC)
Samsara ( talk • contribs) 13:32, 16 March 2006 (UTC)
OK, here is the usage.
One of these notices, when placed in an article, ought to feed right to the cleanup team who can then route the names of the respective articles to an appropriate Peer review pool.
It is possible, and in my opinion preferable, as it makes for a more pleasant reading experience, to place this notice on the Talk page of the article, which will then appear properly alphabetized in the category listings. -- Ancheta Wis 13:58, 16 March 2006 (UTC)
Splash, I have also created the analogous expertScience expertTechnology and expertHistory templates. Others may wish to create Med. and Law templates, but those would need disclaimers. --14:07, 16 March 2006 (UTC)
It should be possible to coordinate this project with the entries in Wikipedia:Pages needing attention/Mathematical and Natural Sciences
The templates can have an embedded category in them to facilitate searches. Example: category:Biology page needing Scientific peer review -- Ancheta Wis 15:46, 16 March 2006 (UTC)
Would a single template do? Called like:-
We would then not have to keep writing new ones. -- Bduke 21:07, 16 March 2006 (UTC)
I think that the idea of a scientific board is great (one of the best idea so far), but I disagree with one point : it seems to me that creating a board with people of diverse origins is a bad idea. I would prefer several boards specialized on one topic (astronomy, biology, chemistry). I think that the community is large enough so that we can find seven high-level scientist in every field. The problem is see right now is that I don't know how an astronom will be able to make any correction on the content of a biology article (for example). So, the goal of the committee would fail. What do you think about creating several committees ? Poppypetty 13:11, 16 March 2006 (UTC)
Let's not overstate the value of multiple expert input into each article; that's what the contributors should supply. If at least one expert in each area is on the board, that's fine: the rest will all be scientists, and so much of critiquing a scientific text is concerned with scientific method and scientific language, in which all board members should have expertise.
Having said that, I think your idea may be prophetic. If this board works well, perhaps it will evolve to subdivide itself naturally. Let's try to get the process right with this overarching board first, don't you think? Tony 14:11, 16 March 2006 (UTC)
Being an expert in the Wikipedia Manual of Style or in editing articles does NOT make anyone an expert in any scientific fields. I fully realize that my next statement may offend some people, but I would be remiss not to say it. Anyone that is still an undergraduate, a post-graduate or a post-doc in any university is NOT per se a scientific expert no matter how brilliant a student he or she may be. Anyone without at least 10 years of real world experience in a specific field is NOT a scientific expert. mbeychok 23:33, 16 March 2006 (UTC)
If my nomination is going to cause a problem, then I shall withdraw. I agree, being a student does not make me a expert but the idea behind my nomination was to be a 2nd opinion, not an expert but someone who can understand what your talking about. I think its important to have a 2nd party (in all forms of review) but physics being physics is not understandable by all, so someone who can understand physics, but who is not an expert, ie, a physics student would be an idea. Above was by User:Eevo 11:53, 17 March 2006.
As the nominee who meets mbeychok's criteria in spades, it being 42 years since I did my D Phil and I worked in academia until 2002 with an Honorary Reserarch position since then, I want to say that I think his comment is quite silly in the Wikipedia context and I do not appreciate being shouted at (Capital letter, NOT). To Eevo, I say that being a student is not the issue. Your lack of experience on Wikipedia may be. -- Bduke 01:09, 17 March 2006 (UTC)
Lack of experience would not be a problem, under the name Evo, I have about 10 edits, under no name, i have about 800 edits (about 200 in physics/astro).Evo
To Turmosis. Nobody is suggesting that you need 10 years of experience to edit Wikipedia. The question is about expert reviews of articles about science. First year students make great Wikipedians. My experience suggests they would get the science wrong quite often. To Linas. I think we have to cover both basic science and advanced science. If we only do the former, we do not need this project. We just need a mechanism on the various WikiProject Science pages to send people over to WP:PR to look at the science and you can put the articles there that need attention. Of course in one sense the whole discussion is not resolvable. I suspect the WP:SPR will try to deal with any article that gets nominated. That might be a weakness, but whatever guidelines we agree on now are likely to be ignored. -- Bduke 01:27, 17 March 2006 (UTC)
I just realized that there really isn’t a point to this discussion. All the people are still in the nomination stage and I am sure the community will make the right decision for who to vote for. I don’t think mbeychok has anything to worry about since I am sure everyone understands that only people with scientific knowledge are suitable for this. Tutmosis 01:46, 17 March 2006 (UTC)
PLEASE let's not invest too much in this election. It's the process of conducting an experiment in a new structure on WP for improving articles that is much more important than any individual. Whoever misses out this time can stand again, and who knows, by then, there might be more 'boards' with narrower specialisations.
I can see that nominees who miss out this time may drop the bucket and be inclined not to participate at all (that would be my immediate impulse if I were standing and lost). For this reason, I wonder whether people might consider that all nominees who miss out be automatically offered the position of 'Special science contributor', or an equivalent. They represent talent that we cannot afford to lose. Tony 02:41, 17 March 2006 (UTC)
On second thought, perhaps not 'all', which might encourage nominations solely for that purpose; but the new board might give serious consideration to motivating those who didn't succeed. Tony 02:46, 17 March 2006 (UTC)
I think what makes someone good for reviewing articles is:
I think the credentials matter-- but they only go so far. [1] Everyone that has spent some time reading peer-reviewed scientific literature has come across papers that have mistakes and others that are completely wrong.
What really matters, in my opinion, is the above three points and a track record that demonstrates that. You can have people that do things for a hobby and are really darn good at 'em. Einstein did a lot of his best work when he was working at the patent office. Srinivasa Ramanujan basically had no credentials whatsoever... but made great contributions to his area.
While I agree that a lot of crap comes from people with a bit of knowledge and a big ego, we should think about how we can design a process that lets the talented contribute, revise and review. The key, I think, is not selecting the people with the best credentials-- it is selecting the best people. Nephron 23:40, 18 March 2006 (UTC)
A wiki depends on a set of close-knit relationships (meaning concord among its members) in support of some project, such as an article, or an intellectual achievement. A scientific community is knit from similar interests. A group of scientific peers, by definition, is a group of equals (in some sense). When a peer review occurs, the participants coöperate (as defined among themselves). The coöperation is the project.
But principled actions can be respected.
It is not cause for celebration when we lose a contributing/producing member of any community. I would like to voice my regret at the circumstances which have prompted Samsara's resignation from Wikipedia and furthermore thank him for his contributions in our behalf. -- Ancheta Wis 09:35, 17 March 2006 (UTC)
Indeed, it is not cause for any kind of celebration. I have no idea however why Samsara has done this. We had a reasonable exchange of views on our talk pages not too long before he left. He expressed a bit of frustration but nothing more. I hope someone can get him back. -- Bduke 11:14, 17 March 2006 (UTC)
This kind of situation is why I (and also people above) suggested a formal 'expert/authority' weighting system. It is also why I have tried to talk about the model and science of SPR, else we seem to leave the scientific community open to internal emotional turmoil, which is not scientific... or is it? I think it is worth repeating my question: Does SPR need to be scientific in the sense that it should follow scientific method? (or am I talking to myself here? :) ) Sholto Maud 12:45, 17 March 2006 (UTC)
I think using the scientific method here is overly bureaucratic. I have no idea why Samsara left, and I too regret the loss. If, at the very minimum, we knew why...-- HereToHelp ( talk • contribs) 12:48, 17 March 2006 (UTC)
I believe the discussion and the experiment has identified several issues that are at this moment unresolved. I'll try to recap here. I'd like discussion to work around getting consensus on these.
What type of article should this project focus on? Generalist and overview articles, such as Science, etc. or narrow, specialist topics? Almost all of the rest of the debate on this issue hinges on the answer to this question, including what the structure and organization will be, who the participants will be, what their qualifications might be, etc.
Generalist articles:
Specialist articles:
Set up two very distinct systems to deal with these two very different classes of articles.
For the Generalist articles, the current suggested board structure might work. The board would be responsible for reviewing content, as well as interfacing with the general WP peer review process. In particular, we've identified a "bug" in he current WP peer review process: it does not recruit or solicit help from qualified Wikiprojects when conducting peer review. There is no mechanism to, for example, contact the Biology Wikiproject when an article about biology is nominated for good article status. This is wrong, and needs fixing.
For the Specialist articles, I suggest setting up a decentralized network of specialist boards: e.g. a board of string theorists who would be responsible for reviewing the content of string theory articles, etc. Each such board would be "licensed", and issued an imprimatur which they could stamp on reviewed articles: for example, "This article has been reviewed for scientific accuracy by the Ichthyologists Association of Wikipedia". These smaller groups would hash out their own procedures for vetting an article, which might vary from discipline to discipline. The pedigree of participants would also vary from discipline to discipline: there are 500 articles in Category:Elementary mathematics which could all be reviewed by the "Middle and High School Teachers Association of Wikipedia": clearly this does not require a PhD in physics, but does require experience in pedagogy. The only real question is whether we have enough participants and enough interest to get the specialist boards going. Its not clear that we do.
Can this work, then? linas 23:09, 17 March 2006 (UTC)
On general issues, there is really so much new material here while I have been watching a Rowing Regatta all day, that I think I have to sleep on it all before commenting. However, I will say one thing. I think Linas' division between generalist and specialist is too sharp. We do not jump from science to supergravity in one leap. There are rather a lot of articles that school students and undergraduates think they know enough about to edit, sometime with good results and sometimes with very bad results. In chemistry, for example, there are rather narrow articles such as Redox or VSEPR theory, which are 1st year undergraduate topics. These often contain errors and they need a reasonable expert to fix at least some of them. We really have a gradual change from general to specialist here. -- Bduke 07:51, 18 March 2006 (UTC)
Many of the issues surrounding the board / reviewers, specialist v. generalist, and other concerns are rooted in a framework of peer review that I think the community might well be better off without. So let me propose an alternative that could address many of the same issues regarding article improvement and maintenance of scientific topics. Technically, what follows could be created in addition to SPR, but I am inclined to believe it might be a better/more effective first step.
My radical suggestion is as follows: How about forming a "Scientific Contributors Noticeboard", where people could come to ask for assistance in the maintenance and improvement of technical topics. It could be broken out into disciplines, and those with appropriate technical training could be encouraged to watch the pages and contribute. To first order, what we need is a way to get scientific expertise to those articles that need it, and I believe that creating such a noticeboard would do so while being less bogged down in the formality of peer review.
This proposal obviously doesn't address the concern that our articles are never checked by what the outside world might view as authority, and so we might still need something like SPR, but I think the Nature study shows our content is already doing pretty well without it. As a first step, I think it makes more sense to try and build the scientific community within Wikipedia by providing a common forum for discussion of the scientific work within Wikipedia. Dragons flight 00:22, 18 March 2006 (UTC)
I've contributed to a fairly wide variety of Wikipedia articles, so I've had some experience with the wide variety of levels of expertise that may be applied to controversial articles. When I'm dealing with writing of people who are writing at or below my own level of expertise, it's usually fairly easy for me to see where there are obvious mistakes or problems, and it is usually fairly easy to find definitive sources that can be cited to establish whatever the current "wisdom of the field" may be. When I am dealing with articles that are out of my field and even a little over my head, then it may be perfectly obvious to me when the article is poorly written. When I have attempted to get these articles straightened out to the point that they at least contain comprehensible sentences I have encountered difficulties because when I have sought help in making clarifications from people who claim expertise (1) many of them have been satisfied with the existing formulations because (it seems to me) they believe that they understand what the formulations were intended to convey, (2) sometimes two "experts" will support opposing positions, and (3) I have been unable to determine who to believe because they stand on the authority of their own superior qualifications and offer no citations or even explications.
Some of the articles in Wikipedia, e.g., storm cellars could be badly written and inaccurate and it probably would not make a great deal of difference. Other articles are on the cutting edge of science, discuss topics where it would be a great service (especially to young people) to be both correct and also very clear, and most likely are controversial in the sense that scientific development has not ironed out all the problems and contending explanations exist.
In the generation of Bohr, Heisenberg, Schrödinger, Dirac, et al., who could have refereed a Wikipedia article on quantum theory? What would have been the correct way out if a physics professor Wikipedian from North Dakota had written an article on Dirac's theory and another physics professor from New Delhi had ripped it up? Who among us would know whether the Indian physicist was to be believed or whether the U.S. physicist had it right? If neither of their versions of such an article had any obvious flaws of logic or English-language presentation, would we go to a Venezuelan physicist to decide between the two of them? Or would we try to get some assessment of each writer/editor's merit in the field and then, if still necessary, find somebody whose qualifications were so much better than theirs that it would be reasonable to expect that s/he could sort things out and put matters that are in doubt in their proper perspective?
In the early 1900s we could have probably asked any of the several physicists mentioned above to "vet" an article, and any one of them would have treated the contributions made by "rivals" to the field in a fair and objective way. In the area of spiders I know of a few people who do not ordinarily write articles but who have international reputations in the field and who are willing to answer the occasional "dumb" question. I'm not sure whether somebody like George Greenstein would do that kind of thing for articles on astrophysics, or whether Brian Greene would straighten us out on quantum physics, but what we really need are people of that caliber if we really want an "imprimatur" on an advanced article. P0M 02:51, 18 March 2006 (UTC)
I decided to do some digging, and I want to share my finds. I really wanted to know how and why this Wikipedian could turn into this Wikipedian. I found that he was discontent with the running of the peer review and verification process: see this and this. Ultimately—and you don't know how bad I feel about this—I think he felt that we didn't want him here: I told him he was welcome to withdraw if he so desired. Sure enough, he did with the edit summary: don't expect me to support this project in any way in future. After saying a few other goodbyes, he simply left. It would be one thing to withdraw his nomination—though his peer review of Science looked great—but I never said I would support him leaving. I would never fully support anyone leaving besides vandals. I don't know whether this is cumulative or some sudden epiphany, or even a misunderstanding, but give Wikipedia another chance. Samsara, please forgive this samsara and join us.-- HereToHelp ( talk • contribs) 02:52, 18 March 2006 (UTC)
I have been following things from the sideline in the last days, and I have not read everything (especially earlier stuff), so maybe I am going to say some very stupid things.
I have the feeling there is a kind of mixing going on. Normally, you have the editorial board, and the outside reviewers. The editorial board can make decisions, but they are at the level of suitability for the journal, or they turn down articles that are blatantly incorrect. In general, they do not do the peer reviews themselves (although depends on the field).
So, why not set it up with this in mind. Have a small editorial board (2, 3 people to start with ), and a longer list of people who list themselves with their credentials and field of expertise. The editorial board should consist of people who can do the initial reading (it is at least written well, and understandable (at least the basic section) to lay people?) and can request people from the list to do the review. They can (in special cases) also go outside Wikipedia, and ask specialists for their comments (aka Nature).
Ok, enough rambling, shoot it down... :-) KimvdLinde 03:00, 18 March 2006 (UTC)
The "editorial board" concept was discussed up top, and many/most academics seemed to cotton on to it, as that is the structure they're used to. But then the discussion sank on whether one even needed to have expert credentials for reviewing certain types of articles. I'm now of the opinion that a light-weight, minimal process "noticeboard" might be the way to start. linas 17:21, 19 March 2006 (UTC)
Chris Hillman brought this up to me. There are some templates (for example Template:expertBiology) which are intended to tag an article as being in category:Articles needing review by a biology expert. The template will include a category. Might we at least settle on a phrase for the category, which can then be placed in the templates? I moot ..."Articles needing review by"... . Suggestions, please? These categories could then serve as organizing lists of Biology articles needing review, under review, already reviewed, Good articles in Biology, etc.
As currently implemented, the generated text can then be modified to suit the article. If an article had been reviewed, then the category could be updated as well, to reflect status. For example, ..."Article reviewed by"... on the Talk page. Thus, after a review of an article, WMC might see fit alter its category to category:Articles reviewed by William M. Connolley. Or perhaps category:Very Good articles reviewed by William M. Connolley.
If a reviewer were to amass enough of these reviews, as tagged in a category of his/her own, this might even constitute an objective list of the reviews by the reviewer on Wikipedia, and possibly even worthy of the notice of a Vice Chancellor. -- Ancheta Wis 12:03, 18 March 2006 (UTC)
I admit this requires administration. For example, there would need to be a page with a list of all the Science peers, containing links of the form category:Articles reviewed by William M. Connolley. But that would be a Good Thing if there were thousands of these reviews to administer. 12:03, 18 March 2006 (UTC)
Maybe we should start to make a list of people who are wiling to do peer reviews Wikipedia:Peer reviewers, and list them selves to a list with the short credentials, along the line of:
This list can be made regardless of the outcome of this discussion here. KimvdLinde 17:44, 18 March 2006 (UTC)
I created the page Wikipedia:Peer reviewers, so if you are available to do peer reviews at request, list yourself. -- KimvdLinde 18:59, 18 March 2006 (UTC)
So many new ideas and I do not have time to address them. I think we need someone, preferably Adam who started this all off, to try to write a summary of the current consensus with space for people to agree or disagree that each item is consensus.
I am about to strike out (note strike out, not delete) my nomination for membership of the Board. I recommend that all other nominees do likewise. It is not clear whether there is going to be a Board. If there is, it is not clear what it would do. It is not clear what the Project is going to do. For example, the above section on Wikipedia:Peer reviewers is possibly a great idea, but it seems to me to be setting up an alternative peer review of scientific articles system before we have decided whether we want this one. I am not clear any more that I want to be on any Board that arises. It all depends on what it is supposed to do and in what sort of project. -- Bduke 22:44, 18 March 2006 (UTC)
Ok, I am confused also. What is the task of a board member? KimvdLinde 23:46, 18 March 2006 (UTC)
Please feel free to edit, add to, delete or rearrange this summary if I have misspoken, especially other Users whom I may have neglected, please. -- Ancheta Wis 01:15, 19 March 2006 (UTC)
Dear contributors
I wonder whether user Tufkat's recent innovation, the Wikipedia:Featured_Music_Project, provides ideas that might be useful in achieving our goal of raising the standards of scientific articles. Tony 06:55, 19 March 2006 (UTC)
This might be interesting, but sorry, No, I for one do not want you to continue and do not think it helps matters.
Review for science in Wikipedia is exactly the same as for other areas. Are we reporting correctly what scientists are saying? Is what we write verified from sources or at the very least verifiable? Is it NPOV? The only difference is that it might need an expert to understand what the scientists are saying. There is no need for philosophy of any kind about the editing process or the review process. We are just writing an encyclopedia. -- Bduke 09:33, 19 March 2006 (UTC)
You are fundamentally misunderstanding the wikipedia process. Verifiability is about whether there is a reputable source for the information. See Wikipedia:Verifiability. It does mean that the information is true in some absolute philosophical sense, although of course we should use sources that we have reason to believe are reliable. We are not "judging sentences true, false and meaningless" in the sense that you use it. Look at Wikipedia:Neutral point of view and Wikipedia:Scientific point of view.
I would add that your suggestions elsewhere on this page that the review process should follow the scientific method are also misplaced.
I am reviewing an article right now for a Royal Society of Chemistry journal. I am not asked to follow the scientific method. I am asked to address a series of questions. There is no scientific way to review. It is a human process with flaws. An attempt here to develop one would be original research and that is not allowed on Wikipedia.
See Wikipedia:No original research. I note also that you defined scientific method as "aim, hypothesis, equipment, Results, Discussion, Conclusion, Bibliography, etc.". It is not. That is just one way (very common, yes, but not the only way) to write up scientific reports and academic papers.
I thought at the time that you were attempting to add a light note to the debate, but it seems you were serious. -- Bduke 21:34, 19 March 2006 (UTC)
You and I (with I think wikipedia on my side) are at loggerheads. Do not confuse peer review of academic journal articles with the review of science articles proposed here. Read how WP uses the term "verify". It is about sources (not OR) and not about repeatability. Your search for a measurement procedure for articles is misplaced. Actually neither review process uses such a thing. Understand the difference between WP:NPOV and WP:SPOV. Any experienced scientist can judge whether an article in their own field is a good summary of what the standard texts and review articles are saying. You are trying to make it more difficult than it is. If there is a disagreement in the texts or articles, then all points of view should be given in a NPOV. We are not trying to resolve truth. This is an encyclopedia. -- Bduke 03:07, 20 March 2006 (UTC)
Ok Bduke lets see where we get with this (maybe I need to be beaten into submission):
Take an example. The scientific article, maximum_power_theorem, has one cited source: H.W. Jackson (1959) Introduction to Electronic Circuits, Prentice-Hall. Which I put there so it does not count. What is left are six external links.
I.e. how do I verify these sources? Sholto Maud 04:20, 20 March 2006 (UTC)
There is a tendency, and a bad one, to think that since WP is an internet encyclopedia that all references should be internet references. It may well be that this article is not well referenced. However, we also look at verifiable. Is, for example, the material in the article well covered by the book you added as a reference? If so you confirmed, perhaps indirectly, that the material is verified. Do the external links actually support what is said? What kind of links are they? Some web links are more plausable than others. If they do support it are they saying the same thing as standard texts. If the facts are not verifiable, then someone is likely to say so. This is a wiki afterall. -- Bduke 05:26, 20 March 2006 (UTC)
Ahem. Now lets take another example. Let's look at the foundational concept of most of natural science; i.e. power. The scientific article, Power_(physics). No sources.... Sholto Maud 05:29, 20 March 2006 (UTC)
To Sholto Maud Several points:-
But wait a second,
"Verified" in this context means that reader2 and/or other editor2, can repeat the process of going to a source to check the accuracy/reliability of editor1's works. Thus repeatability is inherent in verify. In the power article I am unable to repeat the process of going to a source. Thus the article is unverifiable and fails to comply to Wikipedia rules. This, by the way, is the same meaning of "unverifiable" as used in logical positivism. Sholto Maud 06:06, 20 March 2006 (UTC)
The repeatablility of going to a source is a bit different from repeatability of an experiment. However, what you says does mean it is unverifiable (unless the stuff is in the sources I mentioned - the book you say you added and the external links). However, it does not mean it is unveferifiable. Just go to a standard text and see whether the article is supported by the text. Add that text as a reference and the article is verified. Simple. Lots of articles do not comply with Wikipedia rules, but most of them are easily fixed. That is my last post for 15 hours or so. -- Bduke 06:15, 20 March 2006 (UTC)
Thanks User:Bduke! Have a well-earned break. Sholto Maud 06:49, 20 March 2006 (UTC)
The concept of verification appears to be a fundamental feature of the imagined SPR process. Representing the popular opinion of Wikipedia users, User:Bduke seems to agree that the Wikipedia concept of verification is the same as that used by logical positivists. A difficulty arises here because the logical positivist definition of verifiability defers to the correspondence theory of truth: " The correspondence theory of truth states that something is rendered true by the existence of a fact with corresponding elements and a similar structure". And this would appear to be what we do when conducting a scientific peer review. That is, we check whether a Wikipedia article is true to the facts of the reliable sources. But then this makes the Wikipedia philosophy of verification non-standard, because it is convcerned with " verifiability, not truth". Hence the Wikipeda theory of verification is Original Research. This would appear to qualify as a --FATAL ERROR--
Furthermore the Wikipedia theory of verifiability does not cite any reliable sources in defining the concept, thereby failing it's own conditions --FATAL ERROR-- Sholto Maud 09:50, 20 March 2006 (UTC)
I feel motivated to make a statement of the SPR process as given by User:Bduke above.
---!!!SEMANTIC ERROR: "Plausability" variable Y is undefined!!!---
---!!!SEMANTIC ERROR: "standard texts" variable is undefined!!!---
SPRbot demands the article is supported by standard texts or will be excluded in 7 days."
---!!!SEMANTIC ERROR: "article type" variable is undefined!!!---
---!!!SEMANTIC ERROR: "article type requirement" variable is undefined!!!---
---!!!SEMANTIC ERROR: "article content" variable is undefined!!!--- ---!!!SEMANTIC ERROR: "text book content" variable is undefined!!!---
Not so difficult. However this hasn't formalised the criteria by which to evaluate the reliability of a source. Sholto Maud 06:49, 20 March 2006 (UTC)
OK. So perhaps I'm revealing that I work in something more approximating the 'social' sciences, but ``it's the vibe" ( The Castle (film)).-- Limegreen 11:07, 20 March 2006 (UTC)