Operator: GreenC ( talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)
Time filed: 01:14, Friday, December 28, 2018 ( UTC)
Function overview: Add {{
Unreferenced}}
and {{
No footnotes}}
to the tops of pages that have no references or are missing in-line footnotes.
Automatic, Supervised, or Manual: Automatic
Programming language(s): GNU Awk and BotWikiAwk framework
Source code available: Yes (TBU)
Links to relevant discussions (where appropriate):
Edit period(s): one time run
Estimated number of pages affected: ~ 130,000
Namespace(s): Mainspace articles
Exclusion compliant (Yes/No): Yes
Function details:
As background, members of New Page Patrollers ( WP:NPP) have caught up tagging the backlog of new pages. However there is still an older backlist of articles created since day 1 up to about 2012 which still contain many untagged articles. Estimates could be half a million or more untagged. A request was made on BOTREQ by a NPP member. I took a try at creating an algorithm to detect when a page could reasonably be tagged. Dry-run tests on 10,000 articles show it be successful. Discussion at Wikipedia_talk:WikiProject_Unreferenced_articles#List_of_unreferenced_articles shows support for an automated bot to help find articles needing attention and tag the pages.
Test results are available at User:GreenC/data/noref
The bot will start slowly and be fully supervised initially, running in batches, checking results.
What kinds of main namespace pages are you exempting? (e.g. Redirects, Disambiguation pages?) — xaosflux Talk 01:56, 28 December 2018 (UTC) reply
{{
EB1911}}
and dabs), the HTML tag <ol class="references">
, any that begin with "List of ..", "Index of .." or "<year> in .." --
Green
C 02:22, 28 December 2018 (UTC)
reply{{
no footnotes}}
. --
Green
C 13:41, 9 January 2019 (UTC)
replyApproved for trial (10 edits for each tag.). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Let's see it in action then. Headbomb { t · c · p · b} 16:45, 9 January 2019 (UTC) reply
@ GreenC: feel free to proceed with the trial whenever you want. Headbomb { t · c · p · b} 18:49, 13 January 2019 (UTC) reply
@ Headbomb: - please withdraw the BRFA. It is going to prove too controversial. Not that I agree (otherwise i wouldn't have made the BRFA) but there are evidently some old wounds in the community about tagging and this bot will reopen old battle scars. And there is more than 1 way to make use of the tool, it's purpose is to discover and identify potential candidate articles, information others can do with as they please. If there is support to re-open the BRFA it should go through VP or an RFC first. -- Green C 15:28, 14 January 2019 (UTC) reply
I have opened a discussion at Wikipedia:Village_pump_(proposals)#Bot to add Template:Unreferenced and Template:No footnotes to pages (single run) to ease or confirm GreenC's fears. Any watching this page may be interested in commenting there. Thanks all for comments. Sorry to get in the middle of the BRFA here. Cheers! Ajpolino ( talk) 17:20, 14 January 2019 (UTC)' reply
Hi. Would a page such as List of United States Supreme Court cases, volume 586 be tagged by this bot?
I'm not convinced that tagging pages such as
Grand Ducal Highness with {{
no footnotes}}
is particularly helpful, but shrug. If it's a one-time run, people can presumably just remove the ugly tags if they don't like them. --
MZMcBride (
talk) 01:41, 15 January 2019 (UTC)
reply
{{
no footnotes}}
assuming there is consensus. In this page,
User:GreenC/data/noref/14001-15000 it shows three independent bot algorithms. Which algorithm(s) the bot deploys is up to the community. It can do the first, or all three, or some combo. By default it will do all three, but if there is concern about the two {{
no footnotes}}
algos one or both could be dropped. --
Green
C 02:52, 15 January 2019 (UTC)
replyThe bot doesn't have support for a parenthetical referencing either bare parentheticals or ones generated by {{ harv}}. It lists Nummer 5 as an article to be tagged as type 2 {{ no footnotes}} though the article has inline citations generated by {{ harv}} and {{ harvnb}}. It also lists Fossilized affixes in Austronesian languages to be tagged as type 2 {{ no footnotes}} though it uses parenthetical citations with page numbers. Wugapodes [thɑk] [ˈkan.ˌʧɹɪbz] 08:25, 17 January 2019 (UTC) reply
Trial complete. - 20 edits at Special:Contributions/GreenC_bot on January 17 ("via noref bot") -- Green C 20:11, 19 January 2019 (UTC) reply
pageText = pageText.replace(/^\s*(?:((?:\s*\{\{\s*(?:about|correct title|dablink|distinguish|for|other\s?(?:hurricaneuses|people|persons|places|uses(?:of)?)|redirect(?:-acronym)?|see\s?(?:also|wiktionary)|selfref|the)\d*\s*(\|(?:\{\{[^{}]*\}\}|[^{}])*)?\}\})+(?:\s*\n)?)\s*)?/i, "$1" + tagText);
(source: Twinkle)
SD0001 (
talk) 15:46, 25 January 2019 (UTC)
reply
A user has requested the attention of a member of the
Bot Approvals Group. Once assistance has been rendered, please deactivate this tag by replacing it with {{
t|BAG assistance needed}}
.
.
The RfC seems to have run its course (over 30 days). There are at least 4 ways this bot can run, from ultra-conservative to regular-conservative, so it need not be a black and white decision. There is currently some sort of majority that wants the bot to run in some form and their voices will hopefully not be ignored. I also note, many of the opposer arguments are based in FUD, as the bot writer I can assert most of those claims are either technically incorrect are very much on the margins based on real-world testing results. --
Green
C 16:52, 24 February 2019 (UTC)
reply
{{
unreferenced}}
which strongly suggests there is consensus for tagging, so that is not the "contentious" issue it is something else. It's not accuracy, because the bot is even more accurate than human taggers evidently. It's more discriminate than manual tagging. It's not volume because the bot will be limited in total edits it won't go hog wild or over-tag pages. The only thing left is an opinion against tagging generally, which is a minority position on Wikipedia. --
Green
C 23:33, 25 February 2019 (UTC)
reply
{{
unreferenced}}
to categorize the tagged articles separately. If it's possible to do this while also having them added to
Category:All articles lacking sources (which the unreferenced tag currently does), I think that would be most desirable since it would facilitate searching through all unreferenced articles (which would be possible, but a slight pain in the butt if a chunk of unreferenced articles are in some different category). Thanks all for your help in moving this along! I'll get out of your hair now and promise not to start any further trouble. Thoughts on moving forward?
Ajpolino (
talk) 04:13, 1 March 2019 (UTC)
reply
{{
unreferenced}}
or a fork of that template just for this bot. It's not worth introducing all those forking or one-off special case complications that no one will remember soon enough and creates complications for those clearing out categories. Either the bot is working correctly or it's not, and if not, then it will stop. It won't be full speed and walk away, it will be carefully monitored just like every other bot I run. If folks just want a list of articles the bot tagged, that can be done easily by searching on the bot's unique edit-comment string (I can leave instructions on the bot page).|source=
parameter to {{
unreferenced}}
that adds the article to a
per-source category is not a big deal. The majority of instances of the template won't make use of it, but if there is a need for it elsewhere (another bot, another task for this bot, some kind of categorization for the NPP, etc.) it's there and can be used. The category created would typically be a sub-cat of the relevant main maintenance category. It's a bit excessive for the (lack of) risk this task entails, but it's neither excessively onerous nor entirely wasted. *shrug* {{
unreferenced}}
, contingent only on it meeting a reasonable rate of false positives in an extended trial. So get |source=
added to {{
unreferenced}}
; ask BAG for a couple hundred trial edits; and post a link to the diffs at VP/P for review. --
Xover (
talk) 07:17, 1 March 2019 (UTC) Edited: --
Xover (
talk) 14:28, 1 March 2019 (UTC)
reply
{{
Unreferenced}}
has about 220k instances. So an increase of maybe 10% is not a big deal. If you want to implement it in the template I'll add it to the code but it's a needless complication IMO that will soon be forgotten and unused. Now, further VP/P discussions? That could throw this BRFA off the rails and leave the BAG operator uncertain with how to proceed. This is how it should work: The RfC closed in support of the bot. We return to BRFA and the BAG operator takes over. They approve (more) trial edits. Anyone can monitor the BRFA and trials and make comments for an extended period. The BAG operator then ultimately decides to accept or reject the bot based on community input and technical performance. It's always been done this way. The RFC says "notify the community once they have finished the bot's trial", BRFA is "the community" and this BRFA is well advertised and if someone wants to post more notifications elsewhere they are welcome to do so, but the RFC doesn't change the normal BRFA process. Doing so undermines that process and makes it hard for BAG operators to close it out when there are open, unresolved or disputed discussions happening elsewhere. --
Green
C 16:14, 1 March 2019 (UTC)
replyWithdrawn by operator. - there will be a replacement BRFA soon. -- Green C 17:18, 2 March 2019 (UTC) reply
Operator: GreenC ( talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)
Time filed: 01:14, Friday, December 28, 2018 ( UTC)
Function overview: Add {{
Unreferenced}}
and {{
No footnotes}}
to the tops of pages that have no references or are missing in-line footnotes.
Automatic, Supervised, or Manual: Automatic
Programming language(s): GNU Awk and BotWikiAwk framework
Source code available: Yes (TBU)
Links to relevant discussions (where appropriate):
Edit period(s): one time run
Estimated number of pages affected: ~ 130,000
Namespace(s): Mainspace articles
Exclusion compliant (Yes/No): Yes
Function details:
As background, members of New Page Patrollers ( WP:NPP) have caught up tagging the backlog of new pages. However there is still an older backlist of articles created since day 1 up to about 2012 which still contain many untagged articles. Estimates could be half a million or more untagged. A request was made on BOTREQ by a NPP member. I took a try at creating an algorithm to detect when a page could reasonably be tagged. Dry-run tests on 10,000 articles show it be successful. Discussion at Wikipedia_talk:WikiProject_Unreferenced_articles#List_of_unreferenced_articles shows support for an automated bot to help find articles needing attention and tag the pages.
Test results are available at User:GreenC/data/noref
The bot will start slowly and be fully supervised initially, running in batches, checking results.
What kinds of main namespace pages are you exempting? (e.g. Redirects, Disambiguation pages?) — xaosflux Talk 01:56, 28 December 2018 (UTC) reply
{{
EB1911}}
and dabs), the HTML tag <ol class="references">
, any that begin with "List of ..", "Index of .." or "<year> in .." --
Green
C 02:22, 28 December 2018 (UTC)
reply{{
no footnotes}}
. --
Green
C 13:41, 9 January 2019 (UTC)
replyApproved for trial (10 edits for each tag.). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Let's see it in action then. Headbomb { t · c · p · b} 16:45, 9 January 2019 (UTC) reply
@ GreenC: feel free to proceed with the trial whenever you want. Headbomb { t · c · p · b} 18:49, 13 January 2019 (UTC) reply
@ Headbomb: - please withdraw the BRFA. It is going to prove too controversial. Not that I agree (otherwise i wouldn't have made the BRFA) but there are evidently some old wounds in the community about tagging and this bot will reopen old battle scars. And there is more than 1 way to make use of the tool, it's purpose is to discover and identify potential candidate articles, information others can do with as they please. If there is support to re-open the BRFA it should go through VP or an RFC first. -- Green C 15:28, 14 January 2019 (UTC) reply
I have opened a discussion at Wikipedia:Village_pump_(proposals)#Bot to add Template:Unreferenced and Template:No footnotes to pages (single run) to ease or confirm GreenC's fears. Any watching this page may be interested in commenting there. Thanks all for comments. Sorry to get in the middle of the BRFA here. Cheers! Ajpolino ( talk) 17:20, 14 January 2019 (UTC)' reply
Hi. Would a page such as List of United States Supreme Court cases, volume 586 be tagged by this bot?
I'm not convinced that tagging pages such as
Grand Ducal Highness with {{
no footnotes}}
is particularly helpful, but shrug. If it's a one-time run, people can presumably just remove the ugly tags if they don't like them. --
MZMcBride (
talk) 01:41, 15 January 2019 (UTC)
reply
{{
no footnotes}}
assuming there is consensus. In this page,
User:GreenC/data/noref/14001-15000 it shows three independent bot algorithms. Which algorithm(s) the bot deploys is up to the community. It can do the first, or all three, or some combo. By default it will do all three, but if there is concern about the two {{
no footnotes}}
algos one or both could be dropped. --
Green
C 02:52, 15 January 2019 (UTC)
replyThe bot doesn't have support for a parenthetical referencing either bare parentheticals or ones generated by {{ harv}}. It lists Nummer 5 as an article to be tagged as type 2 {{ no footnotes}} though the article has inline citations generated by {{ harv}} and {{ harvnb}}. It also lists Fossilized affixes in Austronesian languages to be tagged as type 2 {{ no footnotes}} though it uses parenthetical citations with page numbers. Wugapodes [thɑk] [ˈkan.ˌʧɹɪbz] 08:25, 17 January 2019 (UTC) reply
Trial complete. - 20 edits at Special:Contributions/GreenC_bot on January 17 ("via noref bot") -- Green C 20:11, 19 January 2019 (UTC) reply
pageText = pageText.replace(/^\s*(?:((?:\s*\{\{\s*(?:about|correct title|dablink|distinguish|for|other\s?(?:hurricaneuses|people|persons|places|uses(?:of)?)|redirect(?:-acronym)?|see\s?(?:also|wiktionary)|selfref|the)\d*\s*(\|(?:\{\{[^{}]*\}\}|[^{}])*)?\}\})+(?:\s*\n)?)\s*)?/i, "$1" + tagText);
(source: Twinkle)
SD0001 (
talk) 15:46, 25 January 2019 (UTC)
reply
A user has requested the attention of a member of the
Bot Approvals Group. Once assistance has been rendered, please deactivate this tag by replacing it with {{
t|BAG assistance needed}}
.
.
The RfC seems to have run its course (over 30 days). There are at least 4 ways this bot can run, from ultra-conservative to regular-conservative, so it need not be a black and white decision. There is currently some sort of majority that wants the bot to run in some form and their voices will hopefully not be ignored. I also note, many of the opposer arguments are based in FUD, as the bot writer I can assert most of those claims are either technically incorrect are very much on the margins based on real-world testing results. --
Green
C 16:52, 24 February 2019 (UTC)
reply
{{
unreferenced}}
which strongly suggests there is consensus for tagging, so that is not the "contentious" issue it is something else. It's not accuracy, because the bot is even more accurate than human taggers evidently. It's more discriminate than manual tagging. It's not volume because the bot will be limited in total edits it won't go hog wild or over-tag pages. The only thing left is an opinion against tagging generally, which is a minority position on Wikipedia. --
Green
C 23:33, 25 February 2019 (UTC)
reply
{{
unreferenced}}
to categorize the tagged articles separately. If it's possible to do this while also having them added to
Category:All articles lacking sources (which the unreferenced tag currently does), I think that would be most desirable since it would facilitate searching through all unreferenced articles (which would be possible, but a slight pain in the butt if a chunk of unreferenced articles are in some different category). Thanks all for your help in moving this along! I'll get out of your hair now and promise not to start any further trouble. Thoughts on moving forward?
Ajpolino (
talk) 04:13, 1 March 2019 (UTC)
reply
{{
unreferenced}}
or a fork of that template just for this bot. It's not worth introducing all those forking or one-off special case complications that no one will remember soon enough and creates complications for those clearing out categories. Either the bot is working correctly or it's not, and if not, then it will stop. It won't be full speed and walk away, it will be carefully monitored just like every other bot I run. If folks just want a list of articles the bot tagged, that can be done easily by searching on the bot's unique edit-comment string (I can leave instructions on the bot page).|source=
parameter to {{
unreferenced}}
that adds the article to a
per-source category is not a big deal. The majority of instances of the template won't make use of it, but if there is a need for it elsewhere (another bot, another task for this bot, some kind of categorization for the NPP, etc.) it's there and can be used. The category created would typically be a sub-cat of the relevant main maintenance category. It's a bit excessive for the (lack of) risk this task entails, but it's neither excessively onerous nor entirely wasted. *shrug* {{
unreferenced}}
, contingent only on it meeting a reasonable rate of false positives in an extended trial. So get |source=
added to {{
unreferenced}}
; ask BAG for a couple hundred trial edits; and post a link to the diffs at VP/P for review. --
Xover (
talk) 07:17, 1 March 2019 (UTC) Edited: --
Xover (
talk) 14:28, 1 March 2019 (UTC)
reply
{{
Unreferenced}}
has about 220k instances. So an increase of maybe 10% is not a big deal. If you want to implement it in the template I'll add it to the code but it's a needless complication IMO that will soon be forgotten and unused. Now, further VP/P discussions? That could throw this BRFA off the rails and leave the BAG operator uncertain with how to proceed. This is how it should work: The RfC closed in support of the bot. We return to BRFA and the BAG operator takes over. They approve (more) trial edits. Anyone can monitor the BRFA and trials and make comments for an extended period. The BAG operator then ultimately decides to accept or reject the bot based on community input and technical performance. It's always been done this way. The RFC says "notify the community once they have finished the bot's trial", BRFA is "the community" and this BRFA is well advertised and if someone wants to post more notifications elsewhere they are welcome to do so, but the RFC doesn't change the normal BRFA process. Doing so undermines that process and makes it hard for BAG operators to close it out when there are open, unresolved or disputed discussions happening elsewhere. --
Green
C 16:14, 1 March 2019 (UTC)
replyWithdrawn by operator. - there will be a replacement BRFA soon. -- Green C 17:18, 2 March 2019 (UTC) reply