New to bots on Wikipedia? Read these primers!
- Approval process – How this discussion works
- Overview/ Policy – What bots are/What they can (or can't) do
- Dictionary – Explains bot-related jargon
Operator: BilledMammal ( talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)
Time filed: 08:51, Monday, July 8, 2024 ( UTC)
Function overview: Adjusts templates based on provided JSON configuration files. This request is limited to Template:Cite news and Template:Cite web, and is primarily intended to correct issues where the work or publisher is linked to the wrong target.
Automatic, Supervised, or Manual: Automatic
Programming language(s): Python
Source code available: Not currently
Links to relevant discussions (where appropriate):
Edit period(s): Initially, irregular one-off runs, with each held after significant expansions to the configuration file. Once most citations have been fixed I will open a request for continuous operation in a maintenance mode.
Estimated number of pages affected: Varies considerably based on configuration. This configuration, which applies to ten sources, will edit approximately 23,000. This configuration, which goes beyond correcting wrong links and also always inserts the correct link when one is missing, will edit approximately 450,000.
Namespace(s): Mainspace
Exclusion compliant (Yes/No): Yes
Function details: Adjusts parameters of Cite news and Cite web based on a configuration file. This configuration can be applied to any parameter, but the intent of this request is to apply it to the following:
- work
- publisher
- publication-place
- department
- agency
- url-access
It determines which change to apply based on current parameter field values. Any field or combination of fields can be used, but the intent of this request is to use the "url" field.
Adjustments can be specified as "always", "onEdit", or "never". When "always" is specified, if a change is identified as being desired for a parameter the article will be edited to implement it. When "onEdit" is specified, desirable changes are only implemented if we are already editing the page. This reduces the impact on watchlists by skipping articles that don't have high priority issues.
Configuration schema
|
|---|
{
"$schema": "
http://json-schema.org/draft-07/schema#",
"type": "array",
"items": {
"type": "object",
"properties": {
"includes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": "url"
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com", "www.bbc.co.uk"]
}
}
}
},
"description": "Lists conditions required to be met for this configuration to be applied to the template."
},
"excludes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": "url"
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com/sport", "www.bbc.co.uk/sport"]
}
}
},
"description": "Lists conditions that must not be met for this configuration to be applied to the template."
}
},
"patternProperties": {
"^[a-zA-Z0-9-]+$": {
"oneOf": [
{
"type": "array",
"description": "Named for the parameter, and defines what will be done with it. Used when there are multiple possible configurations for the parameter.",
"items": {
"$ref": "#/definitions/parameter-config"
}
},
{
"type": "object",
"description": "Named for the parameter, and defines what will be done with it. Used when there is only one possible configuration for the parameter.",
"$ref": "#/definitions/parameter-config"
}
]
}
}
},
"definitions": {
"parameter-config": {
"$schema": "
http://json-schema.org/draft-07/schema#",
"$id": "parameter-config",
"type": "object",
"properties": {
"includes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": ["url"]
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com", "www.bbc.co.uk"]
}
}
}
},
"description": "Lists conditions required to be met for this configuration to be applied to the parameter."
},
"excludes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": ["url"]
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com/sport", "www.bbc.co.uk/sport"]
}
}
}
},
"description": "Lists conditions that must not be met for this configuration to be applied to the parameter."
},
"link": {
"type": "string",
"description": "Where the parameter should normally link to",
"example": ["ABC News (Australia)"]
},
"wikitext": {
"type": "string",
"description": "What the wikitext of the parameter should normally be",
"example": ["
ABC News"]
},
"blacklist": {
"type": "array",
"items": {
"type": "string",
"example": ["ABC News (United States)", "ABC News"]
},
"description": "Links that will always be removed"
},
"greylist": {
"type": "array",
"items": {
"type": "string",
"example": ["Australian Broadcasting Corporation"]
},
"description": "Links that will only be removed when already editing the page. Used to prevent edits that would only fix issues we consider minor."
},
"whitelist": {
"type": "array",
"items": {
"type": "string",
"example": ["The Sunday Telegraph (Sydney)"]
},
"description": "Links that will never be removed. Used when we believe editors may have deliberately provided a non-standard value that we wish to respect."
},
"fixRedirects": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will replace redirects to the provided link with the provided link."
},
"fixDisplay": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will replace the currently displayed text with the displayed version of the provided Wikitext."
},
"fixOthers": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "always",
"description": "Specifies when we will replace links to pages that are neither redirects to the link nor on the provided lists."
},
"fixMissing": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will add a missing value"
},
"priority": {
"type": "integer",
"default": 5,
"description": "Provides a tie-breaker when multiple array objects meet the inclusion or exclusion criteria. Higher value is preferred. It is unspecified which configuration object is used when both have the same priority level.",
"minimum": 1
}
}
}
}
}
}
|
What it does to these parameters depends on the configuration. For example:
"work":
{
"link": "ABC News (Australia)",
"wikitext": "
ABC News",
"blacklist": ["ABC News (United States)", "ABC News"],
"greylist": ["Australian Broadcasting Corporation"],
"fixMissing": "onEdit",
"fixRedirects": "onEdit",
"fixOthers": "always"
}
Will ensure that the "work" parameter only links to ABC News (Australia). When it finds a link to a source other than ABC News (Australia), its redirects, or Australian Broadcasting Corporation, it will edit the article to correct that link.
When it encounters a redirect, or Australian Broadcasting Corporation, or a missing value, it will only correct those if it is already editing the article.
If we change "fixMissing" to "always", it would edit the article to insert the value.
"agency":
{
"includes":
[
{
"key": "agency",
"value": ["Reuters"]
}
],
"remove": "onEdit"
}
Will remove the agency field when it contains "Reuters". This is used to correct when the field has been incorrectly filled with the name of the publisher or work.
"department":
[
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/world/"]
}
],
"wikitext": "World"
},
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/world/reuters-next/"]
}
],
"wikitext": "Reuters Next",
"priority": 6
},
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/business/"]
}
],
"wikitext": "Business"
}
]
This fills in the department field based on the source url. If none of these are met then the department field is not filled.
The current configuration file will do the following:
-
ABC News (Australia)
- Set "work" to ABC News
- Set "publisher" to Australian Broadcasting Corporation
- Remove "publication-place"
- Remove "agency" when incorrect
-
The Daily Telegraph
- Set "work" to The Daily Telegraph
- Set "publisher" to Telegraph Media Group
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
-
Reuters
- Set "work" to Reuters
- Set "publisher" to Thomson Reuters
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
- Remove "agency" when incorrect
-
The New York Times
- Set "work" to The New York Times
- Set "url-access" to "limited"
- Remove "publisher"
- Remove "publication-place"
-
BBC News
- Set "work" to BBC News
- Remove "publisher"
- Remove "publication-place"
- Set "department" when it can be determined
-
BBC Sport
- Set "work" to BBC Sport
- Remove "publisher"
- Remove "publication-place"
-
The Guardian
- Set "work" to The Guardian
- Remove "publisher"
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
-
The Guardian (Swan Hill)
- Set "work" to The Guardian
-
The Daily Telegraph (Sydney)
- Set "work" to The Daily Telegraph
- Set "publisher" to News Corp Australia
- Remove "publication-place"
-
ABC News (United States)
- Set "work" to ABC News
- Set "publisher" to American Broadcasting Company
- Remove "publication-place"
The intent is that the community will expand the configuration file, increasing the number of citations it can fix.
Example of template replacements
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
When editing a template, to improve readability it will also apply a consistent format and naming convention. This involves converting parameters away from aliases to their primary values, and placing the parameters into the following order:
Order
|
|---|
|
Discussion
New to bots on Wikipedia? Read these primers!
- Approval process – How this discussion works
- Overview/ Policy – What bots are/What they can (or can't) do
- Dictionary – Explains bot-related jargon
Operator: BilledMammal ( talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)
Time filed: 08:51, Monday, July 8, 2024 ( UTC)
Function overview: Adjusts templates based on provided JSON configuration files. This request is limited to Template:Cite news and Template:Cite web, and is primarily intended to correct issues where the work or publisher is linked to the wrong target.
Automatic, Supervised, or Manual: Automatic
Programming language(s): Python
Source code available: Not currently
Links to relevant discussions (where appropriate):
Edit period(s): Initially, irregular one-off runs, with each held after significant expansions to the configuration file. Once most citations have been fixed I will open a request for continuous operation in a maintenance mode.
Estimated number of pages affected: Varies considerably based on configuration. This configuration, which applies to ten sources, will edit approximately 23,000. This configuration, which goes beyond correcting wrong links and also always inserts the correct link when one is missing, will edit approximately 450,000.
Namespace(s): Mainspace
Exclusion compliant (Yes/No): Yes
Function details: Adjusts parameters of Cite news and Cite web based on a configuration file. This configuration can be applied to any parameter, but the intent of this request is to apply it to the following:
- work
- publisher
- publication-place
- department
- agency
- url-access
It determines which change to apply based on current parameter field values. Any field or combination of fields can be used, but the intent of this request is to use the "url" field.
Adjustments can be specified as "always", "onEdit", or "never". When "always" is specified, if a change is identified as being desired for a parameter the article will be edited to implement it. When "onEdit" is specified, desirable changes are only implemented if we are already editing the page. This reduces the impact on watchlists by skipping articles that don't have high priority issues.
Configuration schema
|
|---|
{
"$schema": "
http://json-schema.org/draft-07/schema#",
"type": "array",
"items": {
"type": "object",
"properties": {
"includes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": "url"
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com", "www.bbc.co.uk"]
}
}
}
},
"description": "Lists conditions required to be met for this configuration to be applied to the template."
},
"excludes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": "url"
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com/sport", "www.bbc.co.uk/sport"]
}
}
},
"description": "Lists conditions that must not be met for this configuration to be applied to the template."
}
},
"patternProperties": {
"^[a-zA-Z0-9-]+$": {
"oneOf": [
{
"type": "array",
"description": "Named for the parameter, and defines what will be done with it. Used when there are multiple possible configurations for the parameter.",
"items": {
"$ref": "#/definitions/parameter-config"
}
},
{
"type": "object",
"description": "Named for the parameter, and defines what will be done with it. Used when there is only one possible configuration for the parameter.",
"$ref": "#/definitions/parameter-config"
}
]
}
}
},
"definitions": {
"parameter-config": {
"$schema": "
http://json-schema.org/draft-07/schema#",
"$id": "parameter-config",
"type": "object",
"properties": {
"includes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": ["url"]
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com", "www.bbc.co.uk"]
}
}
}
},
"description": "Lists conditions required to be met for this configuration to be applied to the parameter."
},
"excludes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"example": ["url"]
},
"value": {
"type": "array",
"items": {
"type": "string",
"example": ["www.bbc.com/sport", "www.bbc.co.uk/sport"]
}
}
}
},
"description": "Lists conditions that must not be met for this configuration to be applied to the parameter."
},
"link": {
"type": "string",
"description": "Where the parameter should normally link to",
"example": ["ABC News (Australia)"]
},
"wikitext": {
"type": "string",
"description": "What the wikitext of the parameter should normally be",
"example": ["
ABC News"]
},
"blacklist": {
"type": "array",
"items": {
"type": "string",
"example": ["ABC News (United States)", "ABC News"]
},
"description": "Links that will always be removed"
},
"greylist": {
"type": "array",
"items": {
"type": "string",
"example": ["Australian Broadcasting Corporation"]
},
"description": "Links that will only be removed when already editing the page. Used to prevent edits that would only fix issues we consider minor."
},
"whitelist": {
"type": "array",
"items": {
"type": "string",
"example": ["The Sunday Telegraph (Sydney)"]
},
"description": "Links that will never be removed. Used when we believe editors may have deliberately provided a non-standard value that we wish to respect."
},
"fixRedirects": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will replace redirects to the provided link with the provided link."
},
"fixDisplay": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will replace the currently displayed text with the displayed version of the provided Wikitext."
},
"fixOthers": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "always",
"description": "Specifies when we will replace links to pages that are neither redirects to the link nor on the provided lists."
},
"fixMissing": {
"type": "string",
"enum": ["always", "onEdit", "never"],
"default": "onEdit",
"description": "Specifies when we will add a missing value"
},
"priority": {
"type": "integer",
"default": 5,
"description": "Provides a tie-breaker when multiple array objects meet the inclusion or exclusion criteria. Higher value is preferred. It is unspecified which configuration object is used when both have the same priority level.",
"minimum": 1
}
}
}
}
}
}
|
What it does to these parameters depends on the configuration. For example:
"work":
{
"link": "ABC News (Australia)",
"wikitext": "
ABC News",
"blacklist": ["ABC News (United States)", "ABC News"],
"greylist": ["Australian Broadcasting Corporation"],
"fixMissing": "onEdit",
"fixRedirects": "onEdit",
"fixOthers": "always"
}
Will ensure that the "work" parameter only links to ABC News (Australia). When it finds a link to a source other than ABC News (Australia), its redirects, or Australian Broadcasting Corporation, it will edit the article to correct that link.
When it encounters a redirect, or Australian Broadcasting Corporation, or a missing value, it will only correct those if it is already editing the article.
If we change "fixMissing" to "always", it would edit the article to insert the value.
"agency":
{
"includes":
[
{
"key": "agency",
"value": ["Reuters"]
}
],
"remove": "onEdit"
}
Will remove the agency field when it contains "Reuters". This is used to correct when the field has been incorrectly filled with the name of the publisher or work.
"department":
[
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/world/"]
}
],
"wikitext": "World"
},
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/world/reuters-next/"]
}
],
"wikitext": "Reuters Next",
"priority": 6
},
{
"includes":
[
{
"key": "url",
"value": ["reuters.com/business/"]
}
],
"wikitext": "Business"
}
]
This fills in the department field based on the source url. If none of these are met then the department field is not filled.
The current configuration file will do the following:
-
ABC News (Australia)
- Set "work" to ABC News
- Set "publisher" to Australian Broadcasting Corporation
- Remove "publication-place"
- Remove "agency" when incorrect
-
The Daily Telegraph
- Set "work" to The Daily Telegraph
- Set "publisher" to Telegraph Media Group
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
-
Reuters
- Set "work" to Reuters
- Set "publisher" to Thomson Reuters
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
- Remove "agency" when incorrect
-
The New York Times
- Set "work" to The New York Times
- Set "url-access" to "limited"
- Remove "publisher"
- Remove "publication-place"
-
BBC News
- Set "work" to BBC News
- Remove "publisher"
- Remove "publication-place"
- Set "department" when it can be determined
-
BBC Sport
- Set "work" to BBC Sport
- Remove "publisher"
- Remove "publication-place"
-
The Guardian
- Set "work" to The Guardian
- Remove "publisher"
- Set "publication-place" to "London, United Kingdom"
- Set "department" when it can be determined
-
The Guardian (Swan Hill)
- Set "work" to The Guardian
-
The Daily Telegraph (Sydney)
- Set "work" to The Daily Telegraph
- Set "publisher" to News Corp Australia
- Remove "publication-place"
-
ABC News (United States)
- Set "work" to ABC News
- Set "publisher" to American Broadcasting Company
- Remove "publication-place"
The intent is that the community will expand the configuration file, increasing the number of citations it can fix.
Example of template replacements
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
When editing a template, to improve readability it will also apply a consistent format and naming convention. This involves converting parameters away from aliases to their primary values, and placing the parameters into the following order:
Order
|
|---|
|