 | This is an
essay. It contains the advice or opinions of one or more Wikipedia contributors. This page is not an encyclopedia article, nor is it one of
Wikipedia's policies or guidelines, as it has not been
thoroughly vetted by the community. Some essays represent widespread norms; others only represent minority viewpoints. |
 | This page in a nutshell: Pages seem to relink in groups of 500 but some get stuck, must be null-edited to relink. Design Lua modules/templates for less reformatting. |
This essay describes issues about re-rendering or relinking pages when a template or Lua script module is changed.
Even weeks after a change to a template or Lua module has "completed" the relinking process, there might be several pages still stuck with old links, and could be cleared with null-edits. For years, it's as though a database read/ write lock among several pages had caused a "relink conflict" and the pages could not be relinked when whichever wp:job_queue processed the reformatting job ( wp:JQ says Wikipedia runs 3 queues at once). Also, according to wp:JQ, when more than 1,000 pages have the "invalidated cache" to trigger reformat, they are grouped as 500-pages-per-job, and checking WhatLinksHere tends to show progress as sporadic groups by hundreds of pages, either linked or delinked, in "fits and starts" where even the whole process can get delayed during the multi-day ordeal. Yet, always be prepared to null-edit the leftover stragglers when all jobs have finished relinking within the 3(?) English Wikipedia job_queues.
It would be great if a database write-lock timeout would re-queue each stuck page, to be reformatted as another job, re-added to a queue, but perhaps that requeue decision could not be handled down at the level of relinking each template used inside a page.
With the Lua script megatemplates, there are even more cases where changing a Lua megamodule would trigger formatting a "million" pages which formerly were reformatted as smaller groups of separate (near-redundant) fork templates, such as the 24 wp:CS1 cite-template forks now forced to all reformat 2.2 million pages for any Lua edit to the giant Module:Citation/CS1, or the new Module:Convert which would bind all the various 20,000-page separate Convert subtemplates to always trigger reformatting all 554,000 pages for any Lua edit made to Module:Convert. Perhaps Lua has become used in a "pennywise and pound-foolish" architecture, running 6x-13x (or 50x) times faster than markup inside each page, but triggering 100x-1000x times as many page reformats as the prior smaller template forks. Many Lua modules should use forced multi-fork, split designs (as multiple Lua modules) to trigger less system-wide reformatting of all pages, where more Lua sub-modules would be used in only thousands of pages, rather than millions.
- wp:Wikimedia Foundation error - error screen displayed at timeout of page reformat
| This is an
essay. It contains the advice or opinions of one or more Wikipedia contributors. This page is not an encyclopedia article, nor is it one of
Wikipedia's policies or guidelines, as it has not been
thoroughly vetted by the community. Some essays represent widespread norms; others only represent minority viewpoints. |
| This page in a nutshell: Pages seem to relink in groups of 500 but some get stuck, must be null-edited to relink. Design Lua modules/templates for less reformatting. |
This essay describes issues about re-rendering or relinking pages when a template or Lua script module is changed.
Even weeks after a change to a template or Lua module has "completed" the relinking process, there might be several pages still stuck with old links, and could be cleared with null-edits. For years, it's as though a database read/ write lock among several pages had caused a "relink conflict" and the pages could not be relinked when whichever wp:job_queue processed the reformatting job ( wp:JQ says Wikipedia runs 3 queues at once). Also, according to wp:JQ, when more than 1,000 pages have the "invalidated cache" to trigger reformat, they are grouped as 500-pages-per-job, and checking WhatLinksHere tends to show progress as sporadic groups by hundreds of pages, either linked or delinked, in "fits and starts" where even the whole process can get delayed during the multi-day ordeal. Yet, always be prepared to null-edit the leftover stragglers when all jobs have finished relinking within the 3(?) English Wikipedia job_queues.
It would be great if a database write-lock timeout would re-queue each stuck page, to be reformatted as another job, re-added to a queue, but perhaps that requeue decision could not be handled down at the level of relinking each template used inside a page.
With the Lua script megatemplates, there are even more cases where changing a Lua megamodule would trigger formatting a "million" pages which formerly were reformatted as smaller groups of separate (near-redundant) fork templates, such as the 24 wp:CS1 cite-template forks now forced to all reformat 2.2 million pages for any Lua edit to the giant Module:Citation/CS1, or the new Module:Convert which would bind all the various 20,000-page separate Convert subtemplates to always trigger reformatting all 554,000 pages for any Lua edit made to Module:Convert. Perhaps Lua has become used in a "pennywise and pound-foolish" architecture, running 6x-13x (or 50x) times faster than markup inside each page, but triggering 100x-1000x times as many page reformats as the prior smaller template forks. Many Lua modules should use forced multi-fork, split designs (as multiple Lua modules) to trigger less system-wide reformatting of all pages, where more Lua sub-modules would be used in only thousands of pages, rather than millions.
- wp:Wikimedia Foundation error - error screen displayed at timeout of page reformat