But it doesn't look like there is any obvious technical solution - at least, apparently no one has offered any that are agreeable to everyone, at which point someone could actually add the associated price tag.
More importantly, isn't the real question whether Wikipedia's volunteer base is interested and able to use more complex solutions **to any large extent**? If not, then - at least for the next couple of years, or at least until a good technical solution presents itself, a better approach might well be an automated pass at existing graphs, to convert them to images that don't carry security risks. (Store the snipets of code and data so that they are can be used in the future, if the relevant one ever arrives.) -- John Broughton (♫♫) 23:08, 29 March 2024 (UTC)
it is now possible to load a specific gadget when a specific template is used in a page. This opens the door (or perhaps a window?) to interactive content using JavaScript. See for example this article in the Spanish Wikipedia < https://es.wikipedia.org/wiki/Juego_de_la_vida> for an interactive instance of Conway's Game of Life, and scroll down for more instances!"
The continued failure to re-introduce graphs is the latest, most visible instance of Wikimedia's complete inability to do software projects. There are plenty of talented developers who work there, but for some reason the foundation can't project manage their way out of a paper bag. "We plan to, in a couple months, release the date for when we'll have a roadmap for doing the development" is a deranged statement for any software product feature. They could have implemented non-interactive graphs in a month with a single developer. Heck, it's been so long now they could have implemented interactive graphs from scratch. I get that this isn't a priority for them, but I can't help but notice that pretty much every single feature they work on has absurdly long timelines to produce anything. -- Pres N 02:23, 30 March 2024 (UTC)
A somewhat related discussion is ongoing at Wikipedia:Village_pump_(WMF)#Proposal:_WMF_should_hire_a_full-time_developer_to_do_basic_maintenance_on_MediaWiki. Regards, HaeB ( talk) 02:53, 30 March 2024 (UTC)
we have all the downsides of a company and all the downsides of a volunteer foss project at the same time- that's an interesting thought, do you want to elaborate on that? (I know you have been a longtime observer of such matters, so maybe you already wrote about this elsewhere - feel free post a link.) Regards, HaeB ( talk) 21:08, 30 March 2024 (UTC)
I find the comparison to WolframAlpha very silly. WP:NOT is among our largest and most referenced policies because, as it turns out, including too much information in a reference work makes it less useful. We have a lane and we should stick to it. Mach61 04:47, 30 March 2024 (UTC)
I agree with Tisza and TheDJ; we should be focusing on getting the basic graph functionality running rather than trying to devise interactive solutions. While the fanciful visions of interactive content may sound all good and nice, fact is that we've got something broken, and the vast majority of use cases don't currently require interactivity. So we should expend the energy on fixing the broken thing and then the interactive stuff can be added later. And flashy graphs with complex interactive features are better served by other sites anyway; we shouldn't try to be everything to everyone. ― novov (t c) 05:13, 30 March 2024 (UTC)
First, HaeB, thanks for this write-up, I really appreciated getting up to speed without having to read all the mailing list discussions. In my view, this comes down to two lines:
the path forward will require a substantial investment – one that we have not yet started given the other priorities we’ve been working on
and
Still, on English Wikipedia, only 19,160 pages were affected.
The WMF isn't going to spend millions of dollars to fix something that's only a problem on 20k enwiki pages. Bottom line is bottom line, unfortunately: graphs aren't popular enough to care about, by and large. Personally, I think they're great and could be profitably used on millions of pages, but you can't argue with the fact that they just aren't very popular, not very widely used, and thus low-priority when compared with other features that are more widely used. Levivich ( talk) 16:41, 30 March 2024 (UTC)
I think I would be willing and able to develop a Lua module that replaces the Graph extension for the most common use cases (bar charts, line graphs and maybe even pie charts). However, the Wikimedia Technology Fund has been marked as Permanently on hold with no explanation given (at least that I know of). Perhaps this year I'll find the time and energy to attempt it as a volunteer, but it would really help to get some compensation for it, since it will require a lot of time and effort that I could otherwise spend in remunerated work, or just reading a book. Sophivorus ( talk) 23:01, 30 March 2024 (UTC)
Hello everyone. I'm Marshall Miller; I'm a Senior Director of Product at WMF (I was quoted in the article above), and I work with teams that build features for the reading and editing experiences (e.g. Discussion Tools or Night Mode). Thank you all for thinking about and discussing the graphs issue. I know that it is frustrating that this remains unresolved -- it has been a deceptively complex problem, involving issues around security and scalability. I just wanted to post here saying that I am following this discussion, and that I have been working with colleagues to propose a path forward for graphs. We'll be resuming the discussion and updates on Meta, and I will also post a link here once there is a new update over on that wiki. -- MMiller (WMF) ( talk) 05:06, 31 March 2024 (UTC)
I agree with the frustration, but.... Those of you who consult resources hosted by the British Library are aware of the cyberattack that brought down their system in October. Some of the functionality is still not there and won't come back until the architecture is redesigned (see the report Learning Lessons From The Cyber-Attack). Security threats are real. I suspect most people would not want Wikimedia projects to be hacked and all off its content erased in such a way where it was not retrievable. Perhaps the best way forward is to ask WMF to devote more resources to security and replacing those parts of the infrastructure which are at risk. - kosboot ( talk) 14:49, 1 April 2024 (UTC)
I find it hilarious that they had enough time, effort, and energy to debut the unwanted vector 2022 mandatory "upgrade", but can't (or won't) get this fixed. Incompetence be thy name, foundation. TomStar81 ( Talk) 21:12, 1 April 2024 (UTC)
Another Wikipedian reported that "In ruwiki, interactive Lua-based graphs are used in more than 26000 articles about settlements and administrative units through https://ru.wikipedia.org/wiki/Module:Statistical. The primary uses of ruwiki Module:Statistical are in templates like ruwiki Шаблон:Население ("Template:Population"), which for years has had the ability to format population data as either an interactive
<graph>
, or a pure-HTML-based bar chart. When Vega went offline, instead of displaying the "can't do that" message, they simply updated their template so that all requests for the graph version instead get the HTML bar chart. It's
less satisfying (note: discussion in Russian) than the interactive graphs, but it conveys most of the same data.It is worth noting, I think, that parts of the problem have been solved. For example, the much-discussed Template:OSM_Location_map, used by 5,600 pages on the English Wikipedia, is functioning again since 11 March 2024, with some newly added features. See the Return to service article. On the other hand, those 5,600 pages were not included in the count of 19,160 -- a count that underestimates the true impact of the problem, leaving out all the templates that are indirectly rendered inoperable. Renerpho ( talk) 07:46, 5 April 2024 (UTC)
(Those numbers likely already reflect the manual removal of broken graphs from many pages.)But it goes on to say:
In a more detailed 2020 analysis, volunteer developer User:Bawolff had found that "the graph extension is used on 26,238 pages [on English Wikipedia]. However, most of these are in non-content namespaces, from a template that generates a graph of page views for a specific page ( w:Template:PageViews graph). There are 4,140 pages on en.wikipedia.org in the main namespace that use graphs."
<noinclude>
documentation section, so that it's only applied to the template itself, not its consumers.)underestimates the true impact of the problem, leaving out all the templates that are indirectly rendered inoperable. The tracking category is populated by MediaWiki itself, and any use of
<graph>
in a template, no matter how indirect, would cause every page that transcludes the template to be counted. It's only the pages where the transclusion was removed (as the article notes), or where the transcluded template was modified to eliminate its use of graphs (like {{
OSM Location map}}
), that wouldn't be counted.
FeRDNYC (
talk) 17:00, 6 April 2024 (UTC)-- This subject demonstrates Wikimedians' propensity to blather on, building towering walls of text in place of taking action. The way forward seems blindingly obvious: implement a safe static graph facility while separating off the issue of whether and how to provide dynamic and/or interactive graphics safely. The most efficient way is likely to be to subset the Vega framework (or just syntax) to those constructs that can only produce safe, static code. The point is to actually act on that rather than talk about that. -- R. S. Shaw ( talk) 23:09, 8 April 2024 (UTC)
How many of the 4,000 articles that use broken graphs use only pie charts or bar charts? There are working templates for both of those, [1] [2] so long as you don't miss the hover-over feature. And you shouldn't miss it. Pretty sure most users just want a simple, easy-to-read image in most cases - before they spend millions on making something, they should check if people would actually find it more useful. Wizmut ( talk) 03:44, 10 April 2024 (UTC)
Hello everyone. I wanted to follow up here on my earlier comment above. I just posted an update to the Graph project page laying out a proposal in which WMF would build a new extension for making graphs. We've come to this proposal after talking to community members over the past weeks, analyzing data, and thinking through architecture with staff. This would be a substantial amount of work, and I hope that community members can weigh in on whether this seems like the right approach and to help us plan the project. Please join the discussion on the talk page! -- MMiller (WMF) ( talk) 23:15, 10 April 2024 (UTC)
We at WikiProjectMed have funded the building of a template which first shows a still graph from Commons. And than once consent is given by the reader shows the live version piped in from Our World in Data. It is live at eu:Haurdunaldi#Nerabezaroa. Thanks to User:Bawolff who did the programing. We still need to figure out translation aspects. Doc James ( talk · contribs · email) 21:40, 16 April 2024 (UTC)
But it doesn't look like there is any obvious technical solution - at least, apparently no one has offered any that are agreeable to everyone, at which point someone could actually add the associated price tag.
More importantly, isn't the real question whether Wikipedia's volunteer base is interested and able to use more complex solutions **to any large extent**? If not, then - at least for the next couple of years, or at least until a good technical solution presents itself, a better approach might well be an automated pass at existing graphs, to convert them to images that don't carry security risks. (Store the snipets of code and data so that they are can be used in the future, if the relevant one ever arrives.) -- John Broughton (♫♫) 23:08, 29 March 2024 (UTC)
it is now possible to load a specific gadget when a specific template is used in a page. This opens the door (or perhaps a window?) to interactive content using JavaScript. See for example this article in the Spanish Wikipedia < https://es.wikipedia.org/wiki/Juego_de_la_vida> for an interactive instance of Conway's Game of Life, and scroll down for more instances!"
The continued failure to re-introduce graphs is the latest, most visible instance of Wikimedia's complete inability to do software projects. There are plenty of talented developers who work there, but for some reason the foundation can't project manage their way out of a paper bag. "We plan to, in a couple months, release the date for when we'll have a roadmap for doing the development" is a deranged statement for any software product feature. They could have implemented non-interactive graphs in a month with a single developer. Heck, it's been so long now they could have implemented interactive graphs from scratch. I get that this isn't a priority for them, but I can't help but notice that pretty much every single feature they work on has absurdly long timelines to produce anything. -- Pres N 02:23, 30 March 2024 (UTC)
A somewhat related discussion is ongoing at Wikipedia:Village_pump_(WMF)#Proposal:_WMF_should_hire_a_full-time_developer_to_do_basic_maintenance_on_MediaWiki. Regards, HaeB ( talk) 02:53, 30 March 2024 (UTC)
we have all the downsides of a company and all the downsides of a volunteer foss project at the same time- that's an interesting thought, do you want to elaborate on that? (I know you have been a longtime observer of such matters, so maybe you already wrote about this elsewhere - feel free post a link.) Regards, HaeB ( talk) 21:08, 30 March 2024 (UTC)
I find the comparison to WolframAlpha very silly. WP:NOT is among our largest and most referenced policies because, as it turns out, including too much information in a reference work makes it less useful. We have a lane and we should stick to it. Mach61 04:47, 30 March 2024 (UTC)
I agree with Tisza and TheDJ; we should be focusing on getting the basic graph functionality running rather than trying to devise interactive solutions. While the fanciful visions of interactive content may sound all good and nice, fact is that we've got something broken, and the vast majority of use cases don't currently require interactivity. So we should expend the energy on fixing the broken thing and then the interactive stuff can be added later. And flashy graphs with complex interactive features are better served by other sites anyway; we shouldn't try to be everything to everyone. ― novov (t c) 05:13, 30 March 2024 (UTC)
First, HaeB, thanks for this write-up, I really appreciated getting up to speed without having to read all the mailing list discussions. In my view, this comes down to two lines:
the path forward will require a substantial investment – one that we have not yet started given the other priorities we’ve been working on
and
Still, on English Wikipedia, only 19,160 pages were affected.
The WMF isn't going to spend millions of dollars to fix something that's only a problem on 20k enwiki pages. Bottom line is bottom line, unfortunately: graphs aren't popular enough to care about, by and large. Personally, I think they're great and could be profitably used on millions of pages, but you can't argue with the fact that they just aren't very popular, not very widely used, and thus low-priority when compared with other features that are more widely used. Levivich ( talk) 16:41, 30 March 2024 (UTC)
I think I would be willing and able to develop a Lua module that replaces the Graph extension for the most common use cases (bar charts, line graphs and maybe even pie charts). However, the Wikimedia Technology Fund has been marked as Permanently on hold with no explanation given (at least that I know of). Perhaps this year I'll find the time and energy to attempt it as a volunteer, but it would really help to get some compensation for it, since it will require a lot of time and effort that I could otherwise spend in remunerated work, or just reading a book. Sophivorus ( talk) 23:01, 30 March 2024 (UTC)
Hello everyone. I'm Marshall Miller; I'm a Senior Director of Product at WMF (I was quoted in the article above), and I work with teams that build features for the reading and editing experiences (e.g. Discussion Tools or Night Mode). Thank you all for thinking about and discussing the graphs issue. I know that it is frustrating that this remains unresolved -- it has been a deceptively complex problem, involving issues around security and scalability. I just wanted to post here saying that I am following this discussion, and that I have been working with colleagues to propose a path forward for graphs. We'll be resuming the discussion and updates on Meta, and I will also post a link here once there is a new update over on that wiki. -- MMiller (WMF) ( talk) 05:06, 31 March 2024 (UTC)
I agree with the frustration, but.... Those of you who consult resources hosted by the British Library are aware of the cyberattack that brought down their system in October. Some of the functionality is still not there and won't come back until the architecture is redesigned (see the report Learning Lessons From The Cyber-Attack). Security threats are real. I suspect most people would not want Wikimedia projects to be hacked and all off its content erased in such a way where it was not retrievable. Perhaps the best way forward is to ask WMF to devote more resources to security and replacing those parts of the infrastructure which are at risk. - kosboot ( talk) 14:49, 1 April 2024 (UTC)
I find it hilarious that they had enough time, effort, and energy to debut the unwanted vector 2022 mandatory "upgrade", but can't (or won't) get this fixed. Incompetence be thy name, foundation. TomStar81 ( Talk) 21:12, 1 April 2024 (UTC)
Another Wikipedian reported that "In ruwiki, interactive Lua-based graphs are used in more than 26000 articles about settlements and administrative units through https://ru.wikipedia.org/wiki/Module:Statistical. The primary uses of ruwiki Module:Statistical are in templates like ruwiki Шаблон:Население ("Template:Population"), which for years has had the ability to format population data as either an interactive
<graph>
, or a pure-HTML-based bar chart. When Vega went offline, instead of displaying the "can't do that" message, they simply updated their template so that all requests for the graph version instead get the HTML bar chart. It's
less satisfying (note: discussion in Russian) than the interactive graphs, but it conveys most of the same data.It is worth noting, I think, that parts of the problem have been solved. For example, the much-discussed Template:OSM_Location_map, used by 5,600 pages on the English Wikipedia, is functioning again since 11 March 2024, with some newly added features. See the Return to service article. On the other hand, those 5,600 pages were not included in the count of 19,160 -- a count that underestimates the true impact of the problem, leaving out all the templates that are indirectly rendered inoperable. Renerpho ( talk) 07:46, 5 April 2024 (UTC)
(Those numbers likely already reflect the manual removal of broken graphs from many pages.)But it goes on to say:
In a more detailed 2020 analysis, volunteer developer User:Bawolff had found that "the graph extension is used on 26,238 pages [on English Wikipedia]. However, most of these are in non-content namespaces, from a template that generates a graph of page views for a specific page ( w:Template:PageViews graph). There are 4,140 pages on en.wikipedia.org in the main namespace that use graphs."
<noinclude>
documentation section, so that it's only applied to the template itself, not its consumers.)underestimates the true impact of the problem, leaving out all the templates that are indirectly rendered inoperable. The tracking category is populated by MediaWiki itself, and any use of
<graph>
in a template, no matter how indirect, would cause every page that transcludes the template to be counted. It's only the pages where the transclusion was removed (as the article notes), or where the transcluded template was modified to eliminate its use of graphs (like {{
OSM Location map}}
), that wouldn't be counted.
FeRDNYC (
talk) 17:00, 6 April 2024 (UTC)-- This subject demonstrates Wikimedians' propensity to blather on, building towering walls of text in place of taking action. The way forward seems blindingly obvious: implement a safe static graph facility while separating off the issue of whether and how to provide dynamic and/or interactive graphics safely. The most efficient way is likely to be to subset the Vega framework (or just syntax) to those constructs that can only produce safe, static code. The point is to actually act on that rather than talk about that. -- R. S. Shaw ( talk) 23:09, 8 April 2024 (UTC)
How many of the 4,000 articles that use broken graphs use only pie charts or bar charts? There are working templates for both of those, [1] [2] so long as you don't miss the hover-over feature. And you shouldn't miss it. Pretty sure most users just want a simple, easy-to-read image in most cases - before they spend millions on making something, they should check if people would actually find it more useful. Wizmut ( talk) 03:44, 10 April 2024 (UTC)
Hello everyone. I wanted to follow up here on my earlier comment above. I just posted an update to the Graph project page laying out a proposal in which WMF would build a new extension for making graphs. We've come to this proposal after talking to community members over the past weeks, analyzing data, and thinking through architecture with staff. This would be a substantial amount of work, and I hope that community members can weigh in on whether this seems like the right approach and to help us plan the project. Please join the discussion on the talk page! -- MMiller (WMF) ( talk) 23:15, 10 April 2024 (UTC)
We at WikiProjectMed have funded the building of a template which first shows a still graph from Commons. And than once consent is given by the reader shows the live version piped in from Our World in Data. It is live at eu:Haurdunaldi#Nerabezaroa. Thanks to User:Bawolff who did the programing. We still need to figure out translation aspects. Doc James ( talk · contribs · email) 21:40, 16 April 2024 (UTC)