This article was reviewed by member(s) of WikiProject Articles for creation. The project works to allow users to contribute quality articles and media files to the encyclopedia and track their progress as they are developed. To participate, please visit the
project page for more information.Articles for creationWikipedia:WikiProject Articles for creationTemplate:WikiProject Articles for creationAfC articles
This article is within the scope of WikiProject Computer science, a collaborative effort to improve the coverage of
Computer science related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join
the discussion and see a list of open tasks.Computer scienceWikipedia:WikiProject Computer scienceTemplate:WikiProject Computer scienceComputer science articles
This article is within the scope of WikiProject Software, a collaborative effort to improve the coverage of
software on Wikipedia. If you would like to participate, please visit the project page, where you can join
the discussion and see a list of open tasks.SoftwareWikipedia:WikiProject SoftwareTemplate:WikiProject Softwaresoftware articles
The following is an archived discussion of the DYK nomination of the article below. Please do not modify this page. Subsequent comments should be made on the appropriate discussion page (such as
this nomination's talk page,
the article's talk page or
Wikipedia talk:Did you know), unless there is consensus to re-open the discussion at this page. No further edits should be made to this page.
... that technology projects from across the internet, including those of Mozilla and Dropbox, are switching to Pytest from other frameworks for software testing?
Quote: In fact, projects all over the Internet have switched from unittest or nose to pytest, including Mozilla and Dropbox.Okken, Brian (September 2017).
Python Testing with Pytest (1st ed.). The Pragmatic Bookshelf.
ISBN9781680502404. Retrieved 19 March 2022.
An interesting topic and clearly notable. However, the article is correctly tagged as being in need of rewriting, to be less like an instruction manual, and more like a
NPOV article. And in that should make hopefully make the article understandable to a normal reader- I understand the article and its details, but only because I work in the field. This will need to be fixed before this DYK can proceed.
Joseph2302 (
talk)
17:41, 22 March 2022 (UTC)reply
@
Joseph2302: Thank you for your feedback. In the past few days, I took up an effor to fix those issues you mentioned. Now I think the article is in better shape. Please let me know how far it is now from DYK's standard. Thank you.
Thomas Meng (
talk)
01:54, 31 March 2022 (UTC)reply
Apologies, this slipped my mind. Reviewing properly now:
The article still has multiple paragraphs without citations. The minimum amount of sourcing I'd expect is one source per paragraph- if the sources already in the article support the text where I've added citation needed tags, then that should be quick to fix
The text is better, but it still very technical (which does seem to be the case for a lots of computing articles I've noticed). I understand that it's a technical topic, but there's almost nothing in the article that an average reader would understand. Some articles like
Node.js for example has a "History" section, which would be beneficial to a less technical reader. There's still so much code in this article that it's too technical and confusing, and still feels to me like it's a manual on how to use it.
Joseph2302 (
talk)
16:56, 6 April 2022 (UTC)reply
@
Joseph2302: Here's what I've done to address the problems you pointed out:
Modified every section so that each section's first paragraph(s) would only include pytest concepts, and implementation details are saved for the end. Additionally, wording/explanations are improved where possible.
Added a History section for less technical users to read. The lead section should also be understandble for them.
Added ~20 wikilinks for programming related concepts.
The citations problem is also fixed.
Unecessary code templates (e.g. for file, project names) that hinder readability are removed.
Apologies, this dropped off my watchlist. I'm busy at the moment, not much time for Wiki, so would be good if someone could finish the review.
Joseph2302 (
talk)
15:04, 30 May 2022 (UTC)reply
Hmm, I can try to shepherd this through. The prose is okay- not great, it'll need a bit more work, but the more immediate concern is sourcing. Some sources are good, some are iffy due to their status as primary sourcing, and some just shouldn't be used:
Dane Hillard's "Effective Python Testing With Pytest"
tim's "Assertion rewriting in Pytest part 1: Why it’s needed"
Microsoft's "Unit test basics"
Klein's "Testing with pytest"
Perfecto's "Pytest marks"
These all appear to be secondary, yet non-professional sources ranging from personal blogs to coding lessons to company blogs. I don't think any of those meet DYK's reliable sourcing standards, and material relying on it needs a more well-developed source like a book, magazine, newspaper article, scholarly journal, or otherwise.
theleekycauldron (
talk •
contribs) (she/
they)
01:51, 27 May 2022 (UTC)reply
@
Theleekycauldron:, thank you for your patience. I have now fixed the five problems you pointed out by
Removing all four references to Real Python — since three of which had already been corroborated by Okken's book, I only had to delete one short paragraph of actual content.
Removing Tim's blog and replacing content and sourcing with Oliveira's book.
Removing this crowd-contributed source and replacing it with a new book — Unit Testing Principles, Practices, and Patterns.
Removing source and replacing with Okken's book with specific page references.
Thanks
theleekycauldron. I've actually tried to fix the jargon-y issue for several rounds now, and I ended up with the current version. It seems that most programming-related articles do rely on jargons (e.g.
Node.js and
Python (programming language), which is GA), but with wiki-links to them. So that's what I've been trying to do — adding wiki-links and improving explanations for programming concepts where there aren't wiki-links. Maybe could you be more specific on where exactly you'd like the prose improved? I'll be in a better position to fix it then.
The current hook is paraphrased from Okken's book, which I thought was quite indicative of pytest's popularity. Could you please elaborate on why it's not viable so that I know how I should fix it? Thank you.
Thomas Meng (
talk)
03:02, 10 June 2022 (UTC)reply
While pytest is useful for unit testing, integration testing, system or end-to-end testing, and functional testing, the strategy for testing the Tasks project focuses primarily on subcutaneous functional testingOkken, Brian (September 2017).
Python Testing with Pytest (1st ed.). The Pragmatic Bookshelf.
ISBN9781680502404. Retrieved 19 March 2022.
ALT3: ... that pytest is an all-volunteer contributed, open-source software testing tool in
Python and has been classified as a key ecosystem project on the
PyPI with over 9 million weekly downloads? "pytest". pytest.org. Retrieved 15 June 2022.
"Python Package Health Analysis". snyk.io. Retrieved 15 June 2022.
[edit conflict] You're still writing a hook that reads like you are trying to persuade Python programmers to use pytest. That's what it means to be an ad. Try taking the point of view of an encyclopedia reader who is not a programmer. What about pytest could you write that would intrigue a reader and get them to read an encyclopedia article about pytest, even if they have no intention of becoming a programmer? —
David Eppstein (
talk)
19:23, 15 June 2022 (UTC)reply
ALT4: ... that pytest is a
Python framework for writing
software tests that help make sure programs are
bug-free? How about this one?
David Eppstein,
theleekycauldron It's a bit difficult to write a hook that can hook non-programmer's interest into a programming tool — not going into any of pytest's features (e.g. parametrized testing or assert re-writing) due to understandability, while also not including any easy-to-understand facts like download trend or popular usage.
I see that prof
David Eppstein has had experience writing DYK hooks for technical articles. Perhaps you could help compose this hook?. Thank you.
Thomas Meng (
talk)
20:46, 15 June 2022 (UTC)reply
I don't know, my usual strategy is to only make DYK nominations when I think there is something in an article or hook that would stand out to a general audience. When I have a chance to nominate an article, but it is more purely of technical interest, I skip it. Not everything needs to go to DYK. I tried reading through the article a couple of times but nothing stood out to me. —
David Eppstein (
talk)
06:43, 16 June 2022 (UTC)reply
ALT2 or ALT4 are hooked and quoted. Long enough, (posted) new enough, DYKCheck all green, no copyvio. All of the issues raised above have been addressed long ago. Neither the hook nor the article strike me as promotional in their current form. This was posted two months ago and now the hangup is "not everything needs to go to DYK"?! Enough already, this is good to go.
Maury Markowitz (
talk)
18:29, 17 June 2022 (UTC)reply
It's more than that. The hooks are not interesting to a broad audience. Maybe
theleekycauldron or a different prep builder will promote one of those hooks, but I'm against it.
SL93 (
talk)
22:35, 18 June 2022 (UTC)reply
Oddly, our article itself is almost bug-free: its only use of the word "bug" in the whole article involves pypy but not pytest. In fact, it says little or nothing about why you might want to test your software. That is going to make it difficult to write any hook involving bugs. I don't see how the previous hook could have been approved without anything about "bug-free" software appearing in the article. —
David Eppstein (
talk)
19:31, 30 June 2022 (UTC)reply
ALT5 ... that pytest rewrites
assertions in
Python programs, which check data consistency and trigger exceptions if problems are found, into
software tests that can detail the causes of failures?
ALT6... that pytest detects errors in Python computer programs using tests that are structured in an arrange, act and assert sequence known as AAA? Article: "Fixtures practically constitute the arrange phase in the anatomy of a test (AAA, short for arrange, act, assert)." Source 1: "The tests are written in the classical style and use the typical three-phase sequence: arrange, act, and assert (AAA for short...) " Source 2: "I prefer pytest because of some advanced features (fixtures, plug-ins, etc.),... One of the most common patterns you'll find is the 3A or AAA test pattern. AAA stands for Arrange-Act-Assert."
I'm suggesting this as a possible hook because it is in words a general reader can be curious about. :-) This isn't a hook that will catch an expert's interest, but expert hooks are likely to be overly technical or seen as promotional or both.
Mary Mark Ockerbloom (
talk)
20:39, 3 July 2022 (UTC)reply
Sometimes people click on word salad just to see what in the world it could be talking about -- and then they get a glimpse into a topic area they previously knew nothing about. In fact, I'd make it even a bit more obscure by dropping "computer program":
ALT7... that pytest detects Python errors via tests structured in an arrange, act and assert sequence known as AAA?
To forestall going through some of the same bad-promotion-undo cycle we went through already with this: Where in the article do you see a sourced claim that pytest detects errors in Python computer programs? The rest of the hook is ok, but where is that part from? It may seem obvious to people who know about software tests that that's what tests do, but it doesn't seem to be in the article. To be pedantic, it's not the errors in the programs that the tests detect, but the corresponding error in the behavior of the programs. Figuring out where the error is in the program can be significant additional effort from that point. And even if the "detects errors" claim can be found, is an unqualified assertion that it detects errors accurate? All of them? —
David Eppstein (
talk)
17:03, 6 July 2022 (UTC)reply
It actually might be difficult to find a source explicitly stating that the purpose of a test facility is to find bugs -- sort of like finding a source explaining that the purpose of a gun is to fire bullets. But I don't think we can go wrong with this:
ALT8 ... that pytest uses tests structured in an arrange, act and assert sequence known as AAA?
I'd disagree here. ALT7 and its variants are still quite technical, and I'm saying this as a techie myself. I don't think the typical reader would understand this hook, and instead of being inspired to click and understand, they'd probably only be confused further and walk away.
Narutolovehinata5 (
talk ·
contributions)
01:16, 7 July 2022 (UTC)reply
Look, this isn't anyone's Nobel acceptance speech, just a fucking DYK hook. As it stands, ALT8 is the only hook that's actually correct. You're overthinking this.
EEng02:40, 7 July 2022 (UTC)reply
DYK hooks are supposed to be interesting to a broad audience (emphasis mine). It's even in the reviewing instructions. This does not count. And no, before you say "it's vague on purpose because it will make people click the link to find out more", that only works for some hooks. As for this one, I don't think it's one of those times. Non-techies are not going to know what an assert sequence is. Even if we give the hook the benefit of the doubt and say it's the only correct option, it isn't even hooky. It doesn't say anything interesting about the subject.
Narutolovehinata5 (
talk ·
contributions)
05:28, 7 July 2022 (UTC)reply
ALT9 ... that pytest verifies that programs will perform as expected by using tests structured to arrange, act and assert? Article text is updated and cited, citation needed tag has been removed.
Mary Mark Ockerbloom (
talk)
18:42, 8 July 2022 (UTC)reply
"Did you know that a program testing library made Wikipedia editors spend over 2,800 words arguing over how to present it on the site's main page?" Alas, "verifies" is too strong a word, although it could be tweaked. I think ALT8 is passable and it's unlikely we'll be able to milk anything better out of the article. I did a brief reading into pytest's history and found nothing particularly interesting.
Ovinus (
talk)
19:14, 8 July 2022 (UTC)reply
If an article nominated for DYK doesn't have a usable hook, we don't need to force ourselves and squeeze out a hook. A nomination can be rejected for lack of suitable material, there's no rule that requires technically eligible articles to be approved as long as they have a hook, even if said hook is ultimately uninteresting. You can't make pure lemonade out of orange no matter how hard you try.
Narutolovehinata5 (
talk ·
contributions)
23:30, 8 July 2022 (UTC)reply
David suggested the following hook at WT:DYK below. Personally I'm more partial to it even though it's admittedly not the best hook. At least, I think it's better than ALT6/8 and ALT9 above.
ALT10: ... that the software testing framework Pytest has been described as a key ecosystem project for the
Python programming language?
If no hooks can be proposed about a subject, it can be a reason to fail a nomination. Not all articles are meant for DYK, especially if they do not have any information that would be appealing to broad audiences.
Narutolovehinata5 (
talk ·
contributions)
04:31, 9 July 2022 (UTC)reply
Given that nobody else appears to have spelled this out: the nominator is QPQ-exempt, with just one credit to their name. Schwede6601:48, 12 July 2022 (UTC)reply
Approving ALT10. I am late to the party but don't see anything wrong with the article, QPQ excempt, and the hook seems at least moderately interesting to me (and I have nigh zero knowledge of any programming). I could name a dozen only moderately interesting hooks that got approved, so this should not be failed imo. --
LordPeterII (
talk)
09:01, 14 July 2022 (UTC)reply
Happy to have a go at this; love a bit of pytest :) The article overall seems not far off but a few bits that sprung out:
In the lead you say it is "known" for some things, but these seem unsourced assertions, I can't see anything in the text to back it up
The second lead paragraph feels like it should be part of the main article. IN general the lead should simply summarise the article content (i.e. not contain anything original)
considered by some would see to fall into
MOS:WEASEL, worth potentially being clearer - it looks like you have a couple of sources that say it is popular but thats not quite the same thing
Overall, I'd expect the lead to be a tad longer and more detailed to summarise the article
History section; you rely purely on the pytest self-described history, this makes me slightly uncomfortable - are there really no other sources?
Due to pytest's complex history, its initial start date varies based on interpretation.; you reference complexity but then it's not totally clear what you mean, I guess given the split out of pypy?
In fact this whole section cuts back and forth between dates; I would use some of the summary-style words for the lead and re-org the rest to be sequential (remember to tell the story not just document the dates)
was born; feels a bit informal, I'd rework that phrasing
a major focus on testing is very close paraphrasing. Overally I'd probably rework the history section a bit to ensure you're not falling into the trap of getting too close to a source's language
Through the notable features section, I'd compress some of the smaller paragraphs together. The seperation is largely unneeded and breaks flow
Again a lot of reliance on one or few sources in each section - nothing is controversial but a) double check for
MOS:EDITORIAL issues and b) any searching you can do for additional source material (I found a couple of potential decent books and one or two articles and blog posts).
one; go through and rework these - as this is an article not an instruction manual these don't really fit the style
Without using virtual environment, ; this fragment isn't explained (I kbow what it means but assume a lower level of knowledge)
You do rely very heavily on one large source (a book). I'd suggest putting page references in - usually I'd not push on this but you cover a number of topics so clearly it's a decent part of the book being used. Therefore I'd expect to see references to the relevant page to help with validation.
Overall pretty good; it would be great to get some additional sources (especially for the history! even if it is to back up some of the wider pypy history etc.) but I appreciate it's sort of difficult given the topic matter (who makes an academic documentation the history of Open Source projects?!). But find what you can and tweak some of the language and I think you are there. Errant(
chat!)18:25, 8 May 2022 (UTC)reply
@
ErrantX: Thank you very much for the suggestions. Just wanted to let you know that I've seen your comments and will start improving this article based on them in the next few days.
Thomas Meng (
talk)
22:28, 10 May 2022 (UTC)reply
@
ErrantX: I've gone through the issues you pointed out and here's what I've done to fix them, respectively:
Lead section
"known" for things problem: Lead sentence changed to "has been described as a scalable framwork with less boilerplate code". This is a combination of info from the book Python Testing with Pytest and RealPython's article Effective Python Testing with Pytest. Side note: I also moved all the references from the lead into the body per
WP:WHENNOTCITE.
second paragraph problem: Previously the second paragraph on popularity is now a section in the body. Since it's a rather important fact, I also put the same info in the lead (now third paragraph.
"considered by some" problem: Changed to concrete evaluation by Snyk saying that it's classified as a key ecosystem project.
lead too short problem: Added summaries on pytest features mentioned in body (i.e. pytest fixtures, parametrization, and assert rewriting).
History section
Added one other source — the only other source I found on this matter — that talks a bit about PyPy's early testing scheme (i.e. where pytest came from)
[1].
The next four issues are addressed by a rewrite of the section — a lot of unnecessary info is removed to improve clarity and flow, and it is now written in chronological order.
Article body
Many paragraphs combined together to improve flow.
Added additional sourcing from pytest's offial docs.
Not sure what you meant exactly, but I assumed it to be the Test Filtering section where I introduced two ways to filter through tests, so I modified the paragraphs' beginnings a bit to make it less manual-like.
Since a virtual environment is not required for running pytest, I've omitted it entirely since there is a Wikipedia article on that subject already.
Page number references to the most cited book (Python Testing with Pytest) are added to every reference after the first. Also in my opinion this book is perhaps the most comprehensive, well-written book on pytest, and it's one of the two suggested books by the pytest official documentation
[2], so that's why I relied on it heavily.
ErrantX,
Thomas Meng, where does this review stand? As best I can tell, ErrantX hasn't edited Wikipedia since putting the review on hold over a month ago, and it's been three weeks since Thomas Meng signaled that the issues have been addressed. If ErrantX doesn't return soon, there is a chance that the nomination could find a new reviewer from those participating in the June GAN backlog drive. Thanks.
BlueMoonset (
talk)
21:43, 15 June 2022 (UTC)reply
Yeah, seems like ErrantX is extremely busy with his new job (as he said on his Talk). We might need to find someone else then. Thank you for the query.
Thomas Meng (
talk)
01:41, 16 June 2022 (UTC)reply
@
Thomas Meng: This feels nearly hear, the only thing I can see at issue is Pytest has been described as a scalable framework with less boilerplate code than other testing frameworks. . Unless I've missed it (sorry) or it has gotten lost in the edits, I can't see a source for this or work out what it is summarising the article body? Errant(
chat!)03:53, 19 June 2022 (UTC)reply
@
ErrantX: Thanks for raising the issue. That was a left-over summary of the Real Python article on pytest, which has been deleted because a reviewer at the DYK nomination raised
WP:RS. I've now deleted that sentence.
Thomas Meng (
talk)
03:52, 20 June 2022 (UTC)reply
Drive-by comments
Not my article to review, but I thought I should offer some comments:
"Pytest is a software testing framework based on the Python programming language." – Based on? Or for? Or "implemented in, and for"? Test frameworks can be language agnostic
"While Python's built-in assert keyword would only raise AssertionError with no details in cases of failure" – My Python is really rusty, but can't you do assert is_prime(p), f"{p} is not prime!" to raise an AssertionError with a custom msg?
"such as what expressions in the assert statement evaluate to" Vague, it's modifying the actual "evaluation" of the assert statement and its contents? (How is that even possible? By mocking functions, exec/eval, catching and rethrowing AssertionError, ...?)
"Its concision and readability drive many to use pytest over other testing frameworks." Vague and not neutral, remove
"or behave in a certain way as desired by the developer" same here, what does that mean?
"unittest and nose" neither has really been defined
The entire "Installation and running tests" section is unencyclopedic and should be removed per
WP:NOTTUTORIAL. They have a website which can explain this information 20x better than we ever could
@
Ovinus: thank you for your helpful comments. I'll address the issues one by one:
Both for and implemented in. I'll change it to "pytest is a Python testing framework" to make its purpose clearer.
Yes you can do that, but pytest's assert rewriting features also tells the specific index or dictionary item where the values differ, without writing the error message ourselves. As pytest's docs puts it
[3], pytest has support for showing the values of the most common subexpressions including calls, attributes, comparisons, and binary and unary operators. (See Demo of Python failure reports with pytest). This allows you to use the idiomatic python constructs without boilerplate code while not losing introspection information. Here's an example error message from Okken's book:
def test_dict_equality():t1_dict = Task('make sandwich', 'okken')._asdict() t2_dict = Task('make sandwich', 'okkem')._asdict()> assert t1_dict == t2_dictE AssertionError: assert OrderedDict([...('id', None)]) == OrderedDict([(...('id', None)])E Omitting 3 identical items, use -v to showE Differing items:E {'owner': 'okken'} != {'owner': 'okkem'}E Use -v to get the full difftest_task_fail.py:11: AssertionError>
As for how pytest implements this exception rethrowing under the hood, here's a blog post detailing it
[4].
Will remove that sentence.
It means that you can customize pytest markers and make test cases change their behaviour. See
this section in pytest's docs
unittest and nose are also Python testing packages. I will add explanations for those.
Will delete this section.
I will let you know when all the changes are made, which should be straightforward. Please let me know if anything else needs to be done. Regards,
Thomas Meng (
talk)
15:31, 21 June 2022 (UTC)reply
The official project documentation usually refers to the project as "pytest" instead of "Pytest", even at the beginning of sentences. Would it be appropriate to add
Template:Lowercase title to the start of the article? I'm not sure what all the criteria are for doing this and if this is enough to satisfy that.
huntertur (
talk)
18:12, 12 May 2022 (UTC)reply
This article was reviewed by member(s) of WikiProject Articles for creation. The project works to allow users to contribute quality articles and media files to the encyclopedia and track their progress as they are developed. To participate, please visit the
project page for more information.Articles for creationWikipedia:WikiProject Articles for creationTemplate:WikiProject Articles for creationAfC articles
This article is within the scope of WikiProject Computer science, a collaborative effort to improve the coverage of
Computer science related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join
the discussion and see a list of open tasks.Computer scienceWikipedia:WikiProject Computer scienceTemplate:WikiProject Computer scienceComputer science articles
This article is within the scope of WikiProject Software, a collaborative effort to improve the coverage of
software on Wikipedia. If you would like to participate, please visit the project page, where you can join
the discussion and see a list of open tasks.SoftwareWikipedia:WikiProject SoftwareTemplate:WikiProject Softwaresoftware articles
The following is an archived discussion of the DYK nomination of the article below. Please do not modify this page. Subsequent comments should be made on the appropriate discussion page (such as
this nomination's talk page,
the article's talk page or
Wikipedia talk:Did you know), unless there is consensus to re-open the discussion at this page. No further edits should be made to this page.
... that technology projects from across the internet, including those of Mozilla and Dropbox, are switching to Pytest from other frameworks for software testing?
Quote: In fact, projects all over the Internet have switched from unittest or nose to pytest, including Mozilla and Dropbox.Okken, Brian (September 2017).
Python Testing with Pytest (1st ed.). The Pragmatic Bookshelf.
ISBN9781680502404. Retrieved 19 March 2022.
An interesting topic and clearly notable. However, the article is correctly tagged as being in need of rewriting, to be less like an instruction manual, and more like a
NPOV article. And in that should make hopefully make the article understandable to a normal reader- I understand the article and its details, but only because I work in the field. This will need to be fixed before this DYK can proceed.
Joseph2302 (
talk)
17:41, 22 March 2022 (UTC)reply
@
Joseph2302: Thank you for your feedback. In the past few days, I took up an effor to fix those issues you mentioned. Now I think the article is in better shape. Please let me know how far it is now from DYK's standard. Thank you.
Thomas Meng (
talk)
01:54, 31 March 2022 (UTC)reply
Apologies, this slipped my mind. Reviewing properly now:
The article still has multiple paragraphs without citations. The minimum amount of sourcing I'd expect is one source per paragraph- if the sources already in the article support the text where I've added citation needed tags, then that should be quick to fix
The text is better, but it still very technical (which does seem to be the case for a lots of computing articles I've noticed). I understand that it's a technical topic, but there's almost nothing in the article that an average reader would understand. Some articles like
Node.js for example has a "History" section, which would be beneficial to a less technical reader. There's still so much code in this article that it's too technical and confusing, and still feels to me like it's a manual on how to use it.
Joseph2302 (
talk)
16:56, 6 April 2022 (UTC)reply
@
Joseph2302: Here's what I've done to address the problems you pointed out:
Modified every section so that each section's first paragraph(s) would only include pytest concepts, and implementation details are saved for the end. Additionally, wording/explanations are improved where possible.
Added a History section for less technical users to read. The lead section should also be understandble for them.
Added ~20 wikilinks for programming related concepts.
The citations problem is also fixed.
Unecessary code templates (e.g. for file, project names) that hinder readability are removed.
Apologies, this dropped off my watchlist. I'm busy at the moment, not much time for Wiki, so would be good if someone could finish the review.
Joseph2302 (
talk)
15:04, 30 May 2022 (UTC)reply
Hmm, I can try to shepherd this through. The prose is okay- not great, it'll need a bit more work, but the more immediate concern is sourcing. Some sources are good, some are iffy due to their status as primary sourcing, and some just shouldn't be used:
Dane Hillard's "Effective Python Testing With Pytest"
tim's "Assertion rewriting in Pytest part 1: Why it’s needed"
Microsoft's "Unit test basics"
Klein's "Testing with pytest"
Perfecto's "Pytest marks"
These all appear to be secondary, yet non-professional sources ranging from personal blogs to coding lessons to company blogs. I don't think any of those meet DYK's reliable sourcing standards, and material relying on it needs a more well-developed source like a book, magazine, newspaper article, scholarly journal, or otherwise.
theleekycauldron (
talk •
contribs) (she/
they)
01:51, 27 May 2022 (UTC)reply
@
Theleekycauldron:, thank you for your patience. I have now fixed the five problems you pointed out by
Removing all four references to Real Python — since three of which had already been corroborated by Okken's book, I only had to delete one short paragraph of actual content.
Removing Tim's blog and replacing content and sourcing with Oliveira's book.
Removing this crowd-contributed source and replacing it with a new book — Unit Testing Principles, Practices, and Patterns.
Removing source and replacing with Okken's book with specific page references.
Thanks
theleekycauldron. I've actually tried to fix the jargon-y issue for several rounds now, and I ended up with the current version. It seems that most programming-related articles do rely on jargons (e.g.
Node.js and
Python (programming language), which is GA), but with wiki-links to them. So that's what I've been trying to do — adding wiki-links and improving explanations for programming concepts where there aren't wiki-links. Maybe could you be more specific on where exactly you'd like the prose improved? I'll be in a better position to fix it then.
The current hook is paraphrased from Okken's book, which I thought was quite indicative of pytest's popularity. Could you please elaborate on why it's not viable so that I know how I should fix it? Thank you.
Thomas Meng (
talk)
03:02, 10 June 2022 (UTC)reply
While pytest is useful for unit testing, integration testing, system or end-to-end testing, and functional testing, the strategy for testing the Tasks project focuses primarily on subcutaneous functional testingOkken, Brian (September 2017).
Python Testing with Pytest (1st ed.). The Pragmatic Bookshelf.
ISBN9781680502404. Retrieved 19 March 2022.
ALT3: ... that pytest is an all-volunteer contributed, open-source software testing tool in
Python and has been classified as a key ecosystem project on the
PyPI with over 9 million weekly downloads? "pytest". pytest.org. Retrieved 15 June 2022.
"Python Package Health Analysis". snyk.io. Retrieved 15 June 2022.
[edit conflict] You're still writing a hook that reads like you are trying to persuade Python programmers to use pytest. That's what it means to be an ad. Try taking the point of view of an encyclopedia reader who is not a programmer. What about pytest could you write that would intrigue a reader and get them to read an encyclopedia article about pytest, even if they have no intention of becoming a programmer? —
David Eppstein (
talk)
19:23, 15 June 2022 (UTC)reply
ALT4: ... that pytest is a
Python framework for writing
software tests that help make sure programs are
bug-free? How about this one?
David Eppstein,
theleekycauldron It's a bit difficult to write a hook that can hook non-programmer's interest into a programming tool — not going into any of pytest's features (e.g. parametrized testing or assert re-writing) due to understandability, while also not including any easy-to-understand facts like download trend or popular usage.
I see that prof
David Eppstein has had experience writing DYK hooks for technical articles. Perhaps you could help compose this hook?. Thank you.
Thomas Meng (
talk)
20:46, 15 June 2022 (UTC)reply
I don't know, my usual strategy is to only make DYK nominations when I think there is something in an article or hook that would stand out to a general audience. When I have a chance to nominate an article, but it is more purely of technical interest, I skip it. Not everything needs to go to DYK. I tried reading through the article a couple of times but nothing stood out to me. —
David Eppstein (
talk)
06:43, 16 June 2022 (UTC)reply
ALT2 or ALT4 are hooked and quoted. Long enough, (posted) new enough, DYKCheck all green, no copyvio. All of the issues raised above have been addressed long ago. Neither the hook nor the article strike me as promotional in their current form. This was posted two months ago and now the hangup is "not everything needs to go to DYK"?! Enough already, this is good to go.
Maury Markowitz (
talk)
18:29, 17 June 2022 (UTC)reply
It's more than that. The hooks are not interesting to a broad audience. Maybe
theleekycauldron or a different prep builder will promote one of those hooks, but I'm against it.
SL93 (
talk)
22:35, 18 June 2022 (UTC)reply
Oddly, our article itself is almost bug-free: its only use of the word "bug" in the whole article involves pypy but not pytest. In fact, it says little or nothing about why you might want to test your software. That is going to make it difficult to write any hook involving bugs. I don't see how the previous hook could have been approved without anything about "bug-free" software appearing in the article. —
David Eppstein (
talk)
19:31, 30 June 2022 (UTC)reply
ALT5 ... that pytest rewrites
assertions in
Python programs, which check data consistency and trigger exceptions if problems are found, into
software tests that can detail the causes of failures?
ALT6... that pytest detects errors in Python computer programs using tests that are structured in an arrange, act and assert sequence known as AAA? Article: "Fixtures practically constitute the arrange phase in the anatomy of a test (AAA, short for arrange, act, assert)." Source 1: "The tests are written in the classical style and use the typical three-phase sequence: arrange, act, and assert (AAA for short...) " Source 2: "I prefer pytest because of some advanced features (fixtures, plug-ins, etc.),... One of the most common patterns you'll find is the 3A or AAA test pattern. AAA stands for Arrange-Act-Assert."
I'm suggesting this as a possible hook because it is in words a general reader can be curious about. :-) This isn't a hook that will catch an expert's interest, but expert hooks are likely to be overly technical or seen as promotional or both.
Mary Mark Ockerbloom (
talk)
20:39, 3 July 2022 (UTC)reply
Sometimes people click on word salad just to see what in the world it could be talking about -- and then they get a glimpse into a topic area they previously knew nothing about. In fact, I'd make it even a bit more obscure by dropping "computer program":
ALT7... that pytest detects Python errors via tests structured in an arrange, act and assert sequence known as AAA?
To forestall going through some of the same bad-promotion-undo cycle we went through already with this: Where in the article do you see a sourced claim that pytest detects errors in Python computer programs? The rest of the hook is ok, but where is that part from? It may seem obvious to people who know about software tests that that's what tests do, but it doesn't seem to be in the article. To be pedantic, it's not the errors in the programs that the tests detect, but the corresponding error in the behavior of the programs. Figuring out where the error is in the program can be significant additional effort from that point. And even if the "detects errors" claim can be found, is an unqualified assertion that it detects errors accurate? All of them? —
David Eppstein (
talk)
17:03, 6 July 2022 (UTC)reply
It actually might be difficult to find a source explicitly stating that the purpose of a test facility is to find bugs -- sort of like finding a source explaining that the purpose of a gun is to fire bullets. But I don't think we can go wrong with this:
ALT8 ... that pytest uses tests structured in an arrange, act and assert sequence known as AAA?
I'd disagree here. ALT7 and its variants are still quite technical, and I'm saying this as a techie myself. I don't think the typical reader would understand this hook, and instead of being inspired to click and understand, they'd probably only be confused further and walk away.
Narutolovehinata5 (
talk ·
contributions)
01:16, 7 July 2022 (UTC)reply
Look, this isn't anyone's Nobel acceptance speech, just a fucking DYK hook. As it stands, ALT8 is the only hook that's actually correct. You're overthinking this.
EEng02:40, 7 July 2022 (UTC)reply
DYK hooks are supposed to be interesting to a broad audience (emphasis mine). It's even in the reviewing instructions. This does not count. And no, before you say "it's vague on purpose because it will make people click the link to find out more", that only works for some hooks. As for this one, I don't think it's one of those times. Non-techies are not going to know what an assert sequence is. Even if we give the hook the benefit of the doubt and say it's the only correct option, it isn't even hooky. It doesn't say anything interesting about the subject.
Narutolovehinata5 (
talk ·
contributions)
05:28, 7 July 2022 (UTC)reply
ALT9 ... that pytest verifies that programs will perform as expected by using tests structured to arrange, act and assert? Article text is updated and cited, citation needed tag has been removed.
Mary Mark Ockerbloom (
talk)
18:42, 8 July 2022 (UTC)reply
"Did you know that a program testing library made Wikipedia editors spend over 2,800 words arguing over how to present it on the site's main page?" Alas, "verifies" is too strong a word, although it could be tweaked. I think ALT8 is passable and it's unlikely we'll be able to milk anything better out of the article. I did a brief reading into pytest's history and found nothing particularly interesting.
Ovinus (
talk)
19:14, 8 July 2022 (UTC)reply
If an article nominated for DYK doesn't have a usable hook, we don't need to force ourselves and squeeze out a hook. A nomination can be rejected for lack of suitable material, there's no rule that requires technically eligible articles to be approved as long as they have a hook, even if said hook is ultimately uninteresting. You can't make pure lemonade out of orange no matter how hard you try.
Narutolovehinata5 (
talk ·
contributions)
23:30, 8 July 2022 (UTC)reply
David suggested the following hook at WT:DYK below. Personally I'm more partial to it even though it's admittedly not the best hook. At least, I think it's better than ALT6/8 and ALT9 above.
ALT10: ... that the software testing framework Pytest has been described as a key ecosystem project for the
Python programming language?
If no hooks can be proposed about a subject, it can be a reason to fail a nomination. Not all articles are meant for DYK, especially if they do not have any information that would be appealing to broad audiences.
Narutolovehinata5 (
talk ·
contributions)
04:31, 9 July 2022 (UTC)reply
Given that nobody else appears to have spelled this out: the nominator is QPQ-exempt, with just one credit to their name. Schwede6601:48, 12 July 2022 (UTC)reply
Approving ALT10. I am late to the party but don't see anything wrong with the article, QPQ excempt, and the hook seems at least moderately interesting to me (and I have nigh zero knowledge of any programming). I could name a dozen only moderately interesting hooks that got approved, so this should not be failed imo. --
LordPeterII (
talk)
09:01, 14 July 2022 (UTC)reply
Happy to have a go at this; love a bit of pytest :) The article overall seems not far off but a few bits that sprung out:
In the lead you say it is "known" for some things, but these seem unsourced assertions, I can't see anything in the text to back it up
The second lead paragraph feels like it should be part of the main article. IN general the lead should simply summarise the article content (i.e. not contain anything original)
considered by some would see to fall into
MOS:WEASEL, worth potentially being clearer - it looks like you have a couple of sources that say it is popular but thats not quite the same thing
Overall, I'd expect the lead to be a tad longer and more detailed to summarise the article
History section; you rely purely on the pytest self-described history, this makes me slightly uncomfortable - are there really no other sources?
Due to pytest's complex history, its initial start date varies based on interpretation.; you reference complexity but then it's not totally clear what you mean, I guess given the split out of pypy?
In fact this whole section cuts back and forth between dates; I would use some of the summary-style words for the lead and re-org the rest to be sequential (remember to tell the story not just document the dates)
was born; feels a bit informal, I'd rework that phrasing
a major focus on testing is very close paraphrasing. Overally I'd probably rework the history section a bit to ensure you're not falling into the trap of getting too close to a source's language
Through the notable features section, I'd compress some of the smaller paragraphs together. The seperation is largely unneeded and breaks flow
Again a lot of reliance on one or few sources in each section - nothing is controversial but a) double check for
MOS:EDITORIAL issues and b) any searching you can do for additional source material (I found a couple of potential decent books and one or two articles and blog posts).
one; go through and rework these - as this is an article not an instruction manual these don't really fit the style
Without using virtual environment, ; this fragment isn't explained (I kbow what it means but assume a lower level of knowledge)
You do rely very heavily on one large source (a book). I'd suggest putting page references in - usually I'd not push on this but you cover a number of topics so clearly it's a decent part of the book being used. Therefore I'd expect to see references to the relevant page to help with validation.
Overall pretty good; it would be great to get some additional sources (especially for the history! even if it is to back up some of the wider pypy history etc.) but I appreciate it's sort of difficult given the topic matter (who makes an academic documentation the history of Open Source projects?!). But find what you can and tweak some of the language and I think you are there. Errant(
chat!)18:25, 8 May 2022 (UTC)reply
@
ErrantX: Thank you very much for the suggestions. Just wanted to let you know that I've seen your comments and will start improving this article based on them in the next few days.
Thomas Meng (
talk)
22:28, 10 May 2022 (UTC)reply
@
ErrantX: I've gone through the issues you pointed out and here's what I've done to fix them, respectively:
Lead section
"known" for things problem: Lead sentence changed to "has been described as a scalable framwork with less boilerplate code". This is a combination of info from the book Python Testing with Pytest and RealPython's article Effective Python Testing with Pytest. Side note: I also moved all the references from the lead into the body per
WP:WHENNOTCITE.
second paragraph problem: Previously the second paragraph on popularity is now a section in the body. Since it's a rather important fact, I also put the same info in the lead (now third paragraph.
"considered by some" problem: Changed to concrete evaluation by Snyk saying that it's classified as a key ecosystem project.
lead too short problem: Added summaries on pytest features mentioned in body (i.e. pytest fixtures, parametrization, and assert rewriting).
History section
Added one other source — the only other source I found on this matter — that talks a bit about PyPy's early testing scheme (i.e. where pytest came from)
[1].
The next four issues are addressed by a rewrite of the section — a lot of unnecessary info is removed to improve clarity and flow, and it is now written in chronological order.
Article body
Many paragraphs combined together to improve flow.
Added additional sourcing from pytest's offial docs.
Not sure what you meant exactly, but I assumed it to be the Test Filtering section where I introduced two ways to filter through tests, so I modified the paragraphs' beginnings a bit to make it less manual-like.
Since a virtual environment is not required for running pytest, I've omitted it entirely since there is a Wikipedia article on that subject already.
Page number references to the most cited book (Python Testing with Pytest) are added to every reference after the first. Also in my opinion this book is perhaps the most comprehensive, well-written book on pytest, and it's one of the two suggested books by the pytest official documentation
[2], so that's why I relied on it heavily.
ErrantX,
Thomas Meng, where does this review stand? As best I can tell, ErrantX hasn't edited Wikipedia since putting the review on hold over a month ago, and it's been three weeks since Thomas Meng signaled that the issues have been addressed. If ErrantX doesn't return soon, there is a chance that the nomination could find a new reviewer from those participating in the June GAN backlog drive. Thanks.
BlueMoonset (
talk)
21:43, 15 June 2022 (UTC)reply
Yeah, seems like ErrantX is extremely busy with his new job (as he said on his Talk). We might need to find someone else then. Thank you for the query.
Thomas Meng (
talk)
01:41, 16 June 2022 (UTC)reply
@
Thomas Meng: This feels nearly hear, the only thing I can see at issue is Pytest has been described as a scalable framework with less boilerplate code than other testing frameworks. . Unless I've missed it (sorry) or it has gotten lost in the edits, I can't see a source for this or work out what it is summarising the article body? Errant(
chat!)03:53, 19 June 2022 (UTC)reply
@
ErrantX: Thanks for raising the issue. That was a left-over summary of the Real Python article on pytest, which has been deleted because a reviewer at the DYK nomination raised
WP:RS. I've now deleted that sentence.
Thomas Meng (
talk)
03:52, 20 June 2022 (UTC)reply
Drive-by comments
Not my article to review, but I thought I should offer some comments:
"Pytest is a software testing framework based on the Python programming language." – Based on? Or for? Or "implemented in, and for"? Test frameworks can be language agnostic
"While Python's built-in assert keyword would only raise AssertionError with no details in cases of failure" – My Python is really rusty, but can't you do assert is_prime(p), f"{p} is not prime!" to raise an AssertionError with a custom msg?
"such as what expressions in the assert statement evaluate to" Vague, it's modifying the actual "evaluation" of the assert statement and its contents? (How is that even possible? By mocking functions, exec/eval, catching and rethrowing AssertionError, ...?)
"Its concision and readability drive many to use pytest over other testing frameworks." Vague and not neutral, remove
"or behave in a certain way as desired by the developer" same here, what does that mean?
"unittest and nose" neither has really been defined
The entire "Installation and running tests" section is unencyclopedic and should be removed per
WP:NOTTUTORIAL. They have a website which can explain this information 20x better than we ever could
@
Ovinus: thank you for your helpful comments. I'll address the issues one by one:
Both for and implemented in. I'll change it to "pytest is a Python testing framework" to make its purpose clearer.
Yes you can do that, but pytest's assert rewriting features also tells the specific index or dictionary item where the values differ, without writing the error message ourselves. As pytest's docs puts it
[3], pytest has support for showing the values of the most common subexpressions including calls, attributes, comparisons, and binary and unary operators. (See Demo of Python failure reports with pytest). This allows you to use the idiomatic python constructs without boilerplate code while not losing introspection information. Here's an example error message from Okken's book:
def test_dict_equality():t1_dict = Task('make sandwich', 'okken')._asdict() t2_dict = Task('make sandwich', 'okkem')._asdict()> assert t1_dict == t2_dictE AssertionError: assert OrderedDict([...('id', None)]) == OrderedDict([(...('id', None)])E Omitting 3 identical items, use -v to showE Differing items:E {'owner': 'okken'} != {'owner': 'okkem'}E Use -v to get the full difftest_task_fail.py:11: AssertionError>
As for how pytest implements this exception rethrowing under the hood, here's a blog post detailing it
[4].
Will remove that sentence.
It means that you can customize pytest markers and make test cases change their behaviour. See
this section in pytest's docs
unittest and nose are also Python testing packages. I will add explanations for those.
Will delete this section.
I will let you know when all the changes are made, which should be straightforward. Please let me know if anything else needs to be done. Regards,
Thomas Meng (
talk)
15:31, 21 June 2022 (UTC)reply
The official project documentation usually refers to the project as "pytest" instead of "Pytest", even at the beginning of sentences. Would it be appropriate to add
Template:Lowercase title to the start of the article? I'm not sure what all the criteria are for doing this and if this is enough to satisfy that.
huntertur (
talk)
18:12, 12 May 2022 (UTC)reply