This is the
talk page for discussing improvements to the
ChatGPT article. This is not a forum for general discussion of the article's subject. |
Article policies
|
Find sources: Google ( books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL |
Archives:
1Auto-archiving period: 90 days
![]() |
![]() | This talk page is semi-protected due an unmanageable torrent of edits from people who think this is where you may ask ChatGPT a question. It is not. If you cannot edit this page and want to request an edit that is about improving the article, make an edit request instead. |
![]() | This page is not a forum for general discussion about ChatGPT. Any such comments may be removed or refactored. Please limit discussion to improvement of this article. You may wish to ask factual questions about ChatGPT at the Reference desk. |
![]() | This ![]() It is of interest to multiple WikiProjects. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Several people now have also posted messages at Talk:Gemini (chatbot) believing — or pretending — to ask Gemini a question. As the number of chatbots continues to grow, I think it may be beneficial to have a generic editnotice template for talk pages of chatbot or generative AI software. InfiniteNexus ( talk) 06:37, 25 February 2024 (UTC)
|public=no
parameter to change the wording from "to do so, you may visit" to "learn more at". As for the non-chatbot concern, what wording change do you have in mind? Personally, it sounds pretty general to me, but I'm open to suggestions.
InfiniteNexus (
talk)
19:38, 25 February 2024 (UTC)
{{
namespace detect}}
templates to detect if it's being used in mainspace and tweak it a little bit so it can be used on vandalized articles (not just talk pages). ~~
2NumForIce (
speak|
edits)
03:34, 11 March 2024 (UTC)I think there is a need for a discussion with @ CommonKnowledgeCreator and @ JPxG and anyone interested about this modification.
Personally, my impression is that the content removed by CommonKnowledgeCreator seems pretty accurate in regards to the sources (these studies indeed says that, although the phrasing should more clearly indicate that ChatGPT outperformed the average of the 10 investment funds, not necessarily all). And there is a desire to give the full picture of Patronus AI's study. But it's also confusing and I think the part on Patronus AI goes too deep into technical details that should be left in the references for further reading. I would probably prefer having just a few simple sentences with easy-to-interpret statistics concisely explaining what ChatGPT is good at (making quick and performant sentiment analysis and investment advice when fed a lot of data, integration with plugins) and what it's currently bad at (causal reasoning and mathematics, lack of knowledge of recent events, need for double-checking, reading facial expressions), e.g. a synthesis of the points in this article that look still relevant today.
That's my opinion, but I confess that I'm not skilled in financial markets (as for most Wikipedia readers). What do you think? Alenoach ( talk) 17:19, 8 March 2024 (UTC)
I think that we should write the article by using reliable sources trying to figure out what the truth is, and then say that, and if the sources disagree, reflect that in our writing. What we should not do is write "A study proved that it was good.[1] However, a different study later proved that it was bad.[2]
"
The organization of this article, bluntly, makes no sense and seems like the product of many sentences being slapped in piecemeal over the course of years. This section is flatly bizarre -- it's in "criticism by industry". The subsection name is "financial markets". Are the financial markets criticizing ChatGPT? Here is a sentence from it:
An experiment by finder.com revealed that ChatGPT could outperform popular fund managers by picking stocks based on criteria such as growth history and debt levels, resulting in a 4.9% increase in a hypothetical account of 38 stocks, outperforming 10 benchmarked investment funds with an average loss of 0.8%.
What part of this is "criticism"? Here is another.
On the retrieval system version, GPT-4-Turbo and LLaMA-2 both failed to produce correct answers to 81% of the questions, while on the long context window version, GPT-4-Turbo and Claude-2 failed to produce correct answers to 21% and 24% of the questions, respectively.
Is this "criticism"? Why does this belong in this section?
The more serious issue, to me, is that this actually has nothing to do with the previous thing. The first citation -- that it "outperformed popular fund managers" -- is cited to a fluff piece from CNN. Buried in the fourth paragraph of that fluff piece -- the one that boasts 4.9% ROI -- is that "Over the same eight-week period, the S&P 500 index, which tracks the 500 most valuable companies in the United States, rose 3%". There is no link to more detailed information, i.e. what the stocks were, what firms it was being compared against -- literally all we have to go by is that CNN is quoting some guy as saying "some of the most popular investment funds in the United Kingdom". The website itself has a page about this -- note it's written by their "Head of Communications & Content Marketing", not an analyst -- that doesn't explain any of this stuff either.
It doesn't give us information like, say, how the model was actually being used: were they prompting it with actual filings? were they prompting it with stock information? news articles? if this were actual credible research there would be a paper explaining these things, but it is clickbait, so there's not. It's also specifically talking about the United Kingdom. The SEC operates in the United States. Moreover, there is not a basis to claim that this is being refuted by different articles, published later, and by different people, about a different country, which the sources have not claimed is relevant to this. What's the connection supposed to be between this and it "failing" to read SEC filings? As far as I can tell, nobody ever claimed or warranted or represented that it could read SEC filings, so it's difficult to see how this is a "failure". Have I "failed" to learn how to speak Xhosa (I have never attempted to learn Xhosa, and never told anyone I was)? Have I "failed" to bench press eighteen thousand pounds, etc?
Like I said -- if we can't be bothered to find actual research that attempts to analyze these things in a detailed way, we should not just be trawling around for random clickbait from content marketers and then pretending it's research in the voice of the encyclopedia.
jp×
g
🗯️
20:49, 8 March 2024 (UTC)
?The infobox is now out of date, and because I have no idea how to change the website paramater of the infobox because the Website paramater has for some reason not been filled in could someone else please change it Someone-123-321 (I contribute) 02:54, 4 May 2024 (UTC)
I turned the copyright concerns part of "Reception" into a subsection of the said section, as it is evident that there may be more copyright challenges to the platform. One of these is not mentioned in the article: the Authors' Guild lawsuit vs. OpenAI. Relevant sources mentioning it: from ABC News, from Variety, and from WIRED. JWilz12345 ( Talk| Contrib's.) 19:22, 12 May 2024 (UTC)
The introduction describes ChatGPT almost exclusively as a chatbot. Some readers may conclude that it can just read and generate text. I think it would be good to update it, maybe with a paragraph that gives an overview of the current features and potentially talks a bit about GPT-4o, since it's clearly the best model right now and it's multimodal. No need to use complicated words like multimodality, since many readers may not be familiar with the technical jargon. Alenoach ( talk) 22:46, 21 May 2024 (UTC)
This is the
talk page for discussing improvements to the
ChatGPT article. This is not a forum for general discussion of the article's subject. |
Article policies
|
Find sources: Google ( books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL |
Archives:
1Auto-archiving period: 90 days
![]() |
![]() | This talk page is semi-protected due an unmanageable torrent of edits from people who think this is where you may ask ChatGPT a question. It is not. If you cannot edit this page and want to request an edit that is about improving the article, make an edit request instead. |
![]() | This page is not a forum for general discussion about ChatGPT. Any such comments may be removed or refactored. Please limit discussion to improvement of this article. You may wish to ask factual questions about ChatGPT at the Reference desk. |
![]() | This ![]() It is of interest to multiple WikiProjects. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
![]() | Other talk page banners | ||||||
|
Several people now have also posted messages at Talk:Gemini (chatbot) believing — or pretending — to ask Gemini a question. As the number of chatbots continues to grow, I think it may be beneficial to have a generic editnotice template for talk pages of chatbot or generative AI software. InfiniteNexus ( talk) 06:37, 25 February 2024 (UTC)
|public=no
parameter to change the wording from "to do so, you may visit" to "learn more at". As for the non-chatbot concern, what wording change do you have in mind? Personally, it sounds pretty general to me, but I'm open to suggestions.
InfiniteNexus (
talk)
19:38, 25 February 2024 (UTC)
{{
namespace detect}}
templates to detect if it's being used in mainspace and tweak it a little bit so it can be used on vandalized articles (not just talk pages). ~~
2NumForIce (
speak|
edits)
03:34, 11 March 2024 (UTC)I think there is a need for a discussion with @ CommonKnowledgeCreator and @ JPxG and anyone interested about this modification.
Personally, my impression is that the content removed by CommonKnowledgeCreator seems pretty accurate in regards to the sources (these studies indeed says that, although the phrasing should more clearly indicate that ChatGPT outperformed the average of the 10 investment funds, not necessarily all). And there is a desire to give the full picture of Patronus AI's study. But it's also confusing and I think the part on Patronus AI goes too deep into technical details that should be left in the references for further reading. I would probably prefer having just a few simple sentences with easy-to-interpret statistics concisely explaining what ChatGPT is good at (making quick and performant sentiment analysis and investment advice when fed a lot of data, integration with plugins) and what it's currently bad at (causal reasoning and mathematics, lack of knowledge of recent events, need for double-checking, reading facial expressions), e.g. a synthesis of the points in this article that look still relevant today.
That's my opinion, but I confess that I'm not skilled in financial markets (as for most Wikipedia readers). What do you think? Alenoach ( talk) 17:19, 8 March 2024 (UTC)
I think that we should write the article by using reliable sources trying to figure out what the truth is, and then say that, and if the sources disagree, reflect that in our writing. What we should not do is write "A study proved that it was good.[1] However, a different study later proved that it was bad.[2]
"
The organization of this article, bluntly, makes no sense and seems like the product of many sentences being slapped in piecemeal over the course of years. This section is flatly bizarre -- it's in "criticism by industry". The subsection name is "financial markets". Are the financial markets criticizing ChatGPT? Here is a sentence from it:
An experiment by finder.com revealed that ChatGPT could outperform popular fund managers by picking stocks based on criteria such as growth history and debt levels, resulting in a 4.9% increase in a hypothetical account of 38 stocks, outperforming 10 benchmarked investment funds with an average loss of 0.8%.
What part of this is "criticism"? Here is another.
On the retrieval system version, GPT-4-Turbo and LLaMA-2 both failed to produce correct answers to 81% of the questions, while on the long context window version, GPT-4-Turbo and Claude-2 failed to produce correct answers to 21% and 24% of the questions, respectively.
Is this "criticism"? Why does this belong in this section?
The more serious issue, to me, is that this actually has nothing to do with the previous thing. The first citation -- that it "outperformed popular fund managers" -- is cited to a fluff piece from CNN. Buried in the fourth paragraph of that fluff piece -- the one that boasts 4.9% ROI -- is that "Over the same eight-week period, the S&P 500 index, which tracks the 500 most valuable companies in the United States, rose 3%". There is no link to more detailed information, i.e. what the stocks were, what firms it was being compared against -- literally all we have to go by is that CNN is quoting some guy as saying "some of the most popular investment funds in the United Kingdom". The website itself has a page about this -- note it's written by their "Head of Communications & Content Marketing", not an analyst -- that doesn't explain any of this stuff either.
It doesn't give us information like, say, how the model was actually being used: were they prompting it with actual filings? were they prompting it with stock information? news articles? if this were actual credible research there would be a paper explaining these things, but it is clickbait, so there's not. It's also specifically talking about the United Kingdom. The SEC operates in the United States. Moreover, there is not a basis to claim that this is being refuted by different articles, published later, and by different people, about a different country, which the sources have not claimed is relevant to this. What's the connection supposed to be between this and it "failing" to read SEC filings? As far as I can tell, nobody ever claimed or warranted or represented that it could read SEC filings, so it's difficult to see how this is a "failure". Have I "failed" to learn how to speak Xhosa (I have never attempted to learn Xhosa, and never told anyone I was)? Have I "failed" to bench press eighteen thousand pounds, etc?
Like I said -- if we can't be bothered to find actual research that attempts to analyze these things in a detailed way, we should not just be trawling around for random clickbait from content marketers and then pretending it's research in the voice of the encyclopedia.
jp×
g
🗯️
20:49, 8 March 2024 (UTC)
?The infobox is now out of date, and because I have no idea how to change the website paramater of the infobox because the Website paramater has for some reason not been filled in could someone else please change it Someone-123-321 (I contribute) 02:54, 4 May 2024 (UTC)
I turned the copyright concerns part of "Reception" into a subsection of the said section, as it is evident that there may be more copyright challenges to the platform. One of these is not mentioned in the article: the Authors' Guild lawsuit vs. OpenAI. Relevant sources mentioning it: from ABC News, from Variety, and from WIRED. JWilz12345 ( Talk| Contrib's.) 19:22, 12 May 2024 (UTC)
The introduction describes ChatGPT almost exclusively as a chatbot. Some readers may conclude that it can just read and generate text. I think it would be good to update it, maybe with a paragraph that gives an overview of the current features and potentially talks a bit about GPT-4o, since it's clearly the best model right now and it's multimodal. No need to use complicated words like multimodality, since many readers may not be familiar with the technical jargon. Alenoach ( talk) 22:46, 21 May 2024 (UTC)