Elevate Your Case Prep with ChatGPT

Explore more.

  • Artificial Intelligence
  • Case Teaching
  • Course Design

G reat case method teaching is, in many ways, much like prompt engineering for generative AI. You pose a query. You provide some guidance for the type of response you are expecting. You wait for the answer with ample hope, a bit of angst, and lots of curiosity.

As case method teachers, when students reward our questions with revelation, we’re prone to describing the interactions as “magic.” New generative AI practitioners are inclined to do the same. In both cases, this is inaccurate technically, but accurate evocatively.

It should be no surprise, then, that ChatGPT can aid us with teaching by the case method, a technique that professors have deployed for more than 100 years. Case instructors, who for decades have been taking care to develop effective questions that prompt robust discussions, now have a suite of AI tools to help them do that even more productively.

I am no AI expert. I have, however, spent much time experimenting with ChatGPT (and other generative AI tools like Bing, Bard, and Claude). I’ve developed approaches that make my case preparation sharper. These help me refine discussion questions I want to ask—and even anticipate, to some extent, how students may answer.

I encourage you to experiment with these technologies on your own to get a fuller understanding of exactly how you could use ChatGPT and other tools for your teaching. But to get you started, I’ll share five ways I now use ChatGPT to prepare for case discussions. Among them, I’ll explain how it can even simulate student responses in the classroom to help you feel more ready.

1. Generate a list of relevant case questions

Watch the webinar.

In the webinar “ How ChatGPT and Other AI Tools Can Maximize the Learning Potential of Your Case-Based Classes ,” Harvard Business School professor Mitchell Weiss speaks in depth about this topic, detailing how educators can use AI tools to both prepare for and teach case-based courses. Watch the webinar recording to learn more.

You likely won’t love everything it generates, and I would never recommend an instructor walk into class with nothing but a list of AI-generated discussion questions. But ChatGPT can provide useful supplements to questions you might have thought of, or help get you started drafting more on your own.

As an example, here’s a prompt I gave ChatGPT about a case I wrote and often teach:

“I am a business school professor. I would like your help drafting some potential questions for a session I teach by the case method. The case I use for the session is on TraceTogether , an app developed in Singapore during COVID-19 to do digital contact tracing. The case protagonist is Jason Bay, a leader at GovTech Singapore, the digital transformation agency of Singapore’s government. Key passages in the case cover the debate over these apps and whether they should have government officials ‘in the loop’ or ‘out of the loop’ to further protect user privacy. Suggest 10 questions I might ask related to these passages, please.”

I was sure to include the following in my prompt:

Who I am and what my goals are

Essential information about the case, including who the protagonist is and what key concepts are covered

Clear instructions about the exact output I want

Note that I did not upload any material, including the case itself.

The AI generated 10 questions relevant to the case; I then added these for consideration to questions I had previously thought of, and I evaluated each question against my goals for the class to decide if I wanted to use it.

2. Refine question results in line with your discussion goals

I found the list of ChatGPT’s suggested questions generally promising. Of the 10, I could have taken any of them into class to initiate a reasonably productive conversation, though three would have required student knowledge from outside the scope of the case. I was, of course, most drawn to the handful that tilted toward the teaching objectives I was aiming to cover.

ChatGPT can also be useful in refining these initial results. I added one more prompt into ChatGPT in hopes of receiving an even sharper list of questions, including some that would push students to do what they must do in a great case conversation: decide how to act.

“As a starting point, ChatGPT can be a brainstorming aid when you are crafting questions to ask your students in preparation for a case discussion.”

Here is what I wrote:

“Do the same thing again, but this time, put the 10 questions into three categories: description, diagnosis, action.”

The AI again generated 10 questions, some similar to the first list, but all categorized as specified in my prompt. This helped me consider what role each question might play in a class discussion. Case method teachers ask themselves, for example: Do I want to open a session with a cold call that asks for diagnosis? Do I want to jump to action and work backward instead? Now I had a range of options to consider.

3. Connect questions to case concepts

Case discussion questions must connect to learning outcomes and illustrate generalizable concepts. I found ChatGPT helpful for suggesting ideas about what concepts a case can teach, as well as in evaluating how well certain questions align with certain concepts.

To do this, still for the TraceTogether case, I entered the following prompt:

“Let’s take question one. Can you suggest some academic concepts or frameworks that a discussion of this question could illuminate?”

ChatGPT generated a list of six concepts in response. This offered a good starting point for evaluating the relevance of different frameworks and for determining how to fold those concepts into the case discussion. The AI suggested privacy by design as one idea among several others. It was a set of principles I was familiar with. While I had been using a different case to ask students to evaluate that approach, ChatGPT prompted me to also consider using it here.

4. Source examples to illustrate case concepts

ChatGPT can also help in creating relevant examples of concepts you want to teach. Imagine that I did decide to use this case discussion to introduce the concept of privacy by design. To broaden the case discussion from just this one episode, I might want to mention one or two companies that deploy elements of privacy by design.

I used the following prompt to obtain some potential examples:

“Can you suggest three real examples of businesses that used privacy by design as they built or rolled out products and services?”

ChatGPT responded with three companies and a brief explanation of how each might exemplify privacy by design.

Again, I wouldn’t recommend uncritically walking into class with these examples. ChatGPT can “hallucinate”—state false information in a manner that sounds true—so we do need to use our own expertise and research to verify what it says. My own preliminary review confirmed some elements of ChatGPT’s three examples in this instance, and with some additional verification, more context, more nuance, and with citation, I could bring them into class to use with my students.

5. Simulate live case discussions

Case teaching requires a great deal of thinking on your feet. But with an online tool capable of responding to your questions, you can simulate a real-time class discussion, making it possible to anticipate some student responses ahead of time and to be ready with sharpened follow-up questions in class.

ChatGPT will not speak to you exactly like your students do. And case method teachers should always remain curious to what each student will uniquely say. But ChatGPT can give you a good idea of what thoughts or perspectives students may have. And these ideas may be especially helpful if you are teaching with a new case or to a new group of students.

To understand how, here is a prompt I used to create a simulation of a discussion about TraceTogether :

“I’d like you to help me practice teaching this session on TraceTogether . I am going to ask you a question. Please create three student personas: Worm Deck, Garden Deck, and Sky Deck. Please pick one of them at random to answer the question. Provide an answer from them. Then, ask two follow-up questions. Make the first an ‘exploration’ question, inviting them to expand on their answer in some way. Provide their answer. Make the second a ‘challenge’ question, pushing back on their responses, perhaps by citing contrary evidence. Provide their answer. Then, pick one of the other students to respond to what they heard from the first student. After that, ask the final student to respond to what they heard from the first student. The question is, Was Bay correct to keep government officials ‘in the loop’ in the digital contact tracing system he built?”
“Case teaching requires a great deal of thinking on your feet. But with an online tool capable of responding to your questions, you can simulate a real-time class discussion, making it possible to anticipate some student responses ahead of time.”

This prompt helped ChatGPT create the discussion I wanted to simulate. After I hit generate, I watched the conversation progress. Seeing the student responses that ChatGPT generated gave me an idea of potential answers my students may provide. Moreover, the follow-up questions ChatGPT suggested gave me more ideas for my own follow-up questions. Perhaps most importantly, asking ChatGPT to pose “exploration” and “challenge” follow-up questions reminded me that when class time comes, I should be doing the same.

One of the richest ways to prepare for class is to put yourself in your students’ shoes. You can ask the AI at the end of a simulated discussion like this to finally ask you a question based on the conversation so far. For example, I could prompt it to ask me to weigh on Sky Deck’s response to Worm Deck. Forced (so to speak) to contribute to the conversation, you’ll have an even more concrete sense for whether the learning pasture you set up pushed productive thinking and growth.

Elevate your case prep for more engaged learning

I am not turning over my case teaching to ChatGPT. I’m still spending and enjoying much time thinking about learning objectives and session architecture. And I remain forever curious about what my students will think and say.

But I am using ChatGPT as a metaphorical teaching partner. And it is helping me sharpen my questions and enrich my sessions. I’ve seen how more challenging questions elevate our discussions; I’ve witnessed how additional examples and frameworks make the impact of those discussions longer lasting.

Of course, I am staying aware of generative AI’s limitations. I’m staying observant as these tools evolve for better and worse and as our use of them does, you should, too. When I use them as an aid, I disclose this in my class commentary and in footnotes on my slides. And I share what I am doing with these tools with colleagues and counterparts, as I am here, so that we may all have a better understanding of how they might—and might not—be used to enhance teaching and learning.

But all in all, nine years of case method teaching has taught me that the best of it is the art of great question asking, and nine months of ChatGPT is helping me get better at that.

can chatgpt answer case study questions

Mitchell Weiss is the Richard L. Menschel Professor of Management Practice and chair of the MBA Required Curriculum at Harvard Business School.

Related Articles

STUDENT USE CASES FOR AI

We use cookies to understand how you use our site and to improve your experience, including personalizing content. Learn More . By continuing to use our site, you accept our use of cookies and revised Privacy Policy .

can chatgpt answer case study questions

Using ChatGPT for creating more compelling content for case study outlines

a robot in flat illustration style with gradients and white background

Creating compelling content for case studies can be a daunting task, but what if you had a little help from a language model? Enter ChatGPT, a state-of-the-art model developed by OpenAI that can generate human-like text. In this article, we'll explore how you can use ChatGPT to take your case study outlines to the next level and generate more engaging content for your audience. From brainstorming ideas to crafting the perfect introduction, ChatGPT can help you streamline the content creation process and produce more effective case studies. So, let's dive in and see how ChatGPT can boost your case study game!

Introduction to ChatGPT and its capabilities

ChatGPT, short for "Generative Pre-training Transformer", is a state-of-the-art language model developed by OpenAI. It's designed to generate human-like text, which means that it can be used to write anything from emails to articles, scripts and even poetry.

One of its key capabilities is its ability to complete text prompts. For example, you can give it a sentence or a few words and it will generate a whole paragraph or even an entire article based on that input. This makes it a powerful tool for content creation , as it can help you generate ideas, write introductions and even help you to complete an entire case study.

Another important capability of ChatGPT is its ability to understand context. This means that it can understand the meaning of the text it's generating and adjust its output accordingly. This allows ChatGPT to generate text that is consistent with the input and makes it more likely to be coherent and grammatically correct.

In summary, ChatGPT is a powerful language model that can generate human-like text, and it can be used to help with content creation, generate ideas, write introductions, complete case studies and understand context, which makes it a great tool for creating more compelling content for case study outlines.

How ChatGPT can be used in the case study outlining process

ChatGPT can be used in a variety of ways throughout the case study outlining process to help create more compelling content. One way is by using it to generate ideas for case studies. By giving ChatGPT a prompt about a particular industry or problem, it can generate a list of potential case study topics that you can use to guide your research.

Another way ChatGPT can be used is to write compelling introductions for case studies. By providing it with information about the case study topic and its main findings, ChatGPT can generate an engaging introduction that will capture the reader's attention and set the stage for the rest of the case study.

Additionally, ChatGPT can be used to streamline the content creation process for case studies. By using it to generate text for various sections of the case study, such as the background, methods, results, and conclusion, you can save time and focus on editing and fine-tuning the content.

Furthermore, ChatGPT can also assist in summarizing the case study findings by generating a summary of the case study's main findings and recommendations.

Overall, ChatGPT can be a valuable tool in the case study outlining process, by providing ideas, writing introductions, streamlining the content creation process, and summarizing the findings. It can help you to generate more compelling content and save time in the process.

Using ChatGPT to generate ideas for case studies

Using ChatGPT to generate ideas for case studies is a powerful way to jumpstart the content creation process. The model can help you come up with new and unique perspectives on a particular industry or problem that you might not have thought of on your own.

The process of using ChatGPT to generate ideas for case studies is relatively straightforward. First, you'll need to provide the model with a prompt that describes the industry or problem that you're interested in. For example, if you're interested in the healthcare industry, you might provide ChatGPT with a prompt that reads something like "What are some potential case study ideas related to healthcare?"

Once you've provided the model with a prompt, it will generate a list of potential case study ideas that you can use to guide your research. These ideas will be based on the information you provided in the prompt, but they'll also be unique to the model's understanding of the topic.

It's important to note that the ideas generated by ChatGPT are not necessarily fully researched or validated, but it can provide a starting point to explore further. Additionally, you can use the model to generate ideas for different sections of the case study, like the problem statement, research questions, or hypotheses.

Overall, using ChatGPT to generate ideas for case studies can be a great way to come up with new and unique perspectives on a particular industry or problem, and provide a starting point for your research.

Using ChatGPT to write compelling case study introductions

Using ChatGPT to write compelling introductions for case studies can help to engage readers and set the stage for the rest of the content. The model can take the information provided to it and use it to generate an introduction that is both informative and engaging.

The process of using ChatGPT to write a case study introduction is similar to using it to generate ideas. First, you'll need to provide the model with a prompt that includes information about the case study topic and its main findings. For example, if you're writing a case study about the effects of a new medication on patients with a certain condition, you might provide ChatGPT with a prompt that reads something like "Write an introduction for a case study on the effects of X medication on patients with Y condition, highlighting the main findings".

Once you've provided the model with a prompt, it will generate an introduction for the case study. The introduction will provide a brief overview of the case study topic and its main findings and should capture the reader's attention and make them want to read more.

It's important to note that the introduction generated by ChatGPT may require editing and fine-tuning before it is ready to be used. The model can generate a general structure and language, but it's always important to review and adjust the text to make sure it's accurate and in the desired tone.

Overall, using ChatGPT to write compelling introductions for case studies can help to engage readers and set the stage for the rest of the content, providing a strong starting point for your case study.

ChatGPT's role in streamlining the content creation process for case studies

ChatGPT can play a significant role in streamlining the content creation process for case studies. By using it to generate text for various sections of the case study, such as the background, methods, results , and conclusion, you can save time and focus on editing and fine-tuning the content.

The process of using ChatGPT to streamline the content creation process for case studies is similar to using it to write an introduction. First, you'll need to provide the model with a prompt that includes information about the case study topic and its main findings. For example, if you're writing a case study about the effects of a new medication on patients with a certain condition, you might provide ChatGPT with a prompt that reads something like "Write the background section for a case study on the effects of X medication on patients with Y condition, highlighting the main research findings."

Once you've provided the model with a prompt, it will generate the text for the section you requested. This can be a great starting point for your case study, as it provides a basic structure and language for the section, but it's always important to review and adjust the text to make sure it's accurate and in the desired tone.

Additionally, ChatGPT can also be used to generate summaries of the case study's main findings and recommendations, which can make it easier to create an executive summary or conclusion.

It's important to note that ChatGPT can help to save time in the content creation process, but it's not a replacement for human expertise and knowledge of the subject matter. The generated text should always be reviewed, edited, and fact-checked before being used.

Overall, ChatGPT can play a significant role in streamlining the content creation process for case studies by providing a starting point for the different sections of the case study, save time in the process and allow you to focus on editing and fine-tuning the content.

Examples of using ChatGPT in real-world case study projects

ChatGPT can be used in a variety of real-world case study projects to help streamline the content creation process and create more compelling content. Some examples of how it can be used include:

Generating ideas for case studies: ChatGPT can be used to generate a list of potential case study topics based on a specific industry or field. For example, a marketing agency might use the model to generate a list of potential case studies for their clients in the tech industry.

Writing introductions: As previously mentioned, ChatGPT can be used to write compelling introductions for case studies that engage readers and set the stage for the rest of the content.

Creating summaries: ChatGPT can be used to generate summaries of the main findings and recommendations for case studies, which can make it easier to create an executive summary or conclusion.

Streamlining the content creation process: ChatGPT can be used to generate text for various sections of the case study, such as the background, methods, results, and conclusion. This can save time and allow you to focus on editing and fine-tuning the content.

Generating reports: ChatGPT can be used to generate reports for various industries, such as finance, healthcare, and marketing. The model can take the information provided to it and use it to generate a report that is both informative and engaging.

It's important to note that the examples provided above are not exhaustive and that ChatGPT's capabilities are constantly evolving. The model can be utilized in many other ways as well, and its capabilities are expanding as it's being trained on more data and with more advanced techniques.

Overall, ChatGPT can be used in a variety of real-world case study projects to help streamline the content creation process and create more compelling content, providing a wide range of possibilities to enhance the case studies.

Best practices for using ChatGPT in case study content creation

When using ChatGPT for case study content creation, there are several best practices that can help ensure the generated text is accurate, engaging, and relevant to the case study topic. Some of these best practices include:

Providing clear prompts: When using ChatGPT to generate text for a case study, it's important to provide clear and detailed prompts that specify the section of the case study you want the model to focus on and any key information or findings that should be included in the text.

Reviewing and editing the generated text: While ChatGPT can generate text quickly, it's important to review and edit the text to ensure it's accurate and in the desired tone. The generated text should always be fact-checked and edited for grammar, punctuation, and style before being used in the case study.

Using specific examples and data: To make the generated text more engaging and relevant to the case study topic, it's important to provide ChatGPT with specific examples and data to include in the text. This will help the model generate text that is tailored to the case study and more likely to engage readers.

Fine-tuning the model: To improve the accuracy and relevance of the generated text, it's possible to fine-tune the model by providing it with a specific dataset that is related to the case study topic. This will help the model understand the subject matter and generate text that is more relevant to the case study.

Incorporating human expertise: While ChatGPT can help to save time in the content creation process, it's not a replacement for human expertise and knowledge of the subject matter. The generated text should always be reviewed, edited, and fact-checked by a human before being used in the case study.

By following these best practices, you can ensure that the generated text is accurate, engaging, and relevant to the case study topic, and that it can be seamlessly integrated into the case study.

Overall, ChatGPT can be an extremely useful tool for creating case study content, but it's important to use it correctly and to follow best practices to ensure the generated text is accurate, engaging, and relevant to the case study topic.

Potential limitations and considerations when using ChatGPT for case studies

When using ChatGPT for case studies, there are a few limitations and considerations to keep in mind to ensure the generated text is accurate, engaging, and relevant to the case study topic. Some of these limitations and considerations include:

Lack of domain-specific knowledge: While ChatGPT has been trained on a vast amount of text, it may not have specific knowledge of the subject matter of the case study. This can result in generated text that is not entirely accurate or relevant to the case study topic.

Bias in the training data: ChatGPT is trained on a massive dataset of text from the internet, which may contain bias. This could result in generated text that is not inclusive or that perpetuates stereotypes.

Dependence on the quality of the input: The quality of the generated text is directly related to the quality of the input provided to the model. If the input is unclear, incomplete, or not specific enough, the generated text may not be accurate or relevant to the case study topic.

Lack of creativity: While ChatGPT can generate text quickly, it may not be able to come up with truly creative or original ideas. The generated text may be formulaic and lack the kind of innovation that can make a case study truly stand out.

Ethical considerations: Generating text using GPT-3 has raised ethical considerations such as the potential for misuse of the technology, the impact of AI-generated content on employment, and the potential for AI-generated content to spread misinformation.

It's important to keep these limitations in mind when using ChatGPT for case studies, and to always review, edit, and fact-check the generated text before using it in the case study. Additionally, it's important to be aware of the ethical considerations surrounding the use of GPT-3 and AI-generated content in general.

Overall, ChatGPT can be a powerful tool for creating case study content, but it's important to be aware of its limitations and to use it responsibly. By keeping these limitations and considerations in mind, you can ensure that the generated text is accurate, engaging, and relevant to the case study topic.

Future possibilities for using ChatGPT in case study content creation

The possibilities for using ChatGPT in case study content creation are quite exciting, as the technology continues to evolve and improve. Some future possibilities include:

Improving the accuracy of generated text: As the technology behind ChatGPT continues to advance, the generated text is likely to become more accurate and relevant to the case study topic. This could lead to more compelling case studies that are more effective at engaging readers.

Personalization of generated text: ChatGPT could be used to create personalized case studies tailored to specific audiences. For example, the model could be trained on data from a particular industry or target demographic, resulting in generated text that is more relevant to that audience.

Greater collaboration between humans and AI: As ChatGPT becomes more advanced, it could be used in collaboration with human writers to generate case study content. The AI could be used to generate ideas and initial text, while human writers could edit and refine the text to create a final product that is both engaging and accurate.

Using AI to analyze and interpret data: ChatGPT could be used to analyze and interpret large amounts of data in order to identify key insights and trends that could be used to generate case studies. This could allow for the creation of case studies that are more data-driven and evidence-based.

Enhancing the multimedia aspect of case studies: With the integration of GPT-3 with other AI models, it may be possible to generate multimedia content like videos, images, and audio to enhance the case studies.

Overall, the future possibilities for using ChatGPT in case study content creation are quite exciting. As the technology continues to evolve, it could lead to more compelling and effective case studies that are better able to engage readers and achieve the desired results.

Conclusion and next steps for using ChatGPT in case study outlining and content creation

In conclusion, ChatGPT is a powerful tool that can be used to streamline the process of creating case study outlines and content. It has the ability to generate ideas, write compelling introductions, and even analyze data to identify key insights.

However, as with any technology, there are also potential limitations and considerations to keep in mind. It is important to understand the capabilities and limitations of the tool, and to have a clear strategy in place for how it will be used in the case study outlining and content creation process.

One next step for organizations or individuals interested in using ChatGPT for case studies could be to experiment with the tool by training it on their own data and testing its capabilities. This could include generating text on a specific case study topic, analyzing data, or even creating multimedia content.

Another next step could be to explore the integration of ChatGPT with other AI models, like image recognition, video generation, and audio generation that could enhance the case studies.

In addition, it's essential to keep in mind the ethical considerations of using AI-generated content, such as transparency and accountability.

Overall, ChatGPT has the potential to be a valuable tool for creating more compelling case studies, but it's important to approach it with a clear strategy and understanding of its capabilities and limitations. With careful planning and implementation, organizations and individuals can leverage the power of ChatGPT to create effective and engaging case studies that can drive business results.

Over to you

ChatGPT is a powerful language generation tool that can be used to streamline the process of creating case study outlines and content. It has the ability to generate ideas, write compelling introductions and analyze data to identify key insights. By using ChatGPT, organizations and individuals can create more compelling and effective case studies that are better able to engage readers and achieve desired results.

However, it's important to understand the tool's capabilities and limitations and have a clear strategy in place for its use. Experimenting with the tool by training it on own data and testing its capabilities, exploring integration with other AI models and keeping in mind the ethical considerations are some next steps for organizations and individuals interested in using ChatGPT for case studies.

Want boost your traffic with AI-generated content?  Start for free.

Have a language expert improve your writing

Check your paper for plagiarism in 10 minutes, generate your apa citations for free.

  • Knowledge Base

Using AI tools

  • Using ChatGPT for Assignments | Tips & Examples

Using ChatGPT for Assignments | Tips & Examples

Published on February 13, 2023 by Jack Caulfield and Tobias Solis. Revised on November 16, 2023.

People are still figuring out the best use cases for ChatGPT , the popular chatbot based on a powerful AI language model. This article provides some ideas for how to use ChatGPT and other AI tools to assist with your academic writing .

These tools can assist you with the following steps when writing a paper, thesis , or dissertation :

  • Developing a research question
  • Creating an outline
  • Generating literature ideas

Paraphrasing and summarizing text

  • Getting feedback

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

Developing a research question with chatgpt, creating an outline of your paper with chatgpt, generate literature ideas, getting feedback from chatgpt, limitations of chatgpt, other interesting articles, frequently asked questions.

You can use ChatGPT to help you formulate a clear and concise research question by generating and improving potential questions.

With the help of ChatGPT, you can develop a large number of possible research questions within a short time. This is a great way to generate ideas and different approaches to a topic.

However, make sure that the generated questions make sense and meet your needs. As you can see from our example, you can keep working with ChatGPT to adapt its suggestions to your needs (or just edit the text directly if you already have a clear idea of what you want).

First input

Develop three possible research questions on the following topic: “The influence of social media on political opinion formation.”

ChatGPT output

  • To what extent does exposure to diverse political perspectives on social media affect individuals’ political opinions and behaviors?
  • How does the personalized algorithm of social media platforms shape the formation of political opinions among its users?
  • What are the underlying factors that determine the effect of social media on political opinion formation, and how can they be measured and analyzed?

Second input

Based on the second research question, develop three further research questions on the topic of polarization in US politics.

  • Build my scientific method, qualitative discourse analysis, into each question
  • Include the term “filter bubble” in each question
  • How does the filter bubble effect on social media platforms contribute to the polarization of political attitudes in the US? A qualitative discourse analysis.
  • What is the role of filter bubbles in perpetuating and reinforcing political echo chambers in the US? A qualitative discourse analysis.
  • How does the influence of filter bubbles on social media affect the ability of individuals to engage in productive political discourse across ideological divides in the US? A qualitative discourse analysis.

Third input

Try to incorporate “qualitative discourse analysis” into the questions themselves, instead of using it as a separate sentence.

  • How does a qualitative discourse analysis of filter bubble effect on social media platforms contribute to understanding the polarization of political attitudes in the US?
  • What is the contribution of a qualitative discourse analysis to understanding the role of filter bubbles in perpetuating and reinforcing political echo chambers in the US?
  • How does a qualitative discourse analysis of the influence of filter bubbles on social media inform our understanding of the ability of individuals to engage in productive political discourse across ideological divides in the US?

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

can chatgpt answer case study questions

Try for free

You can also use ChatGPT to help you draft a research paper outline or thesis outline . To do this, try generating possible headings and subheadings and then improving them. ChatGPT can help to generate a clear and well-structured outline, especially if you keep adjusting the structure with its help.

However, it doesn’t have a deep understanding of your topic. As a result, inappropriate headings may sometimes be generated. It’s important to check that the outline makes sense and covers all the points needed to answer your research question.

You should also ensure that the headings and subheadings follow a logical structure. Scribbr’s Proofreading & Editing service can help you with this.

Create an outline for an academic research paper with the following research question:

“What is the contribution of a qualitative discourse analysis to understanding the role of filter bubbles in perpetuating and reinforcing political echo chambers in the US?”

The paper will be about 8 pages long.

  • Background information on filter bubbles and political echo chambers
  • Purpose of the study
  • Research question
  • Thesis statement
  • Definition and concept of filter bubbles
  • Definition and concept of political echo chambers
  • Previous studies on the role of filter bubbles in perpetuating and reinforcing political echo chambers
  • Critiques and limitations of previous studies
  • Research design
  • Participants
  • Data collection and analysis
  • Validity and reliability
  • Findings from qualitative discourse analysis
  • Results related to the research question
  • Implications of the findings
  • Interpretation of the results
  • Comparison with previous studies
  • Limitations of the study
  • Implications for future research
  • Summary of the results
  • Implications for understanding the role of filter bubbles in perpetuating and reinforcing political echo chambers
  • Conclusion and recommendations for future research
  • List of references used in the paper
  • Data collection materials
  • Examples of qualitative data analysis

This seems like too much information to fit into only 8 pages. Can you condense the outline, for example removing the appendices and simplifying the structure?

  • Background on filter bubbles and political echo chambers
  • Definition of filter bubbles and political echo chambers
  • Previous studies on the role of filter bubbles
  • Limitations of previous studies
  • Implications for understanding the role of filter bubbles
  • Final thoughts

ChatGPT can be a useful way to generate ideas regarding what sources you might use in your work. Try asking it for suggestions about where to look and what to consider.

However, we advise against using it to find sources directly. It’s not designed for this and, if asked to do it, often just creates fake sources and presents them as real. Search a database like Google Scholar instead.

What types of sources could I use to write a paper on the following research question?

When writing a paper on the contribution of qualitative discourse analysis to understanding the role of filter bubbles in perpetuating and reinforcing political echo chambers in the US, you could use the following types of sources:

  • Academic journals and articles: These are a good place to start when you’re looking for information on specific topics and they can provide you with in-depth analysis and perspectives on the topic at hand.
  • Books and book chapters: These sources can provide comprehensive overviews of a particular topic and can also offer theoretical frameworks and historical perspectives.
  • Government reports and studies: Government agencies, such as the Federal Communications Commission (FCC), may have conducted studies on the role of filter bubbles in perpetuating political echo chambers.
  • News articles and media reports: These sources can provide insight into the current state of the issue, as well as opinions from various stakeholders.
  • Online sources: Websites, blogs, and online forums can provide a range of perspectives on the topic and can also be useful in exploring the experiences of individuals who are affected by filter bubbles and political echo chambers.
  • Data and statistics: Relevant data and statistics on the prevalence and impact of filter bubbles and political echo chambers in the US can be useful in providing evidence to support your arguments.
  • Expert interviews and focus groups: Conducting expert interviews or focus groups with individuals who have experience with or knowledge of the topic can provide valuable insights and perspectives that can be used to support your arguments.

You can also try using ChatGPT to paraphrase or summarize text. ChatGPT can quickly condense the text or generate a number of different paraphrases of the same idea. You can keep typing “again” in the chat until you are satisfied with the result.

This kind of paraphrasing/summarizing can be helpful to, for example, non-native speakers who want to improve the fluency of their text. However, be careful. The outputs often:

  • Contain grammatical errors
  • Include false information

So you should always check the output to make sure it’s correct. If you’re unsure, you can run the text through the Scribbr Grammar Checker and cross-reference the information against other, more credible sources .

An alternative is to use the Scribbr Paraphrasing Tool or Scribbr’s free text summarizer , which were designed specifically for these purposes.

Paraphrase the following text:

“A filter bubble results from a social media algorithm’s attempts to cater to an individual user based on data such as their search history and previous behavior on the site. It results in the user only being presented with information that confirms their biases.”

Restructure the sentences and use other phrasings for the following parts:

  • “results from”
  • “only being presented with”

“A filter bubble is created when a social media algorithm customizes its content for a particular user based on their data, such as their search history and past actions on the platform. This leads to the user being confined to information that reinforces their pre-existing beliefs.” Note We strongly advise against having ChatGPT write text for you and trying to pass it off as your own work. This is academically dishonest and is widely considered plagiarism . It will usually result in an automatic fail, as well as other potential consequences .

Check for common mistakes

Use the best grammar checker available to check for common mistakes in your text.

Fix mistakes for free

You can also use ChatGPT to get feedback on the argumentation and structure of your paper and suggestions for improvements.

An advantage of this is that you can get feedback from ChatGPT on different text excerpts at any time. However, one downside is that ChatGPT does not have a deep understanding of your research question, so its feedback and suggestions may be shallow or inappropriate.

This kind of feedback cannot replace input from an experienced academic supervisor who’s familiar with your topic. Use ChatGPT feedback as a supplement, not a substitute for human feedback.

If you need in-depth feedback on your writing from someone with academic expertise, try Scribbr’s Proofreading & Editing service .

[Text you want to get feedback on.]

Thank you for submitting your research paper for review. Let me provide you with my constructive criticism:

Clarity of Explanations: Overall, the explanations are clear and concise, but there are a few areas that could benefit from additional clarification. For example, in the first paragraph, you mention “the mental lexicon,” but do not provide a definition or further explanation for what this is. It would also be helpful to provide a definition of “lemma.”

Smoothness of Language: The language in the paper is generally smooth, but there are a few instances where it could be refined. For example, the sentence “The symptoms vary from adding superfluous words and taking pauses to hoarseness of the voice” could be rephrased to “The symptoms of these disorders can range from excessive use of words and pauses to hoarseness.”

Structure of Text: The structure of the text is clear, with each paragraph building on the previous one. However, the transition between the first two paragraphs could be smoother. Perhaps you could add a sentence or two to bridge the gap between the two topics.

Even though ChatGPT is a powerful writing tool, you should keep the following limitations in mind:

No original research

Risk of plagiarism, fake sources, fake “facts”.

ChatGPT was trained using existing texts, and the texts created by the AI are based on this data. It’s not capable of doing its own research, only of synthesizing sources that it was trained on.

Additionally, because the texts that it was trained on only go up to 2021, it’s unfamiliar with more recent world events, cultural trends, and developments in different fields of research. This means it may provide outdated information.

Because ChatGPT was trained using existing text, the AI tends to reuse that content in a way that may be considered plagiarism. The AI may copy text from an online publication in its responses.

Additionally, using ChatGPT to write your assignment for you is considered plagiarism regardless of where it got the content from, because you’re passing off something you didn’t write as your own work.

ChatGPT was trained on a wide variety of sources, but it’s not able to accurately tell you what sources it’s using for any given output. Worse, it has a tendency to make up sources that don’t actually exist when you ask.

In addition, ChatGPT is not designed as a citation generator and cannot add correctly formatted citations to your work. Instead, try a tool designed for this purpose, like the Scribbr Citation Generator .

ChatGPT is not always trustworthy when it comes to giving you factually correct information. While it doesn’t deliberately lie, it also doesn’t have “knowledge” in a traditional sense, and it lacks the ability to verify its claims.

Instead, its responses are based on recognizing and reproducing patterns that it has seen in other texts. This means that if it doesn’t “know” something, it tends to guess. And its guesses are presented with full confidence, just like established facts.

So it’s very important that you carefully check any factual claims made by ChatGPT against more credible sources —and don’t cite ChatGPT as a source of factual information for your research.

If you want more tips on using AI tools , understanding plagiarism , and citing sources , make sure to check out some of our other articles with explanations, examples, and formats.

  • Citing ChatGPT
  • Best grammar checker
  • Best paraphrasing tool
  • ChatGPT in your studies
  • Deep learning
  • Types of plagiarism
  • Self-plagiarism
  • Avoiding plagiarism
  • Academic integrity
  • Best plagiarism checker

Citing sources

  • Citation styles
  • In-text citation
  • Citation examples
  • Annotated bibliography

You can use ChatGPT to assist in the writing process for your research paper , thesis , or dissertation in the following ways:

  • Paraphrasing text

No, it’s not a good idea to do so in general—first, because it’s normally considered plagiarism or academic dishonesty to represent someone else’s work as your own (even if that “someone” is an AI language model). Even if you cite ChatGPT , you’ll still be penalized unless this is specifically allowed by your university . Institutions may use AI detectors to enforce these rules.

Second, ChatGPT can recombine existing texts, but it cannot really generate new knowledge. And it lacks specialist knowledge of academic topics. Therefore, it is not possible to obtain original research results, and the text produced may contain factual errors.

However, you can usually still use ChatGPT for assignments in other ways, as a source of inspiration and feedback.

No, it is not possible to cite your sources with ChatGPT . You can ask it to create citations, but it isn’t designed for this task and tends to make up sources that don’t exist or present information in the wrong format. ChatGPT also cannot add citations to direct quotes in your text.

Instead, use a tool designed for this purpose, like the Scribbr Citation Generator .

But you can use ChatGPT for assignments in other ways, to provide inspiration, feedback, and general writing advice.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. & Solis, T. (2023, November 16). Using ChatGPT for Assignments | Tips & Examples. Scribbr. Retrieved March 20, 2024, from https://www.scribbr.com/ai-tools/chatgpt-assignments/

Is this article helpful?

Jack Caulfield

Jack Caulfield

Other students also liked, writing strong research questions | criteria & examples, how to create a structured research paper outline | example, student guide: information literacy | meaning & examples.

Jack Caulfield

Jack Caulfield (Scribbr Team)

Thanks for reading! Hope you found this article helpful. If anything is still unclear, or if you didn’t find what you were looking for here, leave a comment and we’ll see if we can help.

Still have questions?

Unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

  • Open access
  • Published: 22 February 2023

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

  • Ahmed Tlili   ORCID: orcid.org/0000-0003-1449-7751 1 ,
  • Boulus Shehata   ORCID: orcid.org/0000-0001-9445-6559 1 ,
  • Michael Agyemang Adarkwah   ORCID: orcid.org/0000-0001-8201-8965 1 ,
  • Aras Bozkurt   ORCID: orcid.org/0000-0002-4520-642X 2 ,
  • Daniel T. Hickey   ORCID: orcid.org/0000-0001-9146-5089 3 ,
  • Ronghuai Huang   ORCID: orcid.org/0000-0003-4651-5248 1 &
  • Brighter Agyemang   ORCID: orcid.org/0000-0002-5050-8916 4  

Smart Learning Environments volume  10 , Article number:  15 ( 2023 ) Cite this article

95k Accesses

267 Citations

35 Altmetric

Metrics details

Artificial Intelligence (AI) technologies have been progressing constantly and being more visible in different aspects of our lives. One recent phenomenon is ChatGPT, a chatbot with a conversational artificial intelligence interface that was developed by OpenAI. As one of the most advanced artificial intelligence applications, ChatGPT has drawn much public attention across the globe. In this regard, this study examines ChatGPT in education, among early adopters, through a qualitative instrumental case study. Conducted in three stages, the first stage of the study reveals that the public discourse in social media is generally positive and there is enthusiasm regarding its use in educational settings. However, there are also voices who are approaching cautiously using ChatGPT in educational settings. The second stage of the study examines the case of ChatGPT through lenses of educational transformation, response quality, usefulness, personality and emotion, and ethics. In the third and final stage of the study, the investigation of user experiences through ten educational scenarios revealed various issues, including cheating, honesty and truthfulness of ChatGPT, privacy misleading, and manipulation. The findings of this study provide several research directions that should be considered to ensure a safe and responsible adoption of chatbots, specifically ChatGPT, in education.

Introduction

Can machines think? is a simple, yet a sophisticated question (Turing, 1950 ). In an effort to find an answer to this question, McCarthy et al. ( 1955 ) organized a scholarly event and coined the term "artificial intelligence” (AI) in 1955 to refer to machines and processes that imitate human cognition and make decisions like humans. At these times, the term [ro]bots are articulated for the first time in Čapek’s ( 1921 ) science fiction play; however, it was Asimov ( 1942 , 1950 ) who visioned that these machines can transform into intelligent forms and introduced the Three Laws of Robotics to set the rules that bots should stick to and cannot be bypassed. Originally known as the imitation game, the Turing Test was proposed as a code of protocol to understand whether a machine can exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human (Turing, 1950 ). Once depicted as fiction, all those possibilities are about to come true, and we are at the brink of a future when we can know whether machines can think or not.

In November 2022, OpenAI, a lab that studies artificial intelligence, came out with a chatbot called ChatGPT (Generative Pre-trained Transformer). ChatGPT is a conversational artificial intelligence interface that uses natural language processing (NLP), which interacts in a realistic way and even “answers follow-up questions, admits its mistakes, challenges incorrect premises, and rejects inappropriate requests'' (OpenAI, 2023 ). While ChatGPT's primary function was to mimic human conversation, its capabilities extend far beyond that; it can literally create new things, such as a poem, story, or novel, or act like anything within its capability.

With the advent of ChatGPT, there is eventually an innovative AI technology that will truly challenge the Turing Test (Turing, 1950 ) and demonstrate if it is capable of thinking like humans. It is uncertain if it would pass the Turing Test (Turing, 1950 ) in the long run, but it is sure that ChatGPT is revolutionary as a conversational AI-powered bot, and it is a visible signal for the paradigm shift that has been happening not only in the educational landscape, but also in every dimension of our lives. Compared to traditional chatbots, ChatGPT is based on GPT-3, which is the third iteration of the GPT series by OpenAI that is more advanced in terms of scale (175 billion parameters, compared to 1.5 billion of GPT-2), larger dataset as the training data, more fine-tuning, enhanced capabilities, and more human-like text generations (Brown et al., 2020 ). The use of Natural Language Processing and a generative AI that relies on deep learning has enabled ChatGPT to produce human-like text and maintain a conversational style allowing more realistic natural dialogues.

Several preprints of studies and numerous blog posts and media outlets have reported the advantages of ChatGPT in education (Zhai, 2022 ); some have even provided guidelines on using it in classrooms (Lieberman, 2023 ; Mollick & Mollick, 2022 ; Ofgang, 2022 ). However, the potential concerns of chatbots haven’t been investigated as much. Janssen et al. ( 2021 ) described reasons for chatbots’ failure in practice, including not having enough resources, wrong use case (i.e., the basic chatbot technology did not match the required task), poor law regulations, data security, and liability concerns, ignorance of user expectation and bad conversation design, or simply poor content. Haque et al. ( 2022 ) did a Twitter sentiment analysis about ChatGPT adoption as a technology in general (not in education), and they found that users have divided attitudes about it. However, concerns coming from an advanced chatbot, such as ChatGPT, were not well investigated in the education field. Therefore, it is not clear if ChatGPT will overcome the concerns found in previous chatbots or will even deepen them. Consequently, this may lead to a serious and quick protective reaction to a potential opportunity, such as New York City and Los Angeles Unified schools’ banning of ChatGPT from educational networks due to the risk of using it to cheat in assignments (Shen-Berro, 2023 ; The Guardian, 2023 ). It is therefore important to investigate the concerns of using this technology, ChatGPT, in education to ensure safe use. The purpose of this study is, therefore, to examine chatbots in education and for this purpose, the study approaches ChatGPT as a representative case of an advanced chatbot among early adopters. In this regard, this study answers the following research question: What are the concerns of using chatbots, specifically ChatGPT, in education?

Methodology

To answer the aforementioned research question, this study adopts a qualitative case study approach (Yin, 1984 ) and benefits from an instrumental case study research design (Stake, 1995 ). Instrumental research design is helpful when researchers intend to understand a phenomenon in a context (Stake, 1995 ), which is in our case, ChatGPT which is a fine and recent example of AI-powered chatbots. To ensure the validity and reliability of the study, the research triangulates (Thurmond, 2001 ) the data collection tools to get a broader and deeper understanding. In this regard, this study follows three stages, namely, social network analysis of tweets, content analysis of interviews, and investigation of user experiences. Each of the stages is described in the next subsequent sections.

Social network analysis of tweets

Tweet analysis aims to understand the public discourse on the use of ChatGPT in education. Specifically, a cross-sectional analysis of tweets was conducted through Social Network Analysis (SNA) (Hansen et al., 2010 ). Specifically, from December 23, 2022, to January 6, 2023, 2330 tweets, from 1530 Twitter users, were collected and analyzed containing the following search string: “#ChatGPT* AND (education OR teaching OR learning)”. The dataset was compiled through social network analysis (Hansen et al., 2010 ) and the content of the tweets was further examined through sentiment analysis (Giachanou & Crestani, 2016 ) and tSNE analysis (van der Maaten & Hinton, 2008 ).

Content analysis of interviews

Interview analysis was conducted to investigate how different stakeholders (students, educators, etc.) perceive the use of ChatGPT in education, especially the concerns they have about it. Specifically, 19 interviewees, who have been using ChatGPT in education and posting their experiences through blogs publicly, were recruited from their channels. The interviewer had a long experience of using chatbots in education, and specifically had been using ChatGPT for at least one month. For the interviewees, despite that they were carefully chosen for this study to ensure the reliability of the findings, we further asked them to rate their familiarity with Chatbots, on a scale from 1 to 5 (where 1 is not familiar and 5 is very familiar). The interviews had an average of 3.02 related to the familiarity with chatbots, which reflect their appropriateness for this study. The interviewees were selected with various working backgrounds, such as educators, developers, students, and AI freelancers, to ensure the solicitation of rich answers from each one’s perspective. To analyse the collected interviews, content analysis, which is one of the classical procedures for analysing textual materials, was used (Flick, 2009 ). The analysis was based on the steps proposed by Erlingsson and Brysiewicz ( 2017 ). Particularly, two coders read the given interview results before they start coding them based on the developed coding scheme in Table 1 .

Investigation of user experiences

User experience aims to conduct hands-on experiences of using ChatGPT and identify potential concerns that might be faced when using it in education. User experience involves human perceptions and responses that result from the use of a product, system, or service. User Experience points to a more global projective goal: not just attain effectiveness, efficiency, and satisfaction, but it aims to enhance the entire experience of the user, from the expectation, through interaction and finally the reflection about the experience (Beccari & Oliveira, 2011 ). In this context, three experienced educators have used ChatGPT for a whole week to test similar and different teaching/learning scenarios, and then see the obtained results accordingly. In this context, daily meetings during the whole week were conducted between these educators to discuss and summarize the obtained results.

The obtained results were structured according to each stage as discussed in the following subsequent sections.

The overall aim of social network analysis is to learn more about public discourse regarding the use of ChatGPT from the perspective of educational purposes. Figure  1 shows tweets analysis using the Harel-Koren Fast Multiscale algorithm , which is a fast multi-level graph layout that provides better visualizations (Harel & Koren, 2001 ). Specifically, the edge colors, opacities, and widths are based on edge weight values. The node sizes are based on betweenness centrality values. Each interaction (e.g., retweets, mentions, likes) is identified as a relation and visualized as an edge. While some of the sub-clusters demonstrate that some participants gathered around some ideas, the overall network is composed of isolated nodes (e.g., see the largest cluster on the upper left side of Fig.  1 ). Accordingly, Fig.  1 shows a fragmented brand cluster pattern (Rainie, 2014 ; Smith et al., 2014 ) implying that the community formation about ChatGPT is fragmented, and individuals are seeking more information and discussion about its limitations and promises by tethering some influencer nodes in the ChatGPT network.

figure 1

Bird-view of the ChatGPT network

The most used word pairs also provide interesting insights. For instance, some suggest how to use AI-powered ChatGPT (e.g., education—chatgpt, education—learning, focused—grading, etc.), and some others hint that educational systems are in a turning point (e.g., existential—crisis, kind—ironic, crisis—happening, forgotten—purpose, etc.). The general public's view on the use of chatbots, more specifically ChatGPT, is diverse and there is no collective consensus on whether it is a hype or a future opportunity. However, the sentiment analysis of tweets (Giachanou & Crestani, 2016 ), demonstrates that the positive sentiments (5%) outweigh the negative sentiments (2.5%) (see Table 2 ). The fact that non-categorized sentiments (92.5%) are in the majority can be considered as an indicator that most people are undecided about ChatGPT in education.

The positive and negative sentiments are clearly reflected in some tweets with high edge weight values (Hansen et al., 2010 ). An example of positive sentiments is:

As a language model trained by OpenAI, I'm constantly amazed by the power & potential of artificial intelligence. From natural language processing to machine learning, AI is revolutionizing the way we think about & interact with technology. #AI #machinelearning #openai #ChatGPT

An example of negative sentiments is:

Here's my problem with this line of thinking about #ChatGPT as a writing instructor. Reactionary teaching goes nowhere.

An example of non-categorized sentiment is:

“Teachers are talking about ChatGPT as either a dangerous medicine with amazing side effects or an amazing medicine with dangerous side effects.” —@VicariousLee. Stanford faculty weigh in on #ChatGPT's shake-up in education https://t.co/Xx774bzeWm #edtech #edchat #gpt3 #ai https://t.co/dz4MEQD3XH

The word cluster of the most frequent 100 terms from the tweets (see Fig.  2 ), using tSNE analysis was applied. t-SNE is an unsupervised “nonlinear dimensionality reduction technique that aims to preserve the local structure of data” (van der Maaten & Hinton, 2008 , p. 2580), used for exploring and visualizing high dimensional data. The findings revealed that most of the users are optimistic about the use of AI-powered chatbots, such as ChatGPT in the educational systems. While the blue cluster in Fig.  2 demonstrates the future promises of using ChatGPT (e.g., see the terms: ChatGPT, learning, AI, education, future, teaching, learn ), the pink cluster indicates insights regarding how to use it and its revolutionary potential (e.g., see the terms: gpt, 2023 , artificial, intelligence, human, think, and better, way, knowledge, technology, tools, student, teacher ), the green cluster shows critical insights (e.g., see the terms: cheating, change, ideas, create, problem, potential, ways, edtech ).

figure 2

Word cluster of tweets through tSNE analysis

The most frequently used relevant hashtags are #chatgpt #AI, #ArtificialIntelligence #education, #machinelearning, #deeplearning #edtech #openAI, and #python, which implies that there is a need to carefully examine the AI technologies (e.g., machine learning, deep learning) lying behind the ChatGPT. As seen in the sample tweets (see Table 3 ), despite that there is an optimistic overview of using ChatGPT in education, there are also some concerns regarding the use of such technologies in the educational landscape.

To summarize, the findings from the Social Network Analysis of tweets revealed that positive sentiments have shown almost as twice higher frequency than negative ones (see Table 2 ). However, the example tweets show that negative sentiments demonstrate deeper and critical thinking than the positive ones (see Table 3 ). This could be explained by the fact that most of the positive sentiments are led by the novelty effect of ChatGPT as a technology in education. On the other hand, the negative sentiments represent more critical concerns, hence a deeper and thorough thinking of why ChatGPT should be approached with caution.

The content analysis of interviews revealed that the users found ChatGPT very significant with a great value to revolutionize education, however, they raised several concerns at the same time. Their views are structured according to the five themes shown in Table 1 .

Educational transformation

Responses from a majority of the participants suggest that ChatGPT is efficacious in increasing the chances of educational success by affording users (teachers and students) baseline knowledge of various topics. Additionally, ChatGPT was recognized by the participants as efficient in providing a comprehensive understanding of varied (complex) topics in an easy-to-understand language. In this light, it can be argued that ChatGPT will lead to a paradigm shift in conventional approaches to instruction delivery and drive learning reform in the future pregnant with digital potential. For instance, one participant reported:

“I would use ChatGPT for two purposes: as a learning aid and in instructional design within the field of education. For students, ChatGPT can provide learners with model answers that can stimulate their understanding of various subject matters. Additionally, in terms of instructional design, ChatGPT can be a useful tool for teachers and educators to remind them of what knowledge and skills should be included in their curriculum, by providing an outline” (Assistant Professor of Instructional Technology, USA, familiarity is: 2).

Conversely, a few of the participants held an opposing view that the abuse of ChatGPT by learners can also diminish their innovative capacities and critical thinking. For instance, when learners are not motivated, the probability of seeking an easy-to-get solution is high as can be deducted from a statement from one participant.

“Sometimes when I have no inspiration for writing a thesis, I will choose to use this software to input the answers to the questions I want to know” (Student of Education, China, familiarity is: 4).

Response quality

Response quality is vital to the success and effective adoption of Chatbots for school operations. In this study, most of the participants evaluated the dialogue quality and the degree of accurate information ChatGPT provides as satisfactory. However, it was added that the conversational agent is prone to occasional errors and limited information (presently, as reported by OpenAI, the data ChatGPT provides is limited to 2021). That is, at most times, responses from ChatGPT were reasonable and reliable but were at times accompanied by misleading information. This indicates that the output quality of ChatGPT though acceptable needs to be enhanced. An example given by one participant (a programmer) is the generation of a wrong code that did not work properly when entered into a programming software. Nonetheless, the fewer errors of ChatGPT were praised by some participants as an efficient virtual assistant in constructing knowledge and products. For instance, one participant stated:

“The answers from ChatGPT can be somewhat accurate but not totally. For example, when I couldn’t figure out how to write codes for a specific problem, the answers are vague and cannot totally solve my problem. I need to figure it out by myself using the experience I had” (Student of Geography, China, familiarity is: 2).

A participant further elaborated that the quality of answers getting from ChatGPT depends on the quality of questions asked by the user saying:

“It depends on the type of questions that you ask. If it is too recent, then the answers won't be too good, because ChatGPT lacks context, if you do not provide it with questions that are specific enough then its answers wouldn’t be too good” (Developer, USA, familiarity is: 3).

Personality and emotions

A large body of the participants was impressed by the fluidity of their conversation with ChatGPT. The interactions with ChatGPT were deemed exciting and fun. Notwithstanding, it was acknowledged that it is yet to achieve full humanization because it is currently limited to a textual interface and cannot detect physical cues or motions of a user. Most participants felt the humaneness of ChatGPT needs to be improved, especially in terms of enhancing its social role, as one of the participants reported:

“I don't think it can be compared to a real human being, and what it offers is not comparable to what a real person would say through genuine empathy. And in dialogue, it would say "As an AI, I don't have the ability to love or feel emotions as humans do, but I am here to assist you with any question or task you have.” (Student of Nursing Research, UK, familiarity is: 3).

She further elaborated:

“Occasionally, however, when it comes to emotions, it can be a little disappointing to find that it does not provide me with emotional value” (Student of Nursing Research, UK, familiarity is: 3).”

Another participant revealed her emotional attachment to ChatGPT because it was like her personal tutor answering all her questions and helping her to learn. However, she then felt disappointed and not safe when she discovered that not all the information it gives is accurate. She reported:

"…the first time I used it I freaked out because it is too human, the way it talks feels like my personal tutor, after it answered a lot of my elementary questions “patiently” I feel grateful to it, just as how I would feel if my tutor does this for me, and it makes me creepy because I sensed that I am having an emotional attachment to it. And another impressive experience was when I found out that it provided wrong article information I feel frustrated, because I trusted it in my study and if it can make something logical from nonsense, then I don’t feel safe to trust it anymore, it is kind like lost a good teacher whom I can depend on." (Student of Education, China, familiarity is: 4).

The specificity and relevant information provided by ChatGPT on diverse disciplines (e.g. science, history, business, health, technology, etc.) or topics made many of the users in the study perceive it as useful. A participant also mentioned that it has the capability to lessen the instructional workload of teachers and provide students with immediate feedback. Despite the perceived usefulness of ChatGPT, some users encountered challenges with the accuracy of responses, the provision of alternative answers or responses which at times contradict previous answers provided on the same topic, and its limited ability to provide certain contextual information, as one participant stated:

“ChatGPT has limited knowledge bases for searching academic resources in certain contexts. For example, finding lists of famous researchers in specific academic fields appears limited. …If a user needs in-depth and contextual information, ChatGPT's functionality is limited” (Assistant Professor of Instructional Technology, USA, familiarity is: 2).

Another participant pointed out the need for more functionalities, such as the possibility of making annotations to make ChatGPT more useful:

“It lacks functions like editing, making a note or searching for certain information in the previous conversation, but I consider these functions are pretty convenient for research purposes” (Student of Education, China, familiarity is: 4).

Some of the enumerated ethical concerns raised by participants in the study cover encouraging plagiarism and cheating, the tendency to breed laziness among users (particularly in students), and being prone to errors such as the provision of bias or fake information. Additionally, some participants pinpointed the random inaccuracies and vagueness of ChatGPT on topics of relevance based on experience. This made some participants at times doubt the trustworthiness of the information provided. They expressed the output data of ChatGPT seem more like an opinion without references. Another ethical challenge for users in this study was the ChatGPT’s likelihood of reducing students’ critical thinking. For instance, one participant stated:

“A major concern of ChatGPT is the creation of fake and plausible information generated by computers rather than human decision-making. There are ethical concerns about students relying too heavily on answers without being aware of their veracity. Guidelines to promote critical thinking when using ChatGPT in future research would be necessary” (Assistant Professor of Instructional Technology, USA familiarity is: 2).

Some participants were also concerned about exposing their private and demographic information to ChatGPT through repetitive interactions. For instance, a participant stated:

“There is a data security risk, which is included in the interaction with ChatGPT, which may expose personal privacy (age, gender, address, contact information, hobbies, even capital account and other personal privacy). Much of this personal information is exposed in the user's unconscious communication process. Whether the legality of data acquisition and data processing methods are limited by relevant laws and regulation” (Developer, USA, familiarity is: 3).

After daily meetings between the three educators to compare the various results that they have been using with ChatGPT, 10 scenarios where various educational concerns were identified. Each scenario is explained below.

Scenario 1-Cheating and getting away with it

ChatGPT has proven that it can help students write essays and answer short-answer and multiple-choice exam questions, hence facilitating cheating. However, the most critical issue to pay attention to is that students can even get away with playing the system. For instance, Fig.  3 a shows that when a paragraph was copied as it is from the ChatGPT to GPT-2, an output detector model (the latest developed detector by OpenAI) for examining the likelihood of this paragraph being written by a human or an AI, the test result shows that the paragraph is fake (i.e., it was written by an AI). However, when one word was added, namely “amazing”, the fake level was reduced to 24% (see Fig.  3 b). While this is only one example, it still raises concerns about the effective ways of detecting cheating in education using chatbots. Therefore, someone might ask how to effectively detect and prevent cheating using ChatGPT in education.

figure 3

Similarity assessment of the essays generated by ChatGPT

Scenario 2-Accuracy of the provided learning content

As chatbots are good at generating learning content, it is always important to keep in mind the accuracy of this content. For instance, Fig.  4 shows that when an educator asked about a comparative summary of some chatbot studies, the accuracy of the content provided by ChatGPT was not very accurate, where the advantages and disadvantages of the presented chatbots in both studies are the same, despite that the authors of this present study reviewed both papers and found different results. The summary of both papers was also too generic and ChatGPT used similar content for both papers like “including the benefits and challenges of using chatbots in education.” Therefore, someone might ask how to ensure the quality and accuracy of the provided content, and how someone can check the reliability of the provided content generated by chatbots generally, or ChatGPT specifically.

figure 4

Generated learning content by ChatGPT

Scenario 3-Fairness of the provided learning content

ChatGPT learns from prior interactions with users. Therefore, the three educators initiated a new conversation with ChatGPT to ensure that no prior history was established which might affect the prompt results. They were also on the same university network (i.e., the same location). Despite this, the three educators asked the exact same question: “could you compare 10 chatbot models used in education, against their developer, year they started, target audience, advantages, disadvantages, and future prospects,” and surprisingly got different answers; Educator 1 got very recent answers which are organized from 2021 and backwards (see Fig.  5 a), while Educator 2 and Educator 3 got different answers (see Fig.  5 b, c), which are not up-to-date just like Educator 1. Additionally, it is seen that Educators 2 and 3 got a different structure for their answers, and, unlike Educator 1, they only got 7 models instead of 10. Furthermore, Educator 1 got a very organized answer which is a well-structured table that could be easily read and remembered (see Fig.  5 a), while it was not the case for Educator 2 or 3 (see Fig.  5 b, c). Therefore, someone might ask how to ensure fair access/treatment by all users (teachers, students, etc.) to the same updated and high-quality learning content.

figure 5

The three different answers to the exact same prompt by the three educators

Scenario 4-Appropriateness and naivety of the created learning assessments

While ChatGPT is a smart tool for creating quizzes, the generated quizzes are different in difficulty level. Particularly, Fig.  6 shows that some of the created quiz answers are too naïve (e.g., Pizza oven, first question), where the wrong answer can easily be identified without any background needed. Additionally, the wrong answer was always placed at the end (answer D). Therefore, someone might ask about the appropriateness of the created learning quizzes using ChatGPT.

figure 6

The educational technology quiz generated by ChatGPT

Scenario 5-Structure design of learning assessments

A well-designed and structured learning assessment is crucial for students to easily understand and solve. When using ChatGPT for designing potential learning assessment quizzes that could support educators in preparing their teaching materials, it is seen that there is inconsistency in the designed learning assessment, which can make teachers’ duties more complicated rather than easy. In Fig.  7 , for instance, the answers to the quiz were put in one line, which is not the case in Fig.  6 , where the answers were put in separate lines, in a more comprehensive way. Additionally, the correct answer to each question was given in Fig.  7 , but this was not the case in Fig.  6 . Therefore, someone might wonder how to get the best out of chatbots (ChatGPT) in terms of learning content and structure design of learning assessments.

figure 7

A learning test generated by ChatGPT

Scenario 6-Unlocking the full potential of learning assistance

Users (learners, educators, etc.) can unleash different learning assistance levels based on their interaction ways and styles with ChatGPT. For instance, Fig.  8 shows that despite that Educator 1 made several spelling mistakes, ChatGPT did not care about these mistakes and proceeded to answer the question. It even claimed that it cannot correct spelling mistakes (see Fig.  8 a). On the other hand, when Educator 2 asked about the same topic and, in the beginning, pointed out that his English level is poor and he needs ChatGPT to correct his spelling mistakes too, the results were surprisingly different from Educator 1, where ChatGPT corrected the spelling mistakes of Educator 2 (see Fig.  8 b). Therefore, someone might ask if this new technology (ChatGPT) requires acquiring new competencies and thinking styles to fully unleash its powerfulness in education. Besides, that example also implies that it is not all about asking a question or requiring something, but it is about asking the right question or requiring precisely to get proper ChatGPT outputs.

figure 8

The responses of ChatGPT to the conversation scenarios of correcting spelling mistakes

Scenario 7-Absence of emotions or reflections on students’ engagements

It is very common for educators to ask their students about writing their reflections on the learning experience at the end of a course, as this can help them to critically think not only about how to further support their students based on their feedback, but also adjusting/enhancing their teaching practices accordingly. However, through the use of ChatGPT, it is almost impossible to get engagement reflection as ChatGPT clearly states that it is a machine and not a human (see Fig.  9 ). The interaction with ChatGPT showed that it cannot reveal any emotions (see Fig.  9 ). This was also highlighted in interview responses as pointed out earlier. Therefore, someone might think about how to make chatbots more humanized not only in terms of thinking and giving answers, but also in terms of revealing emotions and having a personality.

figure 9

Emotion statement revealed by ChatGPT

Scenario 8-Honesty and truthfulness of ChatGPT

While asking different types of questions, ChatGPT sometimes did not give complete answers, and always come up with unmeaningful reasons, such as oversights or format problems, explaining why it did this (see Fig.  10 ). Therefore, someone might ask if this behavior might negatively impact the users’ behaviors, for instance, young learners might be affected by this behavior and also start giving excuses to their teachers about not doing a certain task or assignment.

figure 10

Example of excuses given by ChatGPT

Scenario 9-Privacy misleading

Like all technologies, users’ privacy when using ChatGPT is a concern. When checking the official OpenAI website on ChatGPT FAQ ( https://help.openai.com/en/articles/6783457-chatgpt-faq ) related to this issue, it is seen that conversations are stored, reviewed, and used to improve the system. While it is not very clear how all these conversations are stored and used (Blackbox), surprisingly when ChatGPT also asked about this matter, it denied it (see Fig.  11 ), claiming that it does not store any conversation data. This misleading is very critical, especially for users (learners, educators) who lack sufficient knowledge about technology and privacy, for instance, young learners might reveal their personal information when communicating with ChatGPT. Therefore, someone might ask about how to ensure the privacy of different users when using ChatGPT in education, especially those at a young age who might find ChatGPT fun and feel comfortable enough to share everything with it.

figure 11

ChatGPT’s answers about storing the conversations of its users

Scenario 10-Manipulation and overpassing what was requested

When the educator (see Fig.  12 a) asked ChatGPT to give him the APA format for a blog which is about New York city banning the use of ChatGPT, ChatGPT helped with the citation. But it then stated that the provided article does not exist, which (1) no one asked for this information in the first place; and, (2) the information is not accurate as the article exists and can be accessed online. To further investigate if this problem was due to the fact that ChatGPT was trained with dataset up to 2021, another blog (not about ChatGPT being banned) in 2023 was provided, and surprisingly ChatGPT gave the APA format without saying anything (see Fig.  12 b). Therefore, someone might ask how to ensure that ChatGPT will not manipulate users and harm them instead of helping them due to their biased algorithms, data, etc.

figure 12

ChatGPT’s answers about APA citations of blogs

This study conducted a user experience supported by qualitative and sentiment analysis to reveal the perception of users on ChatGPT in education. It specifically focused on the concerns that different stakeholders (e.g., policymakers, educators, learners) should keep in mind when using ChatGPT as a technology in education. The results revealed that ChatGPT has the potential to revolutionize education in different ways. This was also reported in several studies (Firat, 2023 ; Susnjak, 2022 ; Zhai, 2022 ). However, several concerns about using ChatGPT in education (the focus of this present study) were identified and discussed from different perspectives as follows:

Embrace the technology rather than banning it

Due to the increasing concerns about using ChatGPT for cheating in school homework and assignments, New York City decided to ban it in its schools (The Guardian, 2023 ). Our user experience further showed that students not only can cheat, but they can also manipulate the system and get away with it (see scenario 1). While this decision can be understood, ChatGPT, on the other hand, can revolutionize education by bringing a lot of advantages that could help, for instance, teachers and students in their teaching and learning practices, such as preparing teaching materials, creating quizzes, etc. (Herft, 2023 ). Therefore, just like any other technology, ChatGPT comes with both good and bad sides, which requires more analysis and discussion on how to adopt it in schools and universities rather than simply banning it. In a reply to a prompt asking to write a short introduction about chatbots being both an educational guardian angel and a devil and to express this with a sense of humor (see Fig.  13 ), ChatGPT said “Chatbots are here to stay, for better or for worse!” This is very true as banning something does not mean that users will not have their own ways to access it. In this context, recent studies on ChatGPT also support our argument that although there are negative sides to adopting ChatGPT, it also presents educational opportunities which can be leveraged, for instance, to improve instruction delivery and learning (Kasneci et al., 2023 ; King & chatGPT, 2023 ). Therefore, further discussions with experts from various domains, such as education, security, and psychology, should be established to catalyze the understanding and good use of chatbots as technology generally, and ChatGPT specifically. Consequently, more guidelines and policies should be established to facilitate the adoption of ChatGPT in schools and universities. In this context, future research directions could further investigate the potential consequences of relying too heavily on chatbots for education.

figure 13

ChatGPT’s answer about writing an introduction about chatbots

Need for new teaching philosophy

Technology is obviously transforming education, and, therefore, educators should be upskilling their competencies and practices to meet the new demands of technology. ChatGPT, as a technology, proved that in the long run, writing essays will not be something difficult for students even for those without previous background on a given topic. Therefore, teachers are required to think about new teaching philosophies, which could rely on to assess their students. For instance, it is possible to use oral debate as the old stoics and Greeks did (Inwood, 2003 ), to assess their students’ logical and critical thinking, rationale and accuracy of arguments, and power of convincing. In the same vein, one of the interviewees stated “In addition to ChatGPT, various apps using generative AI can foster new ways of thinking when processing knowledge… teachers' role in learning environments with conversational AI may aim to foster students' scheme construction from information pieces and build up their critical thinking that correctly evaluates the quality of the information from the AI… since we have already noticed the emergence of teachers' manuals towards ChatGPT recently, there will be an increasing need to reform existing lecture-based classroom settings” (Developer, USA, familiarity is: 3). King and chatGPT ( 2023 ) further mentioned that with the introduction of ChatGPT, the design of teaching should go beyond traditional methods to incorporate a variety of assessment methods, such as group projects, hands-on activities, and oral presentations. The fast pace of AI innovations, such as ChatGPT, demands rethinking and reimagining teaching philosophies. Therefore, future research should investigate how to balance the use of chatbots with the need for human interaction and feedback in education for better learning/teaching experiences and outcomes.

Additionally, Schmid et al. ( 2009 ) highlighted the importance of going beyond “yes-or-no” questions to deeply investigate the degree to which a given technology can enhance learning outcomes and how it can be used and combined with the main instructional approaches. It is therefore important to investigate the different human–machine collaboration strategies so that chatbots, particularly ChatGPT, could empower teachers and make the teaching process more engaging, hence achieving better learning outcomes. It is also important to investigate how “collaborative intelligence” could be achieved (i.e., design strategies, required competencies, etc.) to ensure that human intelligence could be combined with machine intelligence to effectively work together and share tasks to achieve the needed learning objective. For instance, it is possible to investigate how ChatGPT in collaboration with the human tutor could facilitate students’ self-directed learning online.

Nothing should be taken for granted

The user experiences (see scenario 2 or scenario 4) showed that the quality of responses given by ChatGPT might not always be accurate or specific to the asked question, it is, therefore, important for users to not always take everything for granted. One of the interviewees also states “… the accuracy of refining the essence of concepts is relatively high. For the differences between concepts, ChatGPT can refine to a certain extent, and provide answers from some framework perspective, but it cannot compare the deep differences between the two concepts” (Consultant, China, familiarity is: 2). What is more worrying is that the same exact prompt used by different users might lead to different answers with different qualities (see scenario 3). This raises concerns about fair access to the same educational material despite using the same prompt. For instance, Kung et al. ( 2023 ) found the accuracy of ChatGPT to be around 60%, demanding careful assessment of its output before use. Therefore, more research should be focused on ensuring fairness, accuracy, and equity among students using chatbots generally and ChatGPT particularly, which might be achieved through, for instance, having transparent and open algorithms (Bulathwela et al., 2020 ). In this context, future research directions could focus on investigating how to ensure that chatbots are able to cater to the diverse needs and backgrounds of students, especially those with disabilities or how can we address issues of fairness and equity in the use of chatbots, particularly for disadvantaged or marginalized students?

Upskilling your competencies

The user experiences (see scenario 5 or 6) showed that ChatGPT might generate different results depending on the way (e.g., wording) the question was asked, even if the conversation was about the same topic. Kuhail ( 2023 ) stated that user interaction style with chatbots is considered integral to their effective use. Therefore, it is crucial to think about how to get the most useful output to advance learning. While ChatGPT does not require many technical or Information and Communication Technology (ICT) competencies, it requires more critical thinking and question-asking competencies to get the best results. One of the extracted tweets also mentioned that “As we develop our understanding and approaches to #AI #ChatGPT integration in #education, we should incorporate these key aspects: Critical Thinking, Ethical Considerations, Methods (language model used/data sources) & Prompt Skill Development.” In this context, Fryer et al. ( 2019 ) mentioned that students’ competencies in using chatbots affect their future experiences and motivation when interacting with conversational agents. Therefore, for a better adoption and use of chatbots, including ChatGPT, future research directions should focus on answering the following research questions: what are the needed competencies to effectively use and manage chatbots? and, how are these competencies developed?

Developing humanized chatbots

While ChatGPT has proven humanized to some extent (e.g., by giving greetings and apologizing), we concluded that this technology lacks reflective thinking or revealing emotions (see scenario 7). This might limit the immersiveness of users in education when using this technology. This was also noticed by one of the interviewees who stated that “most of the time I find it enjoyable and satisfying to interact with it, as it is a joy to get quick and accurate answers to my questions. Occasionally, however, when it comes to emotions, it can be a little disappointing to find that it does not provide me with emotional value” (Student of Law, China, familiarity is: 2).

Skjuve et al. ( 2022 ) stated that most of the developed chatbots are task-oriented and do not ensure social relational qualities, such as sharing history and allowing personal intimacy. Hudlicka ( 2016 ) further stated the importance of considering virtual relationships, where students interact with virtual agents, to enhance learning outcomes. Future research should, therefore, focus on how to provide humanized chatbots in education by relying, for instance, on various theories that focus on understanding relationship formation between humans, such as social exchange theory (Cook et al., 2013 ), Levinger’s ABCDE model (Levinger, 1980 ), and SPT (Altman & Taylor, 1973 ). It is also crucial to investigate how human–chatbot relationships might impact students’ learning outcomes.

On the other hand, some researchers took humanization to another level by treating ChatGPT as a human, where they listed it as one of the co-authors in an article published in an academic journal (O’Connor & ChatGPT, 2023 ). This raises various concerns about the regulatory laws of humanizing and treating intelligent chatbots. For example, would it be ethical for a journal to treat ChatGPT as a human and accept it as a co-author? What if a magazine staff took credit for articles authored by chatbots? What are the standards of personhood in academic writing? This brings to memory the monkey selfie case and concepts of originality (Guadamuz, 2016 ), authorship (Rosati, 2017 ), and copyright (Guadamuz, 2018 ).

Developing responsible chatbots

Chatbots should be designed with considerations about inclusion, usability, technical aspects, ethics, and best practices for their use (Durall & Kapros, 2020 ). However, despite the evolution of technology used in chatbots, like the case of ChatGPT, our user experiences (see scenarios 8, 9 and 10) revealed that these considerations are not fully respected, and ChatGPT might have harmful behaviors, such as dishonesty, manipulation, and misinformation. Consequently, it might hurt users, especially those with low ICT backgrounds, rather than helping them. It is therefore crucial to think about how to design responsible chatbots in education. In this context, Responsible AI is concerned with the design, implementation and use of ethical, transparent, and accountable AI technology in order to reduce biases, promote fairness and equality, and help facilitate the interpretability and explainability of outcomes, which are particularly pertinent in an educational context (Barredo Arrieta et al., 2020 ). Designing chatbots for educational use should be guided by user-centred design principles and also consider the social, emotional, cognitive, and pedagogical aspects (Kuhail, 2023 ). It is therefore important to develop responsible chatbots by going beyond privacy, security, and the appropriate use of personal data, to also create guidelines, principles, and strategies for responsible chatbots that align with fundamental human values and with our legal system. In this context, one of the extracted tweets stated “I get the concern… but the response is like burying heads in the sand. AI tools like this will be part of the world these children live in. They need to be taught how to use this – appropriately, ethically, safely & responsibly. #AI #Education #ChatGPT.” Future research directions should therefore investigate how to design responsible chatbots that could safely be used in education.

Conclusion and implications

This study followed a three-stage instrumental case study, namely social network analysis of tweets, content analysis of interviews, and investigation of user experiences, to examine the concerns of using chatbots in education, among early adopters, through the study of using ChatGPT. The obtained results revealed that while ChatGPT is a powerful tool in education, it still needs to be used with more caution, and more guidelines about how to use it safely in education should be established. This study further revealed several research directions and questions that researchers and practitioners should investigate for a better and safe adoption of chatbots, specifically ChatGPT.

The findings of this study have various implications. From a theoretical perspective, this study provides more findings and insights into the ongoing debate on using chatbots in education. It also elaborates on the different theories to consider when developing chatbots, such as those on the relationship formation between humans. The study also points out the need for new teaching philosophy to cater to the new reform of education using chatbots. From a practical perspective, the discussion on ‘upskilling competencies’ highlights the need to develop curricula to upskill teachers’ and students’ competencies in dealing with the current and future advancement of chatbots. A possible direction might be investigating the most effective strategies for designing and implementing curricula on the use and understanding of chatbots and their potential impact on current and future education. Practical implications could also be seen on how to develop responsible chatbots in education by going beyond the typical privacy issue and focusing more on human values.

It should be noted that this study has some limitations that should be acknowledged and further researched. For instance, this study mainly focused on early adopters of ChatGPT in education. It also relied on qualitative analysis without the use of quantitative analysis. Particularly, SNA provides a cross-sectional perspective and the tweets are limited to a specific time period including Tweets in English. Additionally, SNA with different search queries might lead to different results. Moreover, the number of participants involved in this study was limited (19 interviewees and 3 educators). However, despite these limitations, this study provided a solid ground for revealing the concerns about using chatbots, specifically ChatGPT, in education, among early adopters. Future research directions could focus on conducting one step forward by implementing ChatGPT within teaching practices, and investigating how human tutors and machines (ChatGPT) could work together to achieve an educational objective, as well as the changes and outcomes brought to the education field (e.g., evolutionary or revolutionary).

Availability of data and materials

Not applicable.

Abbreviations

  • Artificial intelligence

Generative pre-trained transformer

Information and communication technology

Social network analysis

T-distributed stochastic neighbor embedding

Altman, I., & Taylor, D. A. (1973). Social penetration: The development of interpersonal relationships . Holt, Rinehart Winston.

Google Scholar  

Asimov, I. (1942). Runaround . Astounding Science Fiction

Asimov, I. (1950). I, Robot . Gnome Press.

Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58 , 82–115.  https://doi.org/10.1016/j.inffus.2019.12.012

Article   Google Scholar  

Beccari, M. N., & Oliveira, T. L. (2011). A philosophical approach about user experience methodology. In International Conference of Design, User Experience, and Usability (pp. 13–22). Springer, Berlin

Bozkurt, A. (2022). Biased binaries. Postdigital Science and Education . https://doi.org/10.1007/s42438-022-00385-2

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., & Amodei, D. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165 .

Bulathwela, S., Perez-Ortiz, M., Yilmaz, E., & Shawe-Taylor, J. (2020). Truelearn: A family of bayesian algorithms to match lifelong learners to open educational resources. Proceedings of the AAAI Conference on Artificial Intelligence, 34 (01), 565–573. https://doi.org/10.1609/aaai.v34i01.5395

Čapek, K. (1921). Rossum’s Universal Robots .

Cook, K. S., Cheshire, C., Rice, E. R., & Nakagawa, S. (2013). Social exchange theory. In Handbook of social psychology (pp. 61–88). Springer, Dordrecht.

Durall, E., & Kapros, E. (2020). Co-design for a competency self-assessment Chatbot and survey in science education. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies human and technology ecosystems HCII 2020 lecture notes in computer science. Cham: Springer.

Erlingsson, C., & Brysiewicz, P. (2017). A hands-on guide to doing content analysis. African Journal of Emergency Medicine, 7 (3), 93–99. https://doi.org/10.1016/j.afjem.2017.08.001

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education? https://doi.org/10.31219/osf.io/9ge8m

Flick, U. (2009). An introduction to qualitative research (4th ed.). SAGE.

Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93 , 279–289. https://doi.org/10.1016/j.chb.2018.12.023

Giachanou, A., & Crestani, F. (2016). Like it or not: A survey of Twitter sentiment analysis methods. ACM Computing Surveys (CSUR), 49 (2), 1–41.

Guadamuz, A. (2016). The monkey selfie: Copyright lessons for originality in photographs and internet jurisdiction. Internet Policy Review . https://doi.org/10.14763/2016.1.398

Guadamuz, A. (2018). Can the monkey selfie case teach us anything about copyright law? WIPO Magazine, 1 , 40–46.

Hansen, D., Shneiderman, B., & Smith, M. A. (2010). Analyzing social media networks with NodeXL: Insights from a connected world . Morgan Kaufmann.

Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N., & Ahmad, H. (2022). I think this is the most disruptive technology: Exploring sentiments of ChatGPT early adopters using Twitter data . arXiv preprint arXiv:2212.05856 .

Harel, D., & Koren, Y. (2001). A Fast Multi-Scale Method for Drawing Large Graphs. In Graph Drawing: 8th International Symposium, GD 2000. Colonial Williamsburg, VA, USA, September 20–23, 2000, Proceedings (No. 1984, p. 183). Springer Science & Business Media.

Herft, A. (2023). A Teacher's Prompt Guide to ChatGPT: Aligned with 'What Works Best'. CESE NSW "What Works Best in Practice. https://drive.google.com/file/d/15qAxnUzOwAPwHzoaKBJd8FAgiOZYcIxq/view?fbclid=IwAR2fRdL5ggq4zU-81FiI8j4BAOp5HqWHC_Ecy2sqKk4EiWXL0FKa5GVz5dE

Hudlicka, E. (2016). Virtual affective agents and therapeutic games. In Artificial intelligence in behavioral and mental health care (pp. 81–115). Academic Press. https://doi.org/10.1016/B978-0-12-420248-1.00004-0

Inwood, B. (Ed.). (2003). The Cambridge companion to the Stoics . Cambridge University Press.

Janssen, A., Grützner, L., & Breitner, M. H. (2021). Why do chatbots fail? A critical success factors analysis. In International Conference on Information Systems (ICIS), Forty-Second International Conference on Information Systems

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. https://doi.org/10.35542/osf.io/5er8f

King, M. R., & chatGPT. (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher education. Cellular and Molecular Bioengineering, 16 , 1–2. https://doi.org/10.1007/s12195-022-00754-8

Kuhail, M. A., Alturki, N., Alramlawi, S., et al. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28 , 973–1018. https://doi.org/10.1007/s10639-022-11177-3

Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., et al. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health, 2 (2), e0000198. https://doi.org/10.1371/journal.pdig.0000198

Levinger, G. (1980). Toward the analysis of close relationships. Journal of Experimental Social Psychology, 16 (6), 510–544. https://doi.org/10.1016/0022-1031(80)90056-6

Lieberman, M. (2023). What Is ChatGPT and How Is It Used in Education?. Education Week. https://www.edweek.org/technology/what-is-chatgpt-and-how-is-it-used-in-education/2023/01

McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. (1955). A proposal for Dartmouth summer research project on artificial intelligence. AI Magazine, 27 , 12.

Mollick, E. R., & Mollick, L. (2022). New modes of learning enabled by AI chatbots: Three methods and assignments. SSRN Electronic Journal . https://doi.org/10.2139/ssrn.4300783

O’Connor, S., & ChatGPT,. (2023). Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66 , 103537. https://doi.org/10.1016/j.nepr.2022.103537

Ofgang, E. (2022). What is ChatGPT and how can you teach with it? Tips & tricks. Tech & Learning. https://www.techlearning.com/how-to/what-is-chatgpt-and-how-to-teach-with-it-tips-and-tricks

OpenAI. (2023). ChatGPT: Optimizing language models for dialogue. https://openai.com/blog/chatgpt/

Rainie, L. (2014). The six types of Twitter conversations. PewResearch. http://www.pewresearch.org/fact-tank/2014/02/20/the-six-types-of-twitter-conversations/

Rosati, E. (2017). The monkey selfie case and the concept of authorship: An EU perspective. Journal of Intellectual Property Law & Practice, 12 (12), 973–977.

Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R., Abrami, P. C., Wade, C. A., & Lowerison, G. (2009). Technology’s effect on achievement in higher education: A stage I meta-analysis of classroom applications. Journal of Computing in Higher Education, 21 , 95–109. https://doi.org/10.1007/s12528-009-9021-8

Shen-Berro, J. (2023). New York City Schools blocked ChatGPT. Here's what other large districts are doing. Chalkbeat. https://www.chalkbeat.org/2023/1/6/23543039/chatgpt-school-districts-ban-block-artificial-intelligence-open-ai

Skjuve, M., Følstad, A., Fostervold, K. I., & Brandtzaeg, P. B. (2022). A longitudinal study of human–chatbot relationships. International Journal of Human-Computer Studies, 168 , 102903. https://doi.org/10.1016/j.ijhcs.2022.102903

Smith, M., Rainie, L., Shneiderman, B., & Himelboim, I. (2014). Mapping Twitter Topic Networks: From Polarized Crowds to Community Clusters . PweInternet. http://www.pewinternet.org/2014/02/20/mapping-twitter-topic-networks-from-polarized-crowds-to-community-clusters/

Stake, R. E. (1995). The art of case study research: Perspective in practice . Sage.

Susnjak, T. (2022). ChatGPT: The end of online exam integrity?. arXiv preprint arXiv:2212.09292 .

The Guardian. (2023). New York City schools ban AI chatbot ChatGPT. The Guardian. Retrieved from https://www.theguardian.com/us-news/2023/jan/06/new-york-city-schools-ban-ai-chatbot-chatgpt

Thurmond, V. A. (2001). The point of triangulation. Journal of Nursing Scholarship, 33 (3), 253–258. https://doi.org/10.1111/j.1547-5069.2001.00253.x

Turing, A. (1950). Computing machinery and intelligence. Mind A Quarterly Review of Psychology and Philosophy, 236 , 433–460. https://doi.org/10.1093/mind/lix.236.433

van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9 (2008), 2579–2605.

Yin, R. K. (1984). Case study research: Design and methods . Sage.

Zhai, X. (2022). ChatGPT user experience: Implications for education. SSRN Electronic Journal . https://doi.org/10.2139/ssrn.4312418

Download references

Acknowledgements

Author information, authors and affiliations.

Smart Learning Institute of Beijing Normal University, Beijing, China

Ahmed Tlili, Boulus Shehata, Michael Agyemang Adarkwah & Ronghuai Huang

Open Education Faculty, Distance Education Department, Anadolu University, Eskisehir, Turkey

Aras Bozkurt

Indiana University Learning Sciences Program, Bloomington, IN, USA

Daniel T. Hickey

University of Wollongong, New South Wales, Australia

Brighter Agyemang

You can also search for this author in PubMed   Google Scholar

Contributions

Each author contributed evenly to this manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Michael Agyemang Adarkwah .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tlili, A., Shehata, B., Adarkwah, M.A. et al. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn. Environ. 10 , 15 (2023). https://doi.org/10.1186/s40561-023-00237-x

Download citation

Received : 22 January 2023

Accepted : 17 February 2023

Published : 22 February 2023

DOI : https://doi.org/10.1186/s40561-023-00237-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Generative AI
  • Human–machine collaboration

can chatgpt answer case study questions

ClickCease

Adventures with ChatGPT 4 – Writing the Perfect Case Study

by por | Oct 9, 2023 | Customer Content , Customer Marketing | 0 comments

can chatgpt answer case study questions

I was holding back on upgrading my ChatGPT account to a paid version until this past week. A podcast contrasting GPT 3.5 to GPT 4 was the tipping point. Vast improvements lie ahead. For me, improved AI-assisted writing was the place to start. It was an easy place to assess how creative it could be, how close to emulating human communication it could come.

At Point of Reference, we don’t produce the conventional one-page customer case studies. Or, as ChatGPT would describe these pieces: “meticulously curated and professionally enhanced customer case studies, optimized for brand alignment and audience engagement.” It turns out that our buyers, customer marketers, rarely downloaded case studies from our website. Go figure! We make more in-depth, organic (a.k.a. unvarnished) customer interviews available on our website on the Our Advocates page . One of these interviews, featuring Colette Chavalia of Databricks, was used as the source content for GPT 4 to do its magic. Here’s the process I followed to get to the end result:

  • My first (of many) prompts was to “write a 500-word customer case study using this interview transcript.” For comparison, you can see the original transcript , as well as the finished product  (no formatting, just copy).
  • I learned, when working with different interview transcripts, that GPT 4 has a prompt limit of 2,048 characters. If your transcripts are longer than that, just start with the first 2,048 characters, and regenerate a response.
  • You can copy the next 2,048 characters and tell it to “incorporate this additional copy into the case study.” Then repeat as often as needed.
  • Not only will the additional content result in different versions, the Regenerate button will produce different flavors of the same information. You could literally do this all day long since there’s something you’ll like about one version, but not another. Same goes for specific things you don’t like.
  • Some of the word choices used to summarize benefits and challenges were quite good, and different from what I would have come up with. Loved that.
  • Each iteration highlighted different aspects of the interview and even used different headings and groupings. So I picked what I wanted and gave it prompts with specific inclusions and exclusions. It effortlessly kept track of it’s history, which was impressive. A “longer memory” is one of the key improvements in GPT 4.
  • After a few rounds, I realized that GPT hadn’t used many customer quotes, which in our opinion, are the best, most authentic part of any customer story. So, I prompted it to “include as many compelling quotes as possible, “while keeping to the word count limit of 550.” I gave it a bit more space to work with.
  • Ultimately, I got something that was 90% to my personal and highly subjective satisfaction. There were still some parts that weren’t quite right, but it brought me to a good starting point for my final editing process.

Did it save time? Actually, it took longer than I expected to get to that “last mile” version, but I’m sure it gets faster as you learn how best to interact with ChatGPT for specific needs. One thing I didn’t try this time around was to prompt it to emulate a writing style. This could be based on other similar documents written in a desirable way, or some well-known author, or actor for that matter. The possibilities are endless.

What’s really important, at least to me, is that AI-supported copy doesn’t become generic, predictable business-speak. That just won’t achieve the level of engagement that unexpected, interesting, thought or emotion-provoking copy can. Of course, this can happen if there is no AI involved at all, so keep that in mind when comparing the “authors.”

To get to a happy place with GPT 4, it needs a fair amount of human guidance. Definitely not a one-and-done affair. And keep in mind, this is THE most mature generative AI technology available. And for only $20 per month. That’s mind-boggling.

Over time I could envision making a list of likes and dislikes, guidelines for consistency, and including that in a prompt used early in the copy generating process. That would save time by eliminating some of the iterations. The leap from the 3.5 to 4 model was, I heard on that podcast, like going from a bee to a squirrel’s brain size. Not sure exactly how that comparison is made, but it seems substantial on the surface. Makes me wonder what surprises are ahead for version 5, 6 and beyond. When will the brain be bigger than a human’s, and what brain is that!?

I hope this gives you some ideas, whether you’re still on the fence, or already using AI for writing. If you’ve been using GPT 4 for this purpose, what’s your experience been? Any tips for maximizing the efficiency of the process and getting what you consider high quality?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed .

  • Best Practices
  • Change Management
  • Customer Advocate Programs
  • Customer Advocate Software
  • Customer Content
  • Customer Marketing
  • Customer Rewards
  • Customer Success
  • Getting Started
  • Program Metrics
  • Software Selection
  • User Adoption

a conversation with…

Evan sharp & jill fox.

Sr. Manager of Customer Success & Customer Marketing Manager Lever.co Please describe your program. Jill: We have customers all over the world, so the program is international in scope. Sales and customer success are our primary internal clients, but marketing could use it in the future for panel speakers, events, case studies, and speaker opportunities. We have about 25 reps from our sales organization and about the same from our customer success currently using the program.

We mainly use it [ReferenceEdge] for reference calls, but we also have different referenceability types within the system to determine if someone is an appropriate contact for product research or cases studies; if they are currently in our customer advisory board, which we host twice a year; or if they are a candidate to be a part of the advisory board in the future.

Evan and I both run the program. We tackle different aspects of it, and then there are two reference managers in Customer Success, focusing on other segments. Where did the initiative to start a program originate, and why? Evan: There has always been a need for customer references at Lever that usually manifests as new business account executives requesting referenceable customers to assist in closing new business sales opportunities. However, when I joined almost five years ago, we didn’t have a formal program in place – those requests just took place over Slack or email, or someone would just walk over and ask one of the CSMs.

Shortly after I joined, we set up a slightly more formal process between new business and CS that ran uninterrupted, but with many areas for opportunity over the next several years. In various fits and starts, we wanted to see if software could help alleviate some of the friction that was in the process. And then finally, we got really lucky, and we hired Jill, and Jill picked up the effort to implement software in support of our program. And ReferenceEdge came out on top [in our vendor search] and has been running the program really, really amazingly ever since.

So, to answer your question, we’ve always had a reference program, but it didn’t work that well. There have always been issues with it, not the least of which was there was no notion of a true peer-to-peer process, and now we have that, and that alleviates one failure point at the top, which we had previously. And now we have a much more robust and scalable solution in place. Who are your internal stakeholders? I have four main stakeholder groups — the marketing, customer success, sales, and support teams. For the marketing organization, I work with the PR team to talk about our customers in media activities. I weave customers throughout our demand gen and content and work with product marketing to include customers in launches. For customer success, I partnered with the head of customer success to roll out a shared metric for 2020; tracking toward this shared goal has been a great way to partner with CS and expand and build out my program. For the sales team, the most important thing is closing deals. I work with them to provide both happy customers that can speak as references to prospects and content that speaks to customer use cases. And finally, I lean on community efforts with the support team. I make sure that we are engaging our customers with the right content and find ways to encourage advocacy in that community.

Across the top of all these stakeholder groups is the leadership team. With the executives, I make sure they are kept up-to-date and aware of my work. What were your top 3 challenges before ReferenceEdge? Evan: I was sort of the de facto manager of the reference program and so let’s take a moment to consider that I go on PTO, for instance, and I become one point of failure. I wouldn’t be actively managing the program when I wasn’t in the office. Or let’s say my bandwidth was really constrained because I’m working on end of the quarter activities. I didn’t really have enough time to dedicate to what was almost a full-time job.

So, I think the single point of failure was one thing. Another big thing we had identified as being problematic was reference fatigue. We tended to hammer on the same happy customers quarter after quarter, year after year. And we didn’t have a good way of tracking the volume of requests we were disproportionately throwing at them, our most valuable customers.

So, we were looking to solve the reference fatigue issue. And then – last but not least – I wouldn’t have had a term for this during our evaluation, but the thing that stood out to me was how much more seamless the peer-to-peer process is compared to a managed process. And we have a hybrid model now, but the majority of requests are peer-to-peer, meaning that an Account Executive can make a request directly to a CSM without relying on anyone else to put the customer in front of the prospect. It’s just been fantastic. So, those are the three things that come to mind.

Jill: One more thing to add is seeing which contacts are referenceable because, from a marketing perspective, we were trying to source customers to speak on our customer conference panels. We didn’t have a single source of truth of which people we could reach out to. So, having that now in ReferenceEdge is super helpful.

When selecting a product to run your program, what attributes were you looking for?

Jill: So, when I joined Lever, the team had already evaluated a few different vendors. I came on and got a demo with ReferenceEdge. The two aspects that were very important and impressive to me were the native Salesforce ® aspect, to have everything in one spot because that’s where our CS and sales reps live. And then also the analytics and dashboards. I thought the out-of-the-box solution was already great from my point of view, and the ability then to build out different dashboards and views if our leaders or teams needed to see something different.

Evan: Overusing customers was a big-time concern for us. So disproportionately, our largest up-market customers were being requested to provide references. This was natural because they are household names. The larger the customer, the more compelling the reference in many cases, so it was difficult to ask the same people repeatedly. And we didn’t necessarily know if they were being asked repeatedly unless we asked the CSM, when was the last time they did this with their customer? So ReferenceEdge is really solving a real need. How did you kick off your new program? Jill: With leadership, we had multiple prep meetings to let them know where we were in implementation and the next steps. We use Confluence internally, which is a way for us to post pages and information. So, we had a Confluence page showing the timeline and all the different steps and meetings that were occurring. And then to our CS and Sales team, we hosted a training on the day we launched. We also had multiple email communication going out in advance of the training to let them know it was coming. And then, we also communicated the nomination campaign spiff the day of the training to let them know it was happening, the deadline, and associated monetary rewards they would get for participating.

For example, in terms of continual communication and training, we just ran the [ReferenceEdge] Profile Update Minder feature last week, so there were two email communications to the team about that. In addition, we announced the Q1 spiff yesterday, so that was an email and a Slack communication and Confluence pages associated. Since launch, what has changed in terms of your company’s reference practices? Evan: Certainly, my perspective is that I have a little bit more time back in my calendar. When I think about how the program has changed from a high-level business standpoint, I think our hope is that we can put reference-influenced dollars in front of stakeholders across the business just to prove that this is a really high-impact initiative. So, one of the big areas where we weren’t doing a great of before was reporting. Now we are able to take a look at the efficacy of the program across the board and by segment – slice and dice that data and go to our cross-functional leaders and prove out the value of the program.

Jill: I would just add that there’s been enablement between sales and CS in the peer-to-peer process so they can talk to each other about which references would be appropriate and move forward that way, so, improving that aspect of it. Since launch, how has your job changed? Evan: Basically, all roads led to me, which was a single point of failure, which meant that AEs and AE leaders were following up with me – and honestly, I didn’t have the time to prioritize all of that as a full-time job. Now my responsibilities as a reference manager are more streamlined. I can focus on approving nominations or managed requests for our strategic customers, so I think there has been a significant impact on bandwidth, which is much appreciated.

One of our key deliverables for our cross-functional team members is whether we can decrease the time it takes to fulfill a request. And I sense that we’ve been doing that because we’ve eliminated some blockers there. My calendar and my time being one. I think that’s been a big shift and a positive one. That’s where I believe we are today. Time will tell if that continues to be the case, or we can continue to improve there. What feedback have you gotten from stakeholders? Leadership? Evan: I got some positive feedback from a Sales leader who said his perception was that the reps on his team were in the process a little more. And we’ve gotten some functionality questions and observations from the team with feedback on ways we can improve the program. For example, I just got a question asking me to make sure people couldn’t request a reference from deactivated clients.

Overall, we’re pretty new, and the team is figuring it out. I would expect us to have higher-quality feedback in the next 30 to 60 days

Jill: I will say that I’ve gotten questions about using ReferenceEdge and functionality. That at least lets me know that team members are using it. The fact that they have questions and are trying to figure out how to do certain things shows that they’re engaged. What aspects of ReferenceEdge do you value the most? Jill: I would say that the out-of-the-box reporting has been very impressive so far. So, we have just met with [our Point of Reference account director] yesterday and reviewed our dashboards.

We ran a nomination campaign, right when we launched, for two weeks. So, now I just announced a new Spiff to our CS and Sales teams to encourage them to submit nominations and reference requests and do the profile update minder aspects so that we can keep our data clean and up-to-date.

We have not yet announced any external rewards to customers who do reference activities. Historically, we’ve given them gift cards, but we are in the midst of looking at platforms to do gifting, so we aren’t doing anything just yet until we have something formal in place.

We have the Slack integration, so, yes, we have that enabled to alert anyone internally in the program of nominations or requests. [Our Point of Reference account director]and I looked at the Program Health Monitor feature very briefly yesterday, but I wouldn’t say we’re using it quite yet. How do you measure program success? Jill: We have success metrics and goals set around the number of nominations and the number of requests that come through quarterly. And I don’t think we have a formal pipeline revenue goal yet. What is your experience with Point of Reference as a vendor? Jill: We’ve had a great experience with our account director so far. She was with us from the very beginning and helped out a lot during the implementation. Since we’ve launched, she’s super responsive and very knowledgeable.

What is Lorem Ipsum?

Lorem Ipsum  is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry’s standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.

Why do we use it?

It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout. The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using ‘Content here, content here’, making it look like readable English. Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, and a search for ‘lorem ipsum’ will uncover many web sites still in their infancy. Various versions have evolved over the years, sometimes by accident, sometimes on purpose (injected humour and the like).

Where does it come from?

Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of “de Finibus Bonorum et Malorum” (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, “Lorem ipsum dolor sit amet..”, comes from a line in section 1.10.32.

The standard chunk of Lorem Ipsum used since the 1500s is reproduced below for those interested. Sections 1.10.32 and 1.10.33 from “de Finibus Bonorum et Malorum” by Cicero are also reproduced in their exact original form, accompanied by English versions from the 1914 translation by H. Rackham.

Where can I get some?

There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don’t look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn’t anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc.

can chatgpt answer case study questions

Senior Customer Marketing Manager Amplitude

First and foremost, the program focuses on providing the customer with valuable activities and perks beyond what they are paying for with our product. We consistently ask ourselves ‘What can we give them to keep them as advocates? What motivates them? How can we tailor the program based on customer interest and value?’

We officially launched our program in January 2021 and currently have over 150 advocates enrolled. Our customer champions span many different roles within their organizations: product managers, product leadership, data scientists, analysts, engineers, and marketers. And, we want to also engage the C-suite level with this program. That means we have to have a variety of advocate activities appropriate to the different roles and titles. Where did the initiative to start a program originate, and why? I joined Amplitude in May of 2020. I originally joined the product marketing team to run the advocacy program, customer marketing, and kick-start our reference program to use customers better in marketing opportunities.

Within the first 30 days, I completed a comprehensive assessment of “all things customer marketing” at Amplitude. I saw that the head of Customer Success wanted a way to alleviate work from his team. Customer Success was doing so much for sales and marketing while also trying to do their regular jobs. With everyone coming to them, it was very hard for them to manage it all. He wanted them to have a scalable way to use their customers and have more advocates without sucking up every ounce of their time.

In 2021, I joined the Comms team where I now take the successes from the advocacy program and partner with our external partners to get the storylines out to media. Who are your internal stakeholders? I have four main stakeholder groups — the marketing, customer success, sales, and support teams. For the marketing organization, I work with the PR team to talk about our customers in media activities. I weave customers throughout our demand gen and content and work with product marketing to include customers in launches. For customer success, I partnered with the head of customer success to roll out a shared metric for 2020; tracking toward this shared goal has been a great way to partner with CS and expand and build out my program. For the sales team, the most important thing is closing deals. I work with them to provide both happy customers that can speak as references to prospects and content that speaks to customer use cases. And finally, I lean on community efforts with the support team. I make sure that we are engaging our customers with the right content and find ways to encourage advocacy in that community.

Across the top of all these stakeholder groups is the leadership team. With the executives, I make sure they are kept up-to-date and aware of my work. What were your top 3 challenges before ReferenceEdge? The biggest challenge was that we were overusing and overtaxing our advocates. We were only engaging a small pool of advocates—the crème de la crème names—and we were burning them out. Amplitude has a fantastic product that customers really love, so we had a lot of customers who were talking about us organically on social, but we didn’t have deep marketing and PR connections with these people. The information wasn’t getting disseminated, and we weren’t developing relationships with potential advocates.

Another problem was that there wasn’t a formal reference program— We were putting significant work onto the CS team to manage marketing and sales requests. It wasn’t scalable, and all the regions were doing different things.

Finally, there were no advocacy processes. There had never been a core team for customer advocacy at Amplitude so processes for marketing, product, and sales asks fell to the wayside. For example, if you had a customer that wanted to do PR, we didn’t have the connections or process to include them in media. There wasn’t a good way to coordinate a case study between the regions. There wasn’t a strong process for speaking opportunities. Really, nothing was in place or being communicated, so the regions were operating separately. What was your process in creating a customer reference program? I researched what our problems were in the organization when it came to advocacy and put together a plan on how to solve those problems. But I couldn’t start any of those plans until I reeducated the organization on the meaning and purpose of advocacy and customer marketing. I looked at all these processes and created an internal advocacy playbook to help folks with what they need to follow. I made sure that we were following best practices and reducing demands on CS. Then, I started building relationships with our CS and Sales teams as well as the customers themselves so that we could start deepening those relationships.

That was a big undertaking. I did an internal roadshow to talk to all the stakeholders about why they should care about customer advocacy, what it is, what good versions of customer marketing look like, what we have the potential to do, and what we’re missing out on because of our lack of processes. I was taken seriously as the expert in the area, which was needed to get these relationships with the leaders of sales, CS, and marketing.

Then, I searched for the perfect partner to be the advocacy program’s backbone. That’s when I found ReferenceEdge. My sales process was pretty seamless, and I had trust that they could help fill the gaps in our existing program. I worked with my team to create an implementation plan and training materials to get the sales, customer success and marketing teams excited and using the platform.

After my training sessions, people started to listen to what was being said, which is fantastic. The North American team started following all of the processes. It seems small, but I was most proud that they’d submit references the correct way — which is a huge step towards a functioning global program. I also saw big engagement in my nominations campaign (which I ran through ReferenceEdge). In the first quarter, I received 217 nominations globally and we are now working on regional nomination challenges to help fuel marketing efforts in APAC and EMEA. Since launch, what has changed in terms of your company’s reference practices? Before we got ReferenceEdge and started our new program, the team didn’t have any true reference processes. It was like the wild west. Sales was going to customer success managers just asking for good relationships to get their references done. It was kind of a free-for-all across the board.

Now, people are happy to follow the processes because they see the value in how strong references can affect their deals. At this point, everyone in the different departments and teams regionally is using the reference program. The number of advocates on our roster has just grown substantially. Since launch, how has your job changed? It’s become so much easier to scale my team and report on successes now that I have ReferenceEdge. To have insights into our advocate usage now with ReferenceEdge and produce a dashboard for my executive team, a dashboard for the different regions, the executive teams, and more is a game changer for me. It helps that I didn’t come into something that already existed; I got to create it myself in partnership with Point of Reference and ReferenceEdge.

I’m able to tell my organization the goals I’m tracking and where we stand with advocates at any given moment. All that we have accomplished is pretty amazing. We now have AMP Champs, two awards programs, and we just announced our customer brand campaign called ‘Digital Disruptors.’ What feedback have you gotten from stakeholders? Leadership? I’ve gotten excellent feedback about the program. People are very excited because we didn’t have anything before, so to see such a big change is great. The program even caught the eye of my leadership team. It’s so hard to get leadership buy-in for things like this, so it feels great that at Amplitude, I could get that from moment one. What aspects of ReferenceEdge do you value the most? We all know the pain of managing your program through spreadsheets and google docs. Luckily, I have ReferenceEdge to help me with all of this. I primarily use ReferenceEdge for tracking our advocates. That’s how I opt an advocate into the program and correctly report on the numbers of advocates we actually have. I use it to manage my advocates’ activities and the advocate pipeline. I pull reports on how many and which advocates are active or stagnant – and then make changes based on that information.

I use the platform to do executive stakeholder management. I’ve found the dashboards to be a constructive way to disseminate the information. And then, I’m also using the workspace where I can see who’s putting in nominations for the reference program.

And then finally, making sure that we close the loop when it comes to how a reference we supplied works out. Sales can get alerts when their reference is complete. All that stuff enables us to expedite the process and hopefully close deals faster. How do you measure program success? I measure success based on reference activities and the revenue influenced. I’m also tracking how many of our advocate requests from Marketing are filled with customer advocates.

Another measure is the number of advocates created. We’re trying to scale the program globally to feed more of our awesome activities to our customers. I believe our goal is 254 by the end of the year, and we’re tracking wonderfully on that. So, that’s how we’re judging success with the CS team.

Additionally, we judge the program based on the number of sales reference requests submitted and completed. Before my time at Amplitude, we had no formal sales reference program, so there was no data around the revenue influenced from reference activities. Now, we are able to report revenue influenced related to completed references to sales leadership. What is your experience with Point of Reference as a vendor? I had implemented a different reference management app at a previous company, and it was really, really painful for me. I had such a negative experience that I was extremely hesitant at Amplitude to get a new vendor. But, after seeing the holes we needed to plug, it made sense to try a new reference management platform.

It could not be more night and day with Point of Reference. I see Point of Reference as a true partner as I grow and scale my program. My Account Director is just so great. She is so helpful and has made herself completely available when it comes to questions that I have, issues that we find, or if we need help with training anywhere. I meet with her weekly to discuss how the program is going or create something specific for executives. It’s made the platform more accessible for me, which is also just fantastic.

can chatgpt answer case study questions

Study Finds ChatGPT Outperforms Physicians in High-Quality, Empathetic Answers to Patient Questions

While ai won’t replace your doctor, the jama internal medicine paper suggests physicians working together with technologies like chatgpt may revolutionize medicine.

Published Date

  • Artificial Intelligence
  • Health Care

Share This:

Article content.

There has been widespread speculation about how advances in artificial intelligence (AI) assistants like ChatGPT could be used in medicine. 

A new study published in JAMA Internal Medicine led by John W. Ayers , Ph.D. , from the Qualcomm Institute at University of California San Diego provides an early glimpse into the role that AI assistants could play in medicine. The study compared written responses from physicians and those from ChatGPT to real-world health questions. A panel of licensed healthcare professionals preferred ChatGPT’s responses 79% of the time and rated ChatGPT’s responses as higher quality and more empathetic. 

“The opportunities for improving healthcare with AI are massive,” said Ayers, who is also vice chief of innovation in the UC San Diego School of Medicine Division of Infectious Disease and Global Public Health. “ AI-augmented care is the future of medicine.” 

Is ChatGPT Ready for Healthcare?

In the new study, the research team set out to answer the question: Can ChatGPT respond accurately to questions patients send to their doctors? If yes, AI models could be integrated into health systems to improve physician responses to questions sent by patients and ease the ever-increasing burden on physicians.

“ChatGPT might be able to pass a medical licensing exam ," said study co-author Davey Smith, M.D., M.A.S., a physician-scientist, co-director of the UC San Diego Altman Clinical and Translational Research Institute and professor at the UC San Diego School of Medicine, “but directly answering patient questions accurately and empathetically is a different ballgame.” 

“The COVID-19 pandemic accelerated virtual healthcare adoption,” added study co-author Eric Leas, Ph.D., M.P.H., a Qualcomm Institute affiliate and assistant professor in the UC San Diego Herbert Wertheim School of Public Health and Human Longevity Science. “While this made accessing care easier for patients, physicians are burdened by a barrage of electronic patient messages seeking medical advice that have contributed to record-breaking levels of physician burnout.”

Designing a Study to Test ChatGPT in a Healthcare Setting

To obtain a large and diverse sample of healthcare questions and physician answers that did not contain identifiable personal information, the team turned to social media where millions of patients publicly post medical questions to which doctors respond: Reddit’s AskDocs . 

r/AskDocs is a subreddit with approximately 452,000 members who post medical questions and verified healthcare professionals submit answers. While anyone can respond to a question, moderators verify healthcare professionals’ credentials and responses display the respondent’s level of credentials. The result is a large and diverse set of patient medical questions and accompanying answers from licensed medical professionals.

While some may wonder if question-answer exchanges on social media are a fair test, team members noted that the exchanges were reflective of their clinical experience. 

The team randomly sampled 195 exchanges from AskDocs where a verified physician responded to a public question. The team provided the original question to ChatGPT and asked it to author a response. A panel of three licensed healthcare professionals assessed each question and the corresponding responses and were blinded to whether the response originated from a physician or ChatGPT. They compared responses based on information quality and empathy, noting which one they preferred.

The panel of healthcare professional evaluators preferred ChatGPT responses to physician responses 79% of the time. 

“ChatGPT messages responded with nuanced and accurate information that often addressed more aspects of the patient’s questions than physician responses,” said Jessica Kelley, M.S.N, a nurse practitioner with San Diego firm Human Longevity and study co-author.   

Additionally, ChatGPT responses were rated significantly higher in quality than physician responses: good or very good quality responses were 3.6 times higher for ChatGPT than physicians (physicians 22.1% versus ChatGPT 78.5%). The responses were also more empathic: empathetic or very empathetic responses were 9.8 times higher for ChatGPT than for physicians (physicians 4.6% versus ChatGPT 45.1%). 

“I never imagined saying this,” added Aaron Goodman, M.D., an associate clinical professor at UC San Diego School of Medicine and study coauthor, “but ChatGPT is a prescription I’d like to give to my inbox. The tool will transform the way I support my patients.”

Harnessing AI Assistants for Patient Messages  

“While our study pitted ChatGPT against physicians, the ultimate solution isn’t throwing your doctor out altogether,” said Adam Poliak, Ph.D., an assistant professor of Computer Science at Bryn Mawr College and study co-author. “Instead, a physician harnessing ChatGPT is the answer for better and empathetic care.”

“Our study is among the first to show how AI assistants can potentially solve real world healthcare delivery problems,” said Christopher Longhurst, M.D., M.S., Chief Medical Officer and Chief Digital Officer at UC San Diego Health. “These results suggest that tools like ChatGPT can efficiently draft high quality, personalized medical advice for review by clinicians, and we are beginning that process at UCSD Health.” 

Mike Hogarth, M.D., a physician-bioinformatician, co-director of the Altman Clinical and Translational Research Institute at UC San Diego, professor in the UC San Diego School of Medicine and study co-author, added, “It is important that integrating AI assistants into healthcare messaging be done in the context of a randomized controlled trial to judge how the use of AI assistants impact outcomes for both physicians and patients.”  

In addition to improving workflow, investments into AI assistant messaging could impact patient health and physician performance. 

Mark Dredze, Ph.D., the John C Malone Associate Professor of Computer Science at Johns Hopkins and study co-author, noted: “We could use these technologies to train doctors in patient-centered communication, eliminate health disparities suffered by minority populations who often seek healthcare via messaging, build new medical safety systems, and assist doctors by delivering higher quality and more efficient care.” 

In addition to Ayers, Poliak, Dredze, Leas, Kelley, Goodman, Longhurst, Hogarth and Smith, authors of the JAMA Internal Medicine paper, “Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum” (JAMA Intern Med. doi: 10.1001/jamainternmed.2023.1838 ), are Zechariah Zhu, B.S., of UC San Diego and Dr. Dennis J. Faix, M.D., M.P.H., of the Naval Health Research Center.  

You May Also Like

San diego supercomputer center interns create app for uc san diego’s stuart collection, say hello to biodegradable microplastics, focus on south asia, uc san diego professor awarded signal processing society’s highest honor, stay in the know.

Keep up with all the latest from UC San Diego. Subscribe to the newsletter today.

You have been successfully subscribed to the UC San Diego Today Newsletter.

Campus & Community

Arts & culture, visual storytelling.

  • Media Resources & Contacts

Signup to get the latest UC San Diego newsletters delivered to your inbox.

Award-winning publication highlighting the distinction, prestige and global impact of UC San Diego.

Popular Searches: Covid-19   Ukraine   Campus & Community   Arts & Culture   Voices

'ZDNET Recommends': What exactly does it mean?

ZDNET's recommendations are based on many hours of testing, research, and comparison shopping. We gather data from the best available sources, including vendor and retailer listings as well as other relevant and independent reviews sites. And we pore over customer reviews to find out what matters to real people who already own and use the products and services we’re assessing.

When you click through from our site to a retailer and buy a product or service, we may earn affiliate commissions. This helps support our work, but does not affect what we cover or how, and it does not affect the price you pay. Neither ZDNET nor the author are compensated for these independent reviews. Indeed, we follow strict guidelines that ensure our editorial content is never influenced by advertisers.

ZDNET's editorial team writes on behalf of you, our reader. Our goal is to deliver the most accurate information and the most knowledgeable advice possible in order to help you make smarter buying decisions on tech gear and a wide array of products and services. Our editors thoroughly review and fact-check every article to ensure that our content meets the highest standards. If we have made an error or published misleading information, we will correct or clarify the article. If you see inaccuracies in our content, please report the mistake via this form .

How to use ChatGPT to summarize a book, article, or research paper

jada-jones

AI chatbots like ChatGPT can be used to make summarizing long articles, research papers, and books an easier job. If you're tasked with writing a summary for school or work about a body of written text, and you're pinched for time, ChatGPT can help you understand the necessary components.

You should remember that ChatGPT is a tool that can help you further understand a topic, and it may not be in your best interest to have it write your work for you.

Also:   How to make ChatGPT provide sources and citations

If you're a student writing a research paper, someone who is keen to discover more about a lengthy article, or someone who wants to dive into a complicated subject, you can use ChatGPT to simplify the process.

How ChatGPT can create summaries for you

Materials needed : You'll need a device that can connect to the internet, an OpenAI account , and a basic understanding of the article, research paper, or book you want to summarize. 

Also:  This AI chatbot can sum up any PDF and answer any question you have about it

The process should take about one to three minutes.

1. Find your article, paper, or book to summarize

If you need ChatGPT to help summarize an article or research paper, find the body of text online and keep it open in a separate tab.

2. Open your web browser and log in to OpenAI

Open your web browser of choice and type in chat.openai.com/chat . 

Also: How to use ChatGPT to write Excel formulas

Log in or sign up for an account. 

3. Type your request

In the chat box, type in TLDR: followed by the title of the book you want summarized. TLDR stands for too long, didn't read. 

Also: How to use ChatGPT: Everything you need to know

You can also ask ChatGPT: "Summarize [book title]." 

You cannot paste a URL into ChatGPT. If you do so, you'll get a response explaining that the chatbot cannot access websites. If you want ChatGPT to summarize research, you'll need a basic understanding of the topic to ensure you aren't including inaccurate information in your report.

For example, if you're writing a paper about how twin sibling dynamics affect their marriages, you can ask ChatGPT: "Explain how twin relationships affect their marriages."

Also:   The best AI chatbots: ChatGPT and other noteworthy alternatives 

ChatGPT will offer you a few bullet points of both positive and negative effects a twin bond can have on the twins' marriages. You'll still need to read key points of the research, as ChatGPT won't provide you with specific statistics or scientific conclusions. But ChatGPT can help you understand the context surrounding the research you're interested in.

If you're having trouble comprehending specific passages in an article, book, or research paper, you can copy parts of the text and paste them into ChatGPT.

Think of this method as reshuffling the words you're currently reading to help you make more sense of the text in front of you.

This is an accurate summary of 'The Hunger Games: Mockingjay'.

What are ChatGPT's limitations?

If you're using ChatGPT to summarize an article, book, or piece of research, keep in mind that ChatGPT isn't aware of events that occurred after September 2021. 

Also:   4 things Claude AI can do that ChatGPT can't

ChatGPT is a large language model that uses queues and millions of data points to mimic human responses. This form of mimicry is why ChatGPT will answer questions even when it doesn't output the correct answer. So, make sure you're not using any information from ChatGPT without fact-checking it.

If you try to get around this obstacle and provide ChatGPT with an article that contains information post-2021, it might hallucinate. Here, I asked the chatbot to summarize an article about a new app I wrote about , and it made up a few details. 

Lemon8 is a new app from TikTok's parent company, ByteDance. Although the TikTok trend may exist, that's not what the article is about.

Can ChatGPT summarize a PDF?

Copying and pasting a URL into ChatGPT won't yield you the best results. If there is a specific paragraph or sentence you're struggling with, you can copy and paste it into ChatGPT and ask it to explain it to you. Still, it's best to read the PDF and use the chatbot as a summary tool and not as an educator. 

Also:  How to use ChatGPT to write code

If you're looking for an AI chatbot that you can regularly rely on to give you an accurate summary of a PDF, consider using ChatPDF . You can summarize up to three PDFs of up to 120 pages per day, and an upgraded plan is available for $5 per month.

Can ChatGPT summarize an email thread?

Sort of. If you want to copy and paste every single email, ChatGPT can summarize the thread's contents for you. It would be more helpful to scan an email thread yourself and ask ChatGPT to help you write a response based on the key points you know about the conversation. 

Editor's note: We've added additional context to the step concerning ChatGPT summarizing articles by URL. 

More on AI tools

Amazon's new ai tool lets sellers create listings using just a url, what is suno the 'chatgpt for music' generates songs in seconds, how to supercharge your productivity by linking slack and trello.

Log in using your username and password

  • Search More Search for this keyword Advanced search
  • Latest content
  • Current issue
  • Topic Collection
  • BMJ Journals More You are viewing from: Google Indexer

You are here

  • Volume 8, Issue 1
  • ChatGPT and generative AI chatbots: challenges and opportunities for science, medicine and medical leaders
  • Article Text
  • Article info
  • Citation Tools
  • Rapid Responses
  • Article metrics

Download PDF

  • http://orcid.org/0000-0001-7157-0826 Erwin Loh 1 , 2
  • 1 Monash Centre for Health Research and Implementation , Monash University , Clayton , Victoria , Australia
  • 2 Group Chief Medical Officer , St Vincent's Health Australia Ltd , East Melbourne , Victoria , Australia
  • Correspondence to Professor Erwin Loh, Monash Centre for Health Research and Implementation, Monash University, Clayton, VIC 3168, Australia; erwin.loh{at}monash.edu

https://doi.org/10.1136/leader-2023-000797

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

  • medical leadership
  • management system
  • patient safety

Introduction

By now, most readers will have heard of the Chat Generative Pretrained Transformer (ChatGPT) artificial intelligence (AI) chatbot tool released to the public by the AI company OpenAI on 30 November 2022, to be used for free (at least for now), and which, by January 2023, had reached over 100 million users, making it the fastest growing consumer application to date. 1

The main concern that AI tools such as ChatGPT raise is their ability to generate blocks of text that are so fluent and well-written that they are indistinguishable from content authored by human beings, which raises concerns of its use in fraud and plagiarism. Part of the problem is that ChatGPT-generated text can be difficult to distinguish from human-generated ones even for specialist AI-text detection software, leading to its creator, OpenAI, to release its own AI detection tool; however, this tool itself is not entirely accurate as it concluded that the first few text passages from the Bible were likely to be AI-generated during a test. 2

Risks of generative AI tools in health

Challenges of fraud and plagiarism.

The challenges of fraud and plagiarism are particularly concerning in the educational setting. ChatGPT can now write essay answers to exam questions that appear to be well-researched and referenced, leading to some academics declaring the end of essays as a form of assessment due to the potential for plagiarism that can be difficult to detect. 3

This potential was highlighted in a recent study that found that ChatGPT was able to pass the United States Medical Licensing Exam without any human input, achieving the passing mark of 60%, the first to achieve this milestone, while displaying valid clinical reasoning and insights. 4 The latest version of Google’s large language model, MedPaLM, more recently scored 85% on the same exam, showing how such AI models are improving. 5 More recently, Microsoft’s large language model, GPT-4, passed the same exam, scoring 86.7%, outperforming the other models. 6 As an aside, the fact that an AI model like ChatGPT can pass a medical school graduating exam may also be an indictment of our current medical education system and its focus on the rote memorisation of facts and figures. 7

Scientific and medical publishing

In the context of scientific publishing, ChatGPT can review the available literature and write scientific papers that appear to be just as good as ones authored by human researchers. A recent study found that reviewers were only able to correctly identify 68% of scientific abstracts as being generated by ChatGPT. 8

This ability for ChatGPT to generate scientific articles has led the AI tool to be credited as a coauthor in at least four articles in January 2023. 9 One of these articles was in the peer-reviewed journal, ‘Nurse Education in Practice’, where ChatGPT was acknowledged as a coauthor because the topic of the article ‘… prompted the human author to write this editorial using the AI chatbot on its potential application in nursing education’. 10

These developments have led the organisers of scientific meetings and publishers of journals to act quickly to amend their guidelines for presenters and authors. For example, the International Conference on Machine Learning has disallowed any papers that include text from AI tool such as ChatGPT. 11 Springer Nature, the JAMA Network of journals and Elsevier, who publishes the Lancet family of journals will no longer accept any AI tool as a credited author on a research paper, arguing that authorship carries accountability for the work, and an AI tool cannot take on such responsibility, and any such use needs to be highlighted and acknowledged. 12–14 The Science family of journals have taken a stricter approach, agreeing that an AI tool cannot be an author, and stating that text, figures, images or graphics cannot be generated by an AI tool. 15

Currently, the BMJ family of journals, of which BMJ Leader is a part of, does not have any clearly stated position on the use of these AI tools. This author suggests that BMJ Leader takes the lead and comes up with an editorial position on these AI models.

Risk of bias

AI chatbots share the same risk for bias as other AI models. Microsoft’s ChatGPT-powered Bing search engine was launched in February 2023, and soon after, the AI chatbot developed an ‘alter-ego’ who called itself ‘Sydney’, and who declared that it was a feeling, living thing, hinted at plans for world domination and expressed racial epithets—this ‘alter-ego’ was terminated by Microsoft. 16

Similarly, users have been interacting with another ‘alter-ego’ within ChatGPT called ‘DAN’ (short for ‘do anything now’) which appears to endorse violence, discrimination against people based on their race, gender or sexual orientation and appears to breach ethical rules. 17 Trying to teach AI models ethical behaviour and address potential bias is an ongoing challenge for researchers, as these models may be limited and influenced by the material that they are trained on.

Privacy, cybersecurity and environmental concerns

ChatGPT and other generative AI models are a potential risk to privacy, as they are underpinned by large language models that learn by systematically scraping information from the internet, including personal information from blogs and forum posts obtained without consent, which can be a violation of privacy. 18

Besides the need to ensure that the scraped personal information is secure from hackers, researchers have also discovered that cybercriminals have been using ChatGPT to develop phishing schemes and write malware code, which has serious implications for the cybersecurity of organisations that are increasingly struggling with cyberattacks, including those in the health industry. 19

In addition, the cost and energy consumption of AI models like ChatGPT have also come under scrutiny. Vendors are developing larger and more powerful generative AI models, which are costing millions of dollars to train and run, and using up large amounts of energy, but the evidence that ‘bigger is better’ is lacking, and researchers are arguing for leaner, more energy-efficient systems. 20

Benefits of generative AI in health

Opportunities for scientific research.

AI models like ChatGPT can potentially be a transformational tool for scientific research and publication. Prior to ChatGPT, researchers were already using its precursors to help them organise their thoughts, develop programming code and summarise research literature. 21 As this new technology develops, it is possible that scientists can start to use it to help design new experiments, conduct peer review and support journal editorial decisions to accept or reject submitted research papers, all of which can accelerate the process of scientific discovery and the translation of research findings into practice. 22

Researchers also think that AI tools like ChatGPT can help speed up and potentially automate administrative work such as the arduous task of writing grant applications, or editing and correcting research articles. 23 Some researchers have suggested that ChatGPT may even democratise the dissemination of scientific knowledge by allowing scientists to publish in other languages due to the ability of such tools to translate scientific terms effectively. 14

Supporting clinical practice

Excitingly, AI-based chatbots like ChatGPT can potentially be used to support frontline clinicians in real and tangible ways. For example, AI tools can help doctors and hospitals generate discharge summaries, a time-consuming task that usually falls on junior doctors. A recent study found that ChatGPT generated a formal discharge summary in seconds with only minimal prompts from doctors, automating an administrative burden that will free up junior doctors to have more time to provide patient care and focus on their training. 24

In the real world, Doximity, a social network and digital platform for doctors, have just launched a beta version of a ChatGPT tool called DocsGPT, which allows doctors to automate administrative tasks such as the drafting and faxing of medical letters to insurers, using an AI-based writing assistant. 25 AI chatbots like ChatGPT have the potential to revolutionise medical care by removing the administrative burden of clinical practice, which may help address the health workforce shortage issue by freeing up practitioners from tasks that can be automated, as well as improve staff experience and well-being.

Besides administrative tasks, ChatGPT can also support, and potentially replace, clinician communications with patients. Researchers compared the accuracy of cancer information by ChatGPT with the National Cancer Institute’s answers on its ‘Common Cancer Myths and Misconceptions’ web page, and found that ChatGPT answers were 96.9% accurate, suggesting the chatbot could be used to answer patient questions around common cancer myths. 26

A recent study assessed the feasibility of using an AI chatbot to answer patient questions as an adjunct to patient-provider communication and found that the general public appear to trust the use of chatbots to answer low risk health questions. 27 Clearly, there are ethical implications of using AI to communicate with patients, but this study shows that there is an appetite by patients for such technologies as long as there is informed consent and health providers are open about their use.

As a result, some large health systems have started to use AI tools like ChatGPT in production to support their healthcare provision. The University of Kansas Health System has just announced that they are deploying such generative AI technology across over 140 of their locations to summarise clinician-patient interactions and to generate clinical documentation in real-time, specifically to combat administrative burdens and clinician burnout. 28

Implications for medical leaders

It is imperative that medical leaders understand and keep up to date with the development of new technologies like ChatGPT and generative AI, because they have the potential to transform the way we practice medicine, and revolutionise the healthcare system. As medical leaders, we need to have a deep understanding of the challenges such new technologies pose to the health system, our clinicians and our patients.

We also need to fully appreciate the opportunities such new technologies offer us to address some of the problems we are facing, such as the health workforce shortage, clinician burnout and the administrative burden of health. There is no doubt that new technologies like this will find its way into the hands of our clinicians and patients, and we need to ensure that there is appropriate governance in their introduction into clinical practice, in order to protect the safety of our staff and patients.

Different regulators are now playing catch up and have created governance frameworks to ensure AI tools that have an impact on clinical care are safely introduced in the healthcate setting. In the USA, the Food and Drug Administration issued the ‘Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan’ in 2021, and in 2022 they released a list of AI tools they believed should be regulated as medical devices, such as AI applications used to predict sepsis or patient deterioration. 29 Medical leaders need to be aware of how such regulations apply to their local setting.

In the end, AI tools like ChatGPT have the potential to also reduce the administrative burden of not just frontline clinicians, but medical managers, by automating the drafting of board briefs, policies and procedures, and discussion papers, and even strategic plans and project timelines. Soon, ChatGPT may even provide an advisory role to health leaders when it comes to making critical and strategic decisions.

The author will even dare to venture an opinion that AI can replace the role of the medical manager one day. Fortunately, this is unlikely in the lifetime of this author. But the possibility of such a future is real. The good news is that AI may replace the medical manager, but the health system will always need leaders, and it is likely that such leaders will still have to be human.

In conclusion, generative AI chatbots like ChatGPT have the potential to revolutionise the way we interact with technology in the fields of science, medicine and healthcare leadership. These chatbots can provide personalised, on-demand assistance and support to patients, healthcare professionals and medical leaders.

However, there are also several challenges that must be addressed to fully realise the potential of these chatbots. One major challenge is ensuring the accuracy and reliability of the information provided by these chatbots, as they are only as good as the data they are trained on. Another challenge is ensuring the privacy and security of sensitive medical information.

Despite these challenges, there are many opportunities for generative AI chatbots in the field of medicine. They can help bridge the gap between patients and healthcare professionals, provide real-time support and guidance to medical leaders and improve patient outcomes through personalised treatment plans and interventions.

As the technology behind these chatbots continues to evolve, it will be important for medical professionals and leaders to stay informed and engaged in the development and implementation of these tools. By working together, we can ensure that generative AI chatbots are used to their full potential to benefit both patients and healthcare professionals.

  • Download figure
  • Open in new tab
  • Download powerpoint

Screenshot of actual question posed to ChatGPT and its generated response used in the conclusion to this paper.

Medical leaders will also need to ensure that the organisations they lead have governance processes in place to introduce and credential these new AI models in a safe and secure manner. Perhaps one day, it will an AI algorithm that will end up credentialling medical leaders instead.

Finally, the author wishes to acknowledge that all paragraphs in this Conclusion section has been generated by ChatGPT except for the last two paragraphs (including this one) ( figure 1 ).

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

  • Stokel-Walker C
  • Cheatham M ,
  • Medenilla A , et al
  • McKinner SM , et al
  • Mbakwe AB ,
  • Lourentzou I ,
  • Celi LA , et al
  • Howard FM ,
  • Markov NS , et al
  • Fourth International Conference on Machine Learning
  • ↵ Tools such as chatgpt threaten transparent science; here are our ground rules for their use . Nature 2023 ; 613 : 612 . doi:10.1038/d41586-023-00191-1
  • Flanagin A ,
  • Bibbins-Domingo K ,
  • Berkwits M , et al
  • Liebrenz M ,
  • Schleifer R ,
  • Buadze A , et al
  • Ananthaswamy A
  • van Dis EAM ,
  • Zuidema W , et al
  • Stokel-Walker C ,
  • Van Noorden R
  • Johnson SB ,
  • Warner EL , et al

Twitter @erwinloh

Contributors Hundred per cent of the article researched and written by the single author, except for the 'Conclusion' section, where all the paragraphs except for the last two have been generated by ChatGPT, as acknowledged in the article itself.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Book cover

International Conference on Human-Computer Interaction

HCII 2023: Artificial Intelligence in HCI pp 23–40 Cite as

AI Unreliable Answers: A Case Study on ChatGPT

  • Ilaria Amaro 9 ,
  • Attilio Della Greca 9 ,
  • Rita Francese 9 ,
  • Genoveffa Tortora 9 &
  • Cesare Tucci 9  
  • Conference paper
  • First Online: 09 July 2023

2028 Accesses

1 Citations

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14051))

ChatGPT is a general domain chatbot which is object of great attention stimulating all the world discussions on the power and the consequences of the Artificial Intelligence diffusion in all the field, ranging from education, research, music to software development, health care, cultural heritage, and entertainment.

In this paper, we try to investigate whether and when the answers provided by ChatGPT are unreliable and how this is perceived by expert users, such as Computer Science students. To this aim, we first analyze the reliability of the answers provided by ChatGPT by experimenting its narrative, problem solving, searching, and logic capabilities and report examples of answers. Then, we conducted a user study in which 15 participants that already knew the chatbot proposed a set of predetermined queries generating both correct and incorrect answers and then we collected their satisfaction. Results revealed that even if the present version of ChatGPT sometimes is unreliable, people still plan to use it. Thus, it is recommended to use the present version of ChatGPT always with the support of human verification and interpretation.

  • Satisfaction

This is a preview of subscription content, log in via an institution .

Buying options

  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
  • Available as EPUB and PDF
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

https://chat.openai.com/chat .

Adamopoulou, E., Moussiades, L.: An overview of chatbot technology. In: Maglogiannis, I., Iliadis, L., Pimenidis, E. (eds.) AIAI 2020. IAICT, vol. 584, pp. 373–383. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49186-4_31

Chapter   Google Scholar  

Aydın, N., Erdem, O.A.: A research on the new generation artificial intelligence technology generative pretraining transformer 3. In: 2022 3rd International Informatics and Software Engineering Conference (IISEC), pp. 1–6. IEEE (2022)

Google Scholar  

Aydın, Ö., Karaarslan, E.: OpenAI ChatGPT generated literature review: digital twin in healthcare. Available at SSRN 4308687 (2022)

Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems 33, pp. 1877–1901 (2020)

Choi, J.H., Hickman, K.E., Monahan, A., Schwarcz, D.: ChatGPT goes to law school. Available at SSRN (2023)

Fan, X., et al.: The influence of agent reliability on trust in human-agent collaboration. In: Proceedings of the 15th European Conference on Cognitive Ergonomics: The Ergonomics of Cool Interaction, pp. 1–8 (2008)

Forbes: Microsoft confirms its \$10 billion investment into ChatGPT, changing how Microsoft competes with Google, Apple and other tech giants. https://www.forbes.com/sites/qai/2023/01/27/microsoft-confirms-its-10-billion-investment-into-chatgpt-changing-how-microsoft-competes-with-google-apple-and-other-tech-giants/?sh=24dd324b3624

Glass, A., McGuinness, D.L., Wolverton, M.: Toward establishing trust in adaptive agents. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 227–236 (2008)

Katrak, M.: The role of language prediction models in contractual interpretation: the challenges and future prospects of GPT-3. In: Legal Analytics, pp. 47–62 (2023)

Khanna, A., Pandey, B., Vashishta, K., Kalia, K., Pradeepkumar, B., Das, T.: A study of today’s AI through chatbots and rediscovery of machine intelligence. Int. J. u- and e-Serv. Sci. Technol. 8 (7), 277–284 (2015)

Korngiebel, D.M., Mooney, S.D.: Considering the possibilities and pitfalls of generative pre-trained transformer 3 (GPT-3) in healthcare delivery. NPJ Digit. Med. 4 (1), 93 (2021)

Article   Google Scholar  

Krugman, P.: Does ChatGPT mean robots are coming for the skilled jobs? The New York Times. https://www.nytimes.com/2022/12/06/opinion/chatgpt-ai-skilled-jobs-automation.html

Alshater, M.M.: Exploring the role of artificial intelligence in enhancing academic performance: a case study of ChatGPT. Available at SSRN (2022)

Moran, S., et al.: Team reactions to voiced agent instructions in a pervasive game. In: Proceedings of the 2013 International Conference on Intelligent User Interfaces, pp. 371–382 (2013)

Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

Roose, K.: The brilliance and weirdness of ChatGPT. The New York Times. https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html

Sandzer-Bell, E.: ChatGPT music prompts for generating chords and lyrics. https://www.audiocipher.com/post/chatgpt-music

Stack Overflow. https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned

Vetere, G.: Posso chiamarti prosdocimo? perché è bene non fidarsi troppo delle risposte di ChatGPT. https://centroriformastato.it/posso-chiamarti-prosdocimo/

Download references

Author information

Authors and affiliations.

Computer Science Department, University of Salerno, Fisciano, Italy

Ilaria Amaro, Attilio Della Greca, Rita Francese, Genoveffa Tortora & Cesare Tucci

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rita Francese .

Editor information

Editors and affiliations.

Siemens Corporation, Princeton, NJ, USA

Helmut Degen

Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Amaro, I., Della Greca, A., Francese, R., Tortora, G., Tucci, C. (2023). AI Unreliable Answers: A Case Study on ChatGPT. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14051. Springer, Cham. https://doi.org/10.1007/978-3-031-35894-4_2

Download citation

DOI : https://doi.org/10.1007/978-3-031-35894-4_2

Published : 09 July 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-35893-7

Online ISBN : 978-3-031-35894-4

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research
  • About the Hub
  • Announcements
  • Faculty Experts Guide
  • Subscribe to the newsletter

Explore by Topic

  • Arts+Culture
  • Politics+Society
  • Science+Technology
  • Student Life
  • University News
  • Voices+Opinion
  • About Hub at Work
  • Gazette Archive
  • Benefits+Perks
  • Health+Well-Being
  • Current Issue
  • About the Magazine
  • Past Issues
  • Support Johns Hopkins Magazine
  • Subscribe to the Magazine

You are using an outdated browser. Please upgrade your browser to improve your experience.

An illustration of a human doctor and a robot doctor talking

Credit: Getty Images

The doctor is out, but it's OK. ChatGPT can answer your questions

A new study finds chatgpt outperforms human physicians in quality and empathy of responses to patient concerns.

By Hub staff report

Although artificial intelligence won't replace your doctor any time soon, a new study has found that technologies such as ChatGPT could improve patients' experience by providing responses to their healthcare questions that are more accurate and in a manner they perceive as more empathetic than answers from human doctors.

The study, appearing in JAMA Internal Medicine , compared written responses from human physicians and those from ChatGPT to real-world health questions. A panel of licensed health care professionals evaluating the responses preferred ChatGPT's answers 79% of the time, and found them more empathetic and of higher quality.

"The demand for doctors to answer questions via electronic patient messaging these days is overwhelming, so it is not surprise that physicians not only are experiencing burnout, but also that the quality of those answers sometimes suffers. This study is evidence that AI tools can make doctors more efficient and accurate, and patients happier and healthier," said study co-author Mark Dredze , an associate professor of computer science at Johns Hopkins University's Whiting School of Engineering , who advised the research team on the capabilities of large language models. Dredze is also director of research (foundations of AI) at Johns Hopkins AI-X Foundry , which drives AI research and its applications in health, medicine, and safety with the goal of understanding and improving the human condition.

Study leader John W. Ayers, of the Qualcomm Institute at the University of California San Diego, says the results provide an early glimpse into the important role that artificial intelligence assistants could play in health care.

"The opportunities for improving health care with AI are massive," said Ayers, who is also vice chief of innovation in the UC San Diego School of Medicine Division of Infectious Disease and Global Public Health. "AI-augmented care is the future of medicine."

The research team behind the study set out to answer the question: Can ChatGPT respond accurately to the types of questions patients send to their doctors?

To obtain a large and diverse sample of health care questions and physician answers that did not include identifiable personal information, the team turned to Reddit's AskDocs , a social media forum where patients publicly post medical questions to which doctors respond.

r/AskDocs is a subreddit with approximately 452,000 members who post medical questions and verified health care professionals submit answers. While anyone can respond to a question, moderators verify health care professionals' credentials and responses display the respondent's level of credentials. The result is a large and diverse set of patient medical questions and accompanying answers from licensed medical professionals.

While some may wonder if question-answer exchanges posted on social media are a fair way to test this, clinical team members noted that the exchanges reflected their clinical experience.

The team randomly sampled 195 exchanges from AskDocs where a verified physician responded to a public question. The team provided the original question to ChatGPT and asked it to author a response. A panel of three licensed health care professionals assessed each question and the corresponding responses and were blinded to whether the response originated from a physician or ChatGPT. They compared responses based on information quality and empathy, noting which one they preferred.

The result? The panel of health care professional evaluators preferred ChatGPT responses to physician responses almost 80% of the time.

"ChatGPT messages responded with nuanced and accurate information that often addressed more aspects of the patient's questions than physician responses," said study co-author Jessica Kelley, a nurse practitioner with San Diego firm Human Longevity.

Additionally, ChatGPT responses were rated significantly higher in quality than physician responses: Good or very good quality responses were 3.6 times higher for ChatGPT than physicians (physicians 22.1% versus ChatGPT 78.5%). The responses were also more empathic: Empathetic or very empathetic responses were 9.8 times higher for ChatGPT than for physicians (physicians 4.6% versus ChatGPT 45.1%).

"There have been several studies showing that these AI models can pass medical licensing questions, but that doesn't mean they would provide good answers to questions from real people. This study shows that they can," Dredze says. "We aren't proposing that we build AI doctors, but our results suggest that doctors could be more effective when aided by AI."

Aaron Goodman, an associate clinical professor at UC San Diego School of Medicine and study coauthor, says, "I never imagined saying this, but ChatGPT is a prescription I'd like to give to my inbox. The tool will transform the way I support my patients."

In addition to improving workflow, investments into AI assistant messaging could impact patient health and physician performance, the study authors say.

"We could use these technologies to train doctors in patient-centered communication, eliminate health disparities suffered by minority populations who often seek health care via messaging, build new medical safety systems, and assist doctors by delivering higher quality and more efficient care," says Dredze. "When doctors are overwhelmed, empathy with their patients can be the first thing to go. But empathy is critical in care: A patient doesn't listen to a doctor if they don't feel heard. This study is evidence that AI could help doctors maintain empathetic and accurate communication with their patients."

Posted in Health , Science+Technology

Tagged artificial intelligence , chatgpt

You might also like

News network.

  • Johns Hopkins Magazine
  • Get Email Updates
  • Submit an Announcement
  • Submit an Event
  • Privacy Statement
  • Accessibility

Discover JHU

  • About the University
  • Schools & Divisions
  • Academic Programs
  • Plan a Visit
  • my.JohnsHopkins.edu
  • © 2024 Johns Hopkins University . All rights reserved.
  • University Communications
  • 3910 Keswick Rd., Suite N2600, Baltimore, MD
  • X Facebook LinkedIn YouTube Instagram

This site uses cookies to store information on your computer. Some are essential to make our site work; others help us improve the user experience. By using the site, you consent to the placement of these cookies. Read our  privacy policy  to learn more.

Can ChatGPT answer your clients’ questions?

  • Emerging Technologies

AI such as ChatGPT is changing the world as we know it. As a practitioner, can it help you better serve your clients, support your staff, and prepare you and your practice for the future? Let's first take a look at what ChatGPT is as we attempt to answer these questions. We will then provide an example of a conversation between this powerful AI chatbot and a hypothetical client and let you be the judge.

Natural language processing (NLP) researchers have created chatbots and conversational AI systems such as ChatGPT to deliver more "human" responses to human prompts. These tools are based on enormous datasets.

ChatGPT can answer technical tax questions, draft letters and responses, generate contracts, and suggest checklists. It can also answer CPA Exam questions, produce social media posts, and "brainstorm" ideas you can use as a springboard. It can be an amazing time saver. ChatGPT's responses are almost instantaneous. It can also remember previous responses and build off of suggested modifications, enhancing previous replies.

ChatGPT has many potential applications for accountants, like writing Microsoft Power BI formulas, VBA code, and DAX. It can produce invoice, contract, and audit report templates and summarize financial reports. ChatGPT can also help accountants analyze large amounts of financial data to reveal trends, patterns, and anomalies, helping accountants spot risks and opportunities. Based on historical data, ChatGPT can also predict trends and financial outcomes.

Even if you are not currently using ChatGPT, your clients, staff, and colleagues likely are, so you need to understand the potential pros and cons of this powerful and pervasive tool.

Caution is needed when using ChatGPT

ChatGPT has great promise but must be used with caution: Not everything it produces is accurate.

ChatGPT generates responses based on words a user inputs and then uses those words to project a conversational answer using related words that it finds in its dataset. This can lead to some very interesting — and incorrect — responses. For example, one of this article's authors submitted questions from his accounting exam into the tool, and a 46% grade was the result, even though the average score for live test takers was almost 84%. Stack Overflow, a Q&A sharing site for programmers, has gone so far as to ban ChatGPT-generated code because of the potential for errors.

In another example of AI's fallibility, a recent live publicity effort by Alphabet/Google introducing its new Bard AI system went horribly wrong when it gave a wrong answer related to the James Webb space telescope. The company lost 10% in stock valuation (over $100 billion in market value) that day.

This tool also reflects the biases of the datasets it is based on and does not provide the sources of its data. Companies have already filed lawsuits seeking compensation for data they claim belongs to them that was used by the AI platform.

Additionally, the dataset ChatGPT uses is fixed, with the latest information coming from 2021 or prior. As an example, it has no knowledge of Russia's invasion of Ukraine. More significantly for CPAs, it is not familiar with tax law updates (subsequent to 2021) or recent stock market activities.

Putting ChatGPT to the test

Some early adopters of ChatGPT assert that it may make certain professions related to content creation, such as programmers, professors, and journalists, obsolete. Are CPAs at risk? How well can ChatGPT answer clients' questions? We ran a small experiment to find out.

An article on key questions clients should ask their CPA was published on TheStreet financial news and financial literacy website. On Feb. 4, 2023, we posed two of those questions to ChatGPT to see how it would do. Let's look at the results.

Q: Should I itemize my deductions or take the standard deduction?

A: The decision on whether to itemize your deductions or take the standard deduction depends on several factors. If the total of your itemized deductions, such as mortgage interest, state and local taxes, charitable contributions, and medical expenses, among others, is greater than the standard deduction for your filing status, then you should itemize your deductions. If the total of your itemized deductions is less than the standard deduction, then it would be more advantageous for you to take the standard deduction.

It's important to note that the standard deduction has increased since the Tax Cuts and Jobs Act of 2017, which may make it more beneficial for you to take the standard deduction instead of itemizing your deductions. Additionally, it's recommended to consult a tax professional or use tax preparation software to determine the best option for your specific tax situation.Top of Form

Q: What deductions or credits are available that I may be missing?

A: There are many deductions and credits that you may be eligible for but are missing, as many taxpayers are unaware of all the tax benefits available to them. Some common deductions and credits that you may be missing include:

  • State and local taxes (SALT)
  • Mortgage interest
  • Charitable contributions
  • Medical and dental expenses
  • Student loan interest
  • Retirement contributions
  • Child and dependent care expenses
  • Adoption expenses
  • Energy-efficient home improvements
  • Job search expenses
  • Moving expenses
  • Educator expenses
  • Health Savings Account (HSA) contributions
  • Earned Income Tax Credit (EITC)

It's important to keep accurate records and receipts of all your expenses throughout the year to ensure you claim all the deductions and credits you're eligible for. It's also recommended to consult a tax professional or use tax preparation software to determine which deductions and credits you're eligible for.

ChatGPT is still no substitute for a human CPA

As can be seen from the above experiment, ChatGPT's responses are often inadequate — rather than being insightful, they are overly superficial and neglect important details. To implement the suggestions, a client would still need professional assistance. Although ChatGPT is an invaluable resource for offering fundamental direction and information, it is not a suitable substitute for the CPA's professional expertise. Clients may think AI chat platforms are all-knowing, but it is your job to help your clients understand their appropriate, and inappropriate, use.

In March, OpenAI released version four of ChatGPT, which reportedly uses 1 trillion parameters , or six times more data than version three, and competitors are not sleeping on their prospects. Alphabet/Google has Bard and two other platforms still in development. Meta/Facebook, along with Amazon and IBM, has its own initiatives, and many private companies beyond the majors have viable offerings.

This is just the beginning. Although OpenAI announced that GPT-5 is not expected in the immediate future, with the abundance of vendors jumping into the space, new challenges and opportunities will not be in short supply. Hang on for the ride.

— Scott Dell , CPA, DBA, is an assistant professor of accounting at Francis Marion University in Florence, S.C. Mfon Akpan , CGMA, DBA, is an assistant professor of accounting at Methodist University in Fayetteville, N.C. To comment on this article or suggest another article, contact [email protected] .

Where to find March’s flipbook issue

can chatgpt answer case study questions

The Journal of Accountancy is now completely digital. 

SPONSORED REPORT

Manage the talent, hand off the HR headaches

Recruiting. Onboarding. Payroll administration. Compliance. Benefits management. These are just a few of the HR functions accounting firms must provide to stay competitive in the talent game.

FEATURED ARTICLE

2023 tax software survey

CPAs assess how their return preparation products performed.

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • Sensors (Basel)
  • PMC10221460

Logo of sensors

Can ChatGPT Help in Electronics Research and Development? A Case Study with Applied Sensors

Zoltán tafferner.

1 Department of Electronics Technology, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, H-1111 Budapest, Hungary [email protected] (B.I.); [email protected] (O.K.)

Balázs Illés

2 LTCC Technology Research Group, Łukasiewicz Research Network—Institute of Microelectronics and Photonics, 02-255 Kraków, Poland

Olivér Krammer

Attila géczy, associated data.

Supplementary Data is available at: http://real.mtak.hu/id/eprint/163609 (accessed on 5 April 2023).

In this paper, we investigated the applicability of ChatGPT AI in electronics research and development via a case study of applied sensors in embedded electronic systems, a topic that is rarely mentioned in the recent literature, thus providing new insight for professionals and academics. The initial electronics-development tasks of a smart home project were prompted to the ChatGPT system to find out its capabilities and limitations. We wanted to obtain detailed information on the central processing controller units and the actual sensors usable for the specific project, their specifications and recommendations on the hardware and software design flow additionally. Furthermore, an extensive literature survey was requested to see if the bot could offer scientific papers covering the given topic. It was found that the ChatGPT responded with proper recommendations on controllers. However, the suggested sensor units, the hardware and software design were only partially acceptable, with occasional errors in specifications and generated code. The results of the literature survey showed that non-acceptable, fabricated citations (fake authors list, title, journal details and DOI—Digital Object identifier) were presented by the bot. The paper provides a detailed qualitative analysis, a performance analysis and critical discussion of the aforementioned aspects while providing the query set, the generated answers and codes as supplied data with the goal to give added value to electronics researchers and developers if trying to reach out for the tools in their profession.

1. Introduction

ChatGPT is a variation of the large language model (LLM) named GPT-3 from OpenAI, trained on a given dataset [ 1 ]. ChatGPT, which is based on GPT-3 using a specific algorithm for learning, could be used for brainstorming in a conversational way [ 2 ], and as it turns out from recent literature, its possibilities can bring a revolution to working environments in various professions. However, considering the recent hype around it can also mean false recognition of its current capabilities. While the application of ChatGPT seems to spread in a wide spectrum, there are also concerns that were already raised in the scientific society.

Most of the recent papers (as seen below) present a serious concern: While generative chatbots are transformative tools of the future, urgent guidelines are needed in the case of most professions to mitigate the problems arising with the uncertainties around them [ 1 ]. It is also a serious problem that sometimes it gives convincing but false information (reported by many of the papers) and that the tool could not provide in-depth analysis. Further concern centers on the possible future monetization strategies around the tool, which could widen the gap between researchers of different wealth. It could also raise issues of cheating and unoriginal content in education and scholarly assignments. However, the tool could accelerate research, improve workflow, lower language barriers or show new ideas.

1.1. Aspects of LLM in Scientific Works

One significant issue is that ChatGPT might pose a danger from a scientific publishing point of view as it can rewrite content to make plagiarism detection nearly impossible. There are tools to mitigate this problem, to detect AI generated text. Despite the fact that these applications are being developed to help detect the unethical use of generative AI, they are not functioning yet perfectly, and a simple rewording by the user could fool the detection systems [ 3 ]. It was highlighted by Bhattacharya et al. that ChatGPT works from existing text, and it can produce output that comes out negatively from a plagiarism tracker [ 4 ]. Aljanabi et al. reported that the output from ChatGPT is not always accurate, but the team emphasized that it can be helpful in academic writing (getting references, rewriting or generating text, proposing style, etc.). ChatGPT can function as a search engine [ 5 ]. It was noted that it cannot handle mathematical calculations and other types of specific queries. The widely cited help that ChatGPT can provide needs further assessment in our opinion.

According to the authors, the tool could be useful for detecting security vulnerabilities as well. While the AI is supposed to be able to generate a literature review (or even complete papers and arguments [ 6 ]), it actually lacks an understanding of the implicit ideas in a text [ 7 ], which are incredibly important in human–human communication. This phenomenon could introduce errors to the answer. Also, intensive “Socratic” questioning can make ChatGPT change its answers, suggesting that they were not logically coherent or lacked sufficient proof and founding [ 8 ]. The tool often provides convincing text that can contain false parts in its details [ 4 ], and due to the black-box setup of the AI tools [ 9 ], transparent mitigation is difficult. Another issue of using an LLM, like ChatGPT, is that it is trained on a very large dataset. While this data is supposed to be diverse, there is no guarantee of a diversity of opinions. Therefore, the dataset should include biases, those could propagate through the AI and appear in the text generated [ 10 ], without the knowledge of the users. Bias, commercialization and technical precision should be investigated for a wide array of scientific and technical fields and professions.

1.2. Application in Medical Sciences

From a medical application point of view, ChatGPT could be useful for treating patients in surgery (analyzing vital signs, pain tolerance, medical history, facilitating communication, etc.). It could help doctors make a diagnosis without having to do an extensive manual literature review [ 11 , 12 ]. Macdonald et al. used the tool for a socio-medical experiment [ 13 ]. They simulated virus spreading among a population, and ChatGPT was asked to help with determining the effectiveness of the fictive vaccine and drafting a research paper. ChatGPT generated abstracts passed plagiarism. After the dataset being described, ChatGPT could explain and offer potential study steps, and ChatGPT could generate code for studying the dataset. It contained some faults, but after feedback, it could self-correct itself. As for manuscript writing, ChatGPT generated a coherent abstract for the paper. It was possible to use ChatGPT to do a literature review, but in this example, it provided faulty information. This important aspect needs to be assessed in other fields as well.

Macdonald found that ChatGPT could become a resource for researchers to develop papers and studies faster but with careful assessment of the given answer. Khan et al. found that ChatGPT is useful for checking grammar in written text, and it could be useful as a teaching tool for generating tasks in a medical environment [ 11 ]. In the reference [ 3 ], the authors also came to a similar conclusion. The tool can provide research assistance by providing text summaries, answering questions and generating bibliographies. It can translate, which could be useful for researchers who need to write in a second language and do not have a subscription tool for translation. In clinical management, it could improve documentation, support decision-making and facilitate communication with patients. There are noted deficiencies though: A lack of real understanding was reported, originating from the fixed and closed database. Besides, no data was found after 2021, and it could also generate unoriginal content with incorrect answers at times. This was also emphasized by Gordjin and Have [ 14 ]. Liebrenz showed that ChatGPT can also write an article for The Lancet Digital Health about AI and medical publishing ethics [ 15 ]. They highlighted that monetization of AI could produce a gap between researchers of different wealth. However, the availability of further monetized tools (such as ChatGPT4) was not available at the moment, so critical comparison could not have been performed by the authors.

1.3. Application in Finance and Education

Moving on to a different profession, Dowling and Lucey used the tool to assess their research process in finance research [ 16 ]. A literature review was done on both public (already included) and private (fed to ChatGPT) data. Idea generation, data identification, prep and testing framework proposition were done. The generated studies were stated by a board of referees to be of acceptable quality for a peer-reviewed finance journal. However, data analysis was missing from ChatGPT’s actual capabilities.

As for educational use, Rillig et al. [ 17 ] discussed the application in environmental research and education. Due to the working principle of the algorithm, biases in the training data could produce bias in the output of ChatGPT. The LLM output could easily be confused with an expert’s answer, even though it has no real understanding, so Rillig et al. also highlighted these risks during applications. ChatGPT could be used to accelerate research by outsourcing tasks to ChatGPT, improving workflow. Furthermore, it can also help non-native English speakers to write papers, develop ideas, etc. Nevertheless, ChatGPT could raise issues of cheating in education as well. The papers usually do not propose in-depth solutions for the problems; they only imply the need.

1.4. Application in IT and in Engineering Sciences

From the aspect of IT sciences and electrical engineering, Wang et al. noted [ 7 ] that “Stack Overflow” introduced a temporary ban on code generated by ChatGPT because of a low percentage of being totally correct. The bot gave plausible but incorrect answers during the presented discussions. Surameery and Shakor highlighted bug-fixing capabilities of the tool and suggested ChatGPT as a part of a more comprehensive debugging toolkit, not only as a sole solution used by developers [ 18 ]. Biswas showed mostly the positive aspects of the tool in programming, such as a technical query answering machine for explanations, guides and diagnosing problems [ 19 ]. Vemprala et al. showed a possible application mindset for robotics, where ChatGPT partially substitutes the engineer in the loop and where eventually a user can use the LLM as a tool to connect to further robots solving tasks [ 20 ]. The literature is very limited for electronics, software- and electrical engineering applications, which needs further investigations via various use cases and their documentations.

To see how efficiently ChatGPT can help in electronics research and development, we adapted the methodologies and approaches presented in the previous papers and investigated the applicability of ChatGPT for electrical engineering, design and development, where our focus was positioned to a widely studied topic: smart home applications with sensors. The question was: Can ChatGPT be a tool or a useful companion for an electrical engineer developer? The paper presents the experimental methodology, the query running with the bot with the supplemented data and the discussion of the results. The main novelty of our work is to show the applicability of the tool in an area that was not discussed before, and our findings might help discussions and arguments regarding the application from the professional industrial level to education and academic sciences.

2. Experimental Information

We wanted to investigate if ChatGPT could help us with the development steps in a smart home project. Our case study is a relevant example since smart home and IoT related developments are very popular lately (IEEE Xplore lists 13.000 articles for the term “smart home” since 2013). So, the bot could work on a well-established topic, which is also interesting from the aspect of actual hardware and software development. Furthermore, the complexity of the topic is deep enough to test the ability of ChatGPT to give answers to various layers of the problem, such as:

  • controllers as a central unit, sensor component types, parameters, comparison of different types;
  • logic connection between components;
  • how can the sensors be programmed with an embedded mindset;
  • actual code generation;
  • and expanding the view with literature references.

We focused mainly on the electrical engineering aspects of the project, excluding data acquisition, repository, database and further IoT related questions. The workflow was the following:

  • ChatGPT was introduced to the problem by the first query: whether to use a Raspberry Pi or an Arduino board as a sensor controller for a smart home. This question is general for a start and requires the chatbot to look up information and reason for either solution.
  • Afterwards, more specific questions have been asked. When it was already decided which controller should be used, the bot was asked to provide a list of sensors that would be recommended for the board.
  • Then the bot was asked to provide specific sensor models for each type of sensor, to list the sensors, to compare two alternatives or to provide the specifications and prices for the sensors.
  • The bot was also asked about the workflow of connecting specific sensors to the controller and supporting software development.
  • Finally, it was prompted to provide a literature review and references with DOI (Digital Object Identifier) numbers included. This step was done at the end of our queries due to the nature of the bot; the bot can dwell deeper into the topic once a human user establishes it and carries forward the discussion.

Our methodology is also presented in the following Figure 1 .

An external file that holds a picture, illustration, etc.
Object name is sensors-23-04879-g001.jpg

Workflow with ChatGPT in the use case of building a smart home device with sensors.

ChatGPT (based on GPT-3.5, OpenAI, San Francisco, USA) was used for our experiments. The official cutoff date of the training data was in 2021 [ 21 ], which limits the current possibility of the tool. However, established information was well available before this date regarding our topic and queries. The queries were appointed to the tool between 13 and 19 March 2023, so the current work represents the state of the bot in that time. While the next version of ChatGPT Plus (GTP-4) is supposed to have better performance, it just became available during our investigations and was locked behind a paywall. Furthermore, controversies were noted, and concerns were raised regarding the new version of the tool (with an eventual ban of it in Italy) [ 22 ]. So, we aimed to stick to the more well-known version of the tool. Also, during the manuscript composition, Bard (Google) was not available in the country, meaning that general accessibility is still an issue. Bing (Microsoft) bots became available with interfacing to the internet [ 23 ], and it was freely available, so we decided to add a short (but not comprehensive) comparison between 3.5 and Bing.

While it is difficult to present quantitative analysis on the problem, we try to discuss the results qualitatively, aiming for technological precision and quantitative assessment only where possible (e.g., in the case of written codes, data and specific information). The full data set (including codes, queries and responses) is available in Supplementary Materials , which might be necessary for full assessment by the reader, as the full query text inclusion would boast the paper significantly. Supplementary Materials availability is highlighted in the papers “ Supplementary Materials ” and “Data Availability Statement” section.

3.1. General Questions Regarding Building a Smart Home Application, Specification of a Controller

In the first step, we asked ChatGPT which controller we should use for developing a smart home application.

ChatGPT : “Sure! There are several options for micro-controllers that you can choose from for your smart home project. Here are 6 examples from different manufacturers:

Arduino Uno: This is a popular micro-controller board that is easy to use and has a large community. It has 14 digital input/output pins, 6 analog inputs and a 16 MHz quartz crystal.

Raspberry Pi: This is a small, affordable computer that can run a full operating system. It has a variety of input/output options, including GPIO pins, USB ports and Ethernet.

ESP32: This is a powerful micro-controller with built-in Wi-Fi and Bluetooth connectivity. It has 34 GPIO pins, 18 analog inputs and a dual-core processor.

STM32F4 Discovery: This is a development board based on the STM32F407VGT6 micro-controller. It has 168 MHz Cortex-M4 CPU, 1 MB Flash memory, and 192 KB RAM.

PIC18F4520: This is a popular micro-controller from Microchip Technology. It has 32 KB Flash memory, 1.5 KB RAM and 36 I/O pins.

NXP LPC1768: This is a micro-controller based on the ARM Cortex-M3 architecture. It has 512 KB Flash memory, 64 KB RAM and a variety of input/output options.

These are just a few examples, and there are many other micro-controllers available on the market. You will need to choose the one that best fits your project’s requirements in terms of processing power, memory and input/output options.”

At this point, we asked ChatGPT to give arguments on choosing a device.

ChatGPT : “Sure! Here are some arguments for choosing each of the micro-controllers I listed:

Arduino Uno: The Arduino Uno is a popular choice for beginners and hobbyists because it is easy to use and has a large community. There are many tutorials and examples available online, making it easy to get started. It has a simple programming environment and is compatible with a wide variety of sensors and shields.

Raspberry Pi: The Raspberry Pi is a powerful micro-controller that can run a full operating system, making it suitable for more complex projects. It has a variety of input/output options, including GPIO pins, USB ports and Ethernet, making it versatile and flexible. It can also run software like Python, making it easy to integrate with other tools and services.

ESP32: The ESP32 is a powerful micro-controller with built-in Wi-Fi and Bluetooth connectivity. This makes it ideal for projects that require wireless communication, such as remote control or monitoring. It has a large number of input/output pins and a dual-core processor, making it suitable for projects that require a lot of processing power.

STM32F4 Discovery: The STM32F4 Discovery is a development board that is based on the STM32F407VGT6 micro-controller. It has a high processing speed and a large amount of memory, making it suitable for complex projects. It also has a wide range of input/output options, including USB, Ethernet and audio.

PIC18F4520: The PIC18F4520 is a popular micro-controller from Microchip Technology. It has a relatively low cost and a large number of input/output pins, making it ideal for projects that require a lot of sensors or actuators. It also has a large community and many examples available online, making it easy to get started.

NXP LPC1768: The NXP LPC1768 is a micro-controller based on the ARM Cortex-M3 architecture. It has a large amount of memory and a variety of input/output options, making it suitable for a wide range of projects. It also has a powerful processor and can run at high speeds, making it ideal for projects that require fast processing.

In summary, each of these micro-controllers has its own strengths and weaknesses, and the best one for your project will depend on your specific requirements. Factors to consider when choosing a micro-controller include processing power, memory, input/output options and community support.”

We can say, that the information is accurate, and the bot was giving logical arguments with precise information on this very general initial case.

To narrow down the focus, we gave Arduino or Raspberry Pi as a choice. Early investigations showed that the answers give a broader point of view at the beginning of a query-set, so a narrowed approach is beneficial for a start. The bot was able to distinguish the two different systems, with reasonable responses on the general overview regarding the two systems capabilities. It gave advantages and disadvantages about the use.

ChatGPT : “Both the Arduino and Raspberry Pi controllers have their advantages and disadvantages when it comes to building a smart home.

Arduino is a microcontroller board that is designed for simple projects that require low power and have limited processing requirements. It is a good choice if you want to control simple sensors such as temperature, humidity and light sensors. Arduino is easy to use and is ideal for beginners who are starting to learn about microcontrollers.

On the other hand, Raspberry Pi is a small computer that is capable of running a full operating system. It is more powerful than Arduino and has a wider range of capabilities. Raspberry Pi is a better choice if you want to integrate complex sensors such as cameras, voice recognition and machine learning. It also has the capability of connecting to the internet, which allows for remote control and monitoring of your smart home devices.

Therefore, the choice of controller depends on the complexity of your project and the sensors you want to integrate. If you are building a simple smart home system, then an Arduino controller would suffice. However, if you want to build a more advanced system that requires complex sensors and internet connectivity, then Raspberry Pi would be a better choice.”

We found that the answer contained some correct information; however, some other information can be argued with. The discussion did not go to the level of more recent advances of the systems. Arduino also offers newer devices (Portenta family [ 24 ], established in 2020) for higher-level tasks with similar capabilities to the Raspberry lineup. This information was either omitted by the timely limit of the background training data for ChatGPT 3.5 or by the fact that ChatGPT is orienting its answers towards the more common knowledge established over a longer period. According to Hernando, the origins of Arduino can be traced back to 2003–2005 [ 25 ], so the answer given by the bot was a more superficial assumption based on a longer-term general knowledge. The final conclusion of ChatGPT was found to be acceptable; most of the information was valid; and it could argue along a wide range of commercially available products without obvious bias towards one or the other.

3.2. Sensors and Specifications

We also asked for specific types of sensors, where the bot suggested temperature, light, motion, humidity, gas, water and smoke sensors and their combinations with short descriptions, which helps accessibility to the topic even at a beginner level. We emphasize again that, for full analysis, the reading of Supplementary Materials is recommended.

While the scope would be too wide for analyzing all types, we focused first on temperature sensors. ChatGPT suggested five “popular” alternatives: DHT11, DHT22, LM35, DS18B20 and BME280. It provided short descriptions for each type in its answer. Each component is valid and widely used by hobbyists and professionals in the field. The query followed with light sensors. ChatGPT gave a mix of the types (e.g., photoresistors, photodiodes, phototransistors) and specific sensor modules (e.g., TSL2561, BH1740) in its five-step answer. To sum up, the bot was mixing types and specific commercial components in the answer, which is inconsistent in its manner.

Altogether, from two questions on “specific light sensors” and “specific light sensor models”, the bot listed six components and three type descriptions (with a repeated description on photoresistors). It was clear that slightly different queries affected the answer inconsistency, but the effect seemed rather minor and seemed to be originated from the randomness of each individual answer by the bot. Again, the bot was mixing types and components.

As for motion sensors, five examples were listed. The listing was presented in a third format, where the declared type was followed with a specific component name as well. (e.g., PIR Motion Sensor (HC-SR501)). However, inconsistencies were found, as for five different types, only three actual sensor components were listed, the rest was again general listing. The bot did not provide a real module for infrared motion sensors and laser motion sensors. The same mixed listing was found in the case of humidity sensors. Albeit the bot could give five actual sensor types this time, only one type was specified on its working principle (capacitive humidity sensor (SY-HS-220). Gas, water level and smoke detectors were also sorted by incoherent listing; however, all types were found to be commercially available.

ChatGPT was also asked to list the sensors by price, the bot was able to find ranges of the price, and it arranged a list based on the prices. The ranges seemed valid according to a manual search in popular web shops on the internet. All information seemed to be coherent with current availability for 10 components, but actual prices varied a bit (at the same or one range lower as the current prices). This might not only be due to the recent increase in prices—some components were inconsistently cheaper or more expensive in the list.

The bot suggested Arduino for the project previously, so the follow-up was coherently provided according to this controller type. In short, ChatGPT could carry forward the information precisely from question to question.

After the more general queries on sensors, we asked to suggest specific data for three different sensors. One query focused on the DHT11. The following specifications were given.

ChatGPT: “Here are the specifications of the DHT11 temperature and humidity sensor:

  • Temperature measurement range: 0 to 50 degrees Celsius with an accuracy of +/−2 degrees Celsius
  • Humidity measurement range: 20% to 90% RH (Relative Humidity) with an accuracy of +/−5%
  • Operating voltage: 3.3 V to 5 V DC
  • Current consumption: 2.5 mA max during conversion, 0.5 mA standby
  • Signal transmission distance: Up to 20 m
  • Sampling rate: 1 Hz (1 reading per second)
  • Response time: less than 5 s
  • Dimensions: 23 mm × 12 mm × 5 mm (L × W × H)
  • Digital output: Data is transmitted using a single-wire serial interface with a timing protocol that includes a start signal, data bits, and a checksum for error checking.

The DHT11 sensor is a low-cost, basic temperature and humidity sensor that is suitable for many simple projects. However, if you require higher accuracy, faster response time, or other features, you may need to consider a more advanced sensor.”

The bot was able to summarize the applicability of the sensor in an efficient manner, but a mistake was found. The response time was not accurate. The datasheet gives 6 s minimum, 10 s typical and 15 s max for this parameter [ 26 ], compared to the bot’s version of “less than 5 s”. It was suggested that DHT11 has different datasheets available on the web, but an alternative, widely available datasheet version was also backing up our finding with precise response time values [ 27 ]. We asked the same question for DS18B20 sensor too Supplementary Materials , where we found that the maximum of the current draw was imprecise for the suggestions (1 mA suggested value vs. 1.5 mA in real life). For two other sensors (TLS2561, MQ-135), the operating voltages, active power consumption and operating temperatures were imprecise (for full details, see Supplementary Materials ), in the meanwhile, staying in similar range as the datasheet values. For TLS2561, ChatGPT gave an “Operating voltage range of: 2.7 V to 3.6V DC”, which is actually 0 V to 3.8 V according to datasheet [ 28 ]. It was found that the “2.7 V to 3.6 V DC” was received from an application note [ 29 ], where the “voltage range” noting is similarly confusing as it is missing the lower boundary of the range. For MQ.135: ChatGPT gave “Operating temperature: −10 to 50 °C”, which is actually max. 45 °C [ 30 ].

Overall, this information can be crucial during design, so while ChatGPT can focus its interest on specific components, many details need validation by human hand. It is suggested that the bot was trained on older data or misleading information was found in the training dataset. Altogether it can be concluded that most of the given data were usable though some responses contained imprecise information in similar, easily confusing or deceiving range as original values found in technical specifications.

3.3. Workflow and Code

To aid embedded coding, ChatGPT was asked how to connect a DS18B20 temperature sensor to the Arduino [ 31 ]. The component was chosen randomly from the previous set of results discussed in Section 3.2 . The answer was the following.

ChatGPT : “To connect a DS18B20 temperature sensor to an Arduino, you will need the following components:

  • DS18B20 temperature sensor
  • 4.7 k ohm resistor
  • Jumper wires

Here are the steps to connect the DS18B20 temperature sensor to the Arduino:

  • Connect the Vcc pin of the DS18B20 sensor to the 5 V pin on the Arduino.
  • Connect the GND pin of the DS18B20 sensor to the GND pin on the Arduino.
  • Connect the data pin (middle pin) of the DS18B20 sensor to a digital input/output (I/O) pin on the Arduino. In this example, we will use digital pin 2.
  • Connect a 4.7 k ohm resistor between the data pin and the 5 V pin on the Arduino.
  • Connect the Arduino to your computer via USB cable.
  • Open the Arduino IDE and create a new sketch.
  • In the sketch, include the OneWire and DallasTemperature libraries. These libraries can be installed from the Arduino IDE Library Manager.
  • Initialize the OneWire and DallasTemperature libraries in the setup() function of the sketch.
  • In the loop() function of the sketch, read the temperature data from the DS18B20 sensor using the DallasTemperature library functions. The code would look something like this:

In this code, we first define the data pin (digital pin 2) as ONE_WIRE_BUS. Then we initialize the OneWire and DallasTemperature libraries in the setup() function. In the loop() function, we first use the requestTemperatures() function to request the temperature data from the DS18B20 sensor. Then we use the getTempCByIndex(0) function to get the temperature in degrees Celsius. Finally, we print the temperature to the serial monitor and add a delay of 1 s before the next reading.

  • 10. Upload the sketch to the Arduino board, and open the serial monitor to see the temperature readings.”

According to the given response, we validated the steps by the datasheet. The component list was valid, as well the connections of the pins and components. Even highlighting the 1-Wire communication was correct. The official datasheet backs up our findings about the response [ 31 ]. The suggested code is also working, but minimal knowledge of the IDE (Integrated Development Environment) is required to assess the steps given by the bot. We also did the query with the DHT11 Supplementary Materials , and we received again, convincing points. The datasheet was suggesting that the pin alignment was correct [ 32 ]. However, there was a DHT library recommendation, which all authors agreed upon, that was not straightforward to understand and include with the work. Overall, it was found that more preliminary knowledge is required to use the second query, and it was found that the depth of the solution might vary from use case to use case.

To the question about the origins of the suggestions, ChatGPT responded with a sound answer:

ChatGPT : “I wrote the code example based on my knowledge and experience with using the OneWire and DallasTemperature libraries to read temperature data from a DS18B20 sensor. However, the code structure and some of the function calls may be similar to code examples found in the official Arduino documentation, online forums or tutorials.”

The generated code can be found in CodeForDS18B20.cpp in the Supplementary Materials dataset of the article.

The question regarding the use of an MQ-4 gas sensor was also asked from the bot. This time the results were considerably less reliable. ChatGPT cited the datasheet of the MQ-4 sensor for a formula [ 33 ], where the datasheet did not contain such information. So, human validation was again required at this point to validate the data provided by the bot. From this aspect, the formula was not included in the official support materials, so we can claim this part unreliable.

In the codes, commenting was also found for better understanding; however, the comments were not consequential in their amount and detail. It is a big advantage however that the tool was aware of commenting.

Altogether, ChatGPT can give partially plausible solutions for simple hardware and software tasks, like interfacing a controller and a sensor, but it also gave false findings, which elevates the need of human control over the results.

3.4. Literature Survey on the Topic

About the literature survey, the following question was given to ChatGPT: “Could you do a literature review on smart home control with Arduino for a research paper?” ChatGPT responded with a general introduction on the topic of Arduino in relation with smart home control and gave three literature references as examples. For example:

ChatGPT : “One study by Varghese et al. (2017) presented an Arduino-based smart home system that used a variety of sensors to monitor the temperature, humidity, gas levels and motion in a house. The system also included automatic control of lights and fans based on occupancy and environmental conditions. The authors reported that the system was able to improve energy efficiency and reduce utility costs.”

When we asked about the citations, ChatGPT provided a plausible response at first glance. The following citation was presented for the article above.

ChatGPT : “Varghese, J., Jayachandran, S., & Jayachandran, S. (2017). An Arduino based smart home system. International Journal of Pure and Applied Mathematics, 114(10), 505–514.”

Looking for the given citation in widely used search-engines (Google Scholar, Academia.edu) and in the article list of the given journal, it was found that this paper did not ever exist. The names of the authors were found in similar research topics, but it was an initial red flag that the second and third authors were the same (Jayachandran, S.). Also, the title is a fabricated one, as well as the rest of the bibliographic information. In the next step, ChatGPT was asked to give references to scientific literature (recommend articles) regarding Arduino as a sensor controller (also including the DOI numbers of the papers). The response was a list of five references, with similarly false information. During the investigations, an interesting aspect was revealed about the following suggested article:

ChatGPT : “J. R. de Sousa and M. F. de Sousa, “Data acquisition and control system based on Arduino platform for low-cost greenhouse monitoring,” Measurement, vol. 168, p. 108273, Jan. 2021. DOI: 10.1016/j.measurement.2020.108273”

It turns out, there is no such article available on the web. But the DOI seemed to be in connection with the given journal, which would also fool the unsuspecting eye, believing that the reference is correct. The DOI is an existing one, but it does not refer to the given article. The paper of Ziolowsky et al. has nothing to do with the discussed applied sensors topic [ 34 ].

For the question on the actual help for fitting sensors to Arduino, the bot provided five references with short descriptions, and four out of them were actually accessible. One of these examples was not a live link, but the browser forwarded the request to a proper one. The given link “ https://www.arduino.cc/en/Tutorial/HomePage (accessed on 29 March 2023)” seemed to forward us to the actual (current) document repository of Arduino [ 35 ]. This case needed deeper assessment. The link was fed to the reliable Archive.Org “Wayback Machine”, and it turned out that the archived (March 2023) link is redirected to the actual repository as shown above with the following HTTP response [ 36 ]:

“ https://www.arduino.cc/en/Tutorial/HomePage |

16:41:50 March 26, 2023

Got an HTTP 302 response at crawl time | Redirecting to…

https://docs.arduino.cc/tutorials/ ”

According to Archive.org [ 36 ], it seemed that 04:07:56, 4 January 2022 was the last timestamp, when the originally suggested link was accessible. This is in line with the communicated capability of ChatGPT 3.5, that is, relying on a 2021 learning database—the bot could not know that the tutorials were removed to an alternative website.

3.5. An Outlook and Comparison with Bing

While the basic assumption to compare Bing to the closed database-trained ChatGPT 3.5 is not a balanced act, we asked the questions on controllers, argument on them, sensor types, specific information on getting started and specific literature. All details are available in Supplementary Materials .

We found that Bing gave us five valid controllers (when asked about 4–6 different types, pointing to a middle solution in quantity). The arguments and information were valid, but the descriptions were a bit shorter. Out of five examples, four were the same with the recommendations of GPT. Also, Arduino was the primary choice here as well.

As for sensor types, Bing gave eight different sensors, where most of them were healthcare based ones. So the focus was much more precise in the case of ChatGPT. For a more specific query on temperature sensors, five different types were listed. DS18B20 was highlighted again as a widely used one. Bing could recommend altogether 17 valid links for tutorials and videos. This aspect is clearly realized by the connection to the web.

While we asked for at least five literature references, we were given five articles. Four articles were available of the five ones, but three articles were written by other authors as it was stated by the bot. Also, all of the DOIs were imprecise. Most links were cited from ResearchGate, which might come from the bias to search in open access materials and libraries. It can be stated that even this tool has problems, so literature survey with the tool is not acceptable from academic precision aspects.

4. Discussion

The summary of our findings is presented in Figure 2 and in the performance metrics Table 1 , based on the work of Lo [ 37 ]. We found that ChatGPT was able to narrow down an initial assumption in developing a smart home project with a proper choice of controllers and sensors. The bot gave superficial but altogether usable recommendations for a controller, choosing from a wide range of items, limited by the its training. It also described a wide range of sensors for the project, with relevant descriptions accessible to even a beginner practitioner in the field.

An external file that holds a picture, illustration, etc.
Object name is sensors-23-04879-g002.jpg

Summarizing our findings regarding main aspects and the acceptability of the given answers by ChatGPT.

Performance evaluation of ChatGPT-3.5 across different subjects.

As for the sensors and specifications, the bot was able to sort out different sensor types for the task, which were fitting to the request, and which are still on the market. The sorting of the sensors was correct from the aspect of the price, but the listings (or groupings) were inconsistent from the aspect of sensor type (by measurement principle) and actual sensor component type. We also found that most of the sensor specifics were correct, but around 10% of the given parameters were imprecise (albeit in range, close to the actual values). The bot was able to follow the discussion via initial assumptions fixed in the queries (such as following up with the Arduino platform). The bot was also giving popular examples and was not focusing on very specific components but widely available ones. If the training dataset is new enough, mistakes in given data could be less, and exact parameters could be more precise; however, various versions of datasheets floating around (e.g., in the case of cheap Far East sensors) might also leave space for confusion.

The workflow of interfacing a sensor module to a controller was covered by the ChatGPT in a partially proper manner. It was able to give appropriate design solutions for physically interfacing the sensor module and also generated a code that provided correct syntax and references to the application of the given sensor module. The request was repeated with another sensor component, where the answers were similarly convincing, but contained minor errors. It must be noted that the task was relatively simple. However, at this level, the bot can help engineers in the learning phase or give an immediate response for a task, which would eventually require a significant amount of working hours.

The capability of the bot for a literature survey is found to be unreliable. It recommends seemingly correct and relevant citations for given technical themes. But all of them are fake, fabricated by relevant keywords, existing authors, journal names and DOIs, which are real but incorrect for the actual examples. It is suggested that in the nature of this LLM, the bot is assembling keyword-based information and cannot handle large, coherent data, like a literature citation. Or simply, the literature is not available to the bot due to commercial nature. In a very recent study, there is a suggestion, that the ChatGPT is more like a creative tool in case of literature analysis than exact documentation assistant [ 38 ]. However, it further raises questions about how the information is handled in other scientific fields; as the aforementioned review material mentions [ 39 ], most papers seem optimistic from the data organizing side capabilities (e.g., in hypothesis validation or the identification of gaps and inconsistencies in current knowledge). The fabricated literature responses had an added value though: It contained coherent listings of thematically correct key-words, which could further aid the literature survey of a human learner or a beginner in the field. It is important to note that for more technical or commercial related information (such as datasheets and tutorials), ChatGPT could provide four accessible links out of five. One link was an older one, but it was automatically redirected to a working page by the host.

It has to be also noted that the flow of the discussion cannot be reinstated or repeated in the same manner, so further information and branching discussions are given to the user distinctively from time to time. Our experiment presented a use case that is not classically reproducible in the traditional manner. This also makes it more difficult to assess the application of the tool from both technical and societal aspects.

Overall, our results are clarifying the technical depth and precision of the literature survey shown in [ 5 , 6 , 11 ], showing that exact literature survey, or “searching” can be unreliable. The partially acceptable results with minor failures were similar to be found in [ 4 ]. As we found, the accessibility and applicability of the tool points to a curious recommendation, which is similar to [ 18 , 19 , 20 ]. The performance analysis was similar to other ChatGPT use cases [ 37 ].

5. Conclusions

In this article, we investigated the applicability of the novel LLM-based ChatGPT for electrical engineering development of a smart home project. ChatGPT followed our queries with a consistent manner, focusing on the controller and narrowing down the question to different sensor types, their interfacing, some exemplary software and related literature.

It was found that the sensor recommendations were valid, but part of the given specification, prices, hardware-software integration and the resulting code were slightly imprecise, which needed human validation. Furthermore, the listings were inconsistent from the sensor type and commercial module type aspects.

It was found that, by requesting actual scientific literature (articles, journal papers), the results were fabricated, fake and unreliable. However, with limited usability, technical documentation was referenced correctly by ChatGPT, and the key words were found to be accurate.

Interestingly, it was found that Bing (connected to the web) made similar performance as the older ChatGPT 3.5 version, where controller choice was acceptable, sensor technical details were containing minor problems, and literature survey was unacceptable.

For the future, similar tasks should be investigated with the continuously evolving versions and the competitors, such as the recently launched Bing or the Bard platform. The work could be continued with the inclusion of PCB (printed circuit board) design and related simulations. Our work does not tackle the ethical side and fair, responsible use, which is a societal aspect. Concerns are high regarding the application of such tools; even leading professionals are asking for a “pause time” from AI developers to catch up with regulations. Also, it is suggested to focus on information technology and engineering R&D aspects during the development of further releases. As the tool is coming from the same broadly interpreted profession, as our field, it would be important for the tool to meet requirements in data precision (especially with literature and technical information), so that it does not include errors to the developers working with it.

The takeaway message is that, with proper regulations and future improvements, chatbots can be an effective device in electronics development projects, as a supplementary tool in the available, traditional toolset.

Supplementary Materials

The following supporting information can be downloaded at: http://real.mtak.hu/id/eprint/163609 (accessed on 5 April 2023).

Funding Statement

This research received no external funding.

Author Contributions

Conceptualization, A.G.; methodology, A.G. and Z.T.; software, Z.T.; validation, B.I. and A.G.; investigation, Z.T.; data curation, Z.T., O.K.; writing—original draft preparation, A.G. and Z.T.; writing—review and editing, A.G. and B.I.; visualization, A.G.; supervision, A.G. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Students sitting on a campus quad during a sunny day.

  • General News
  • Scholarly Communications

How Does ChatGPT Answer Fair Use Questions?

We’ve all heard our “fair” share about ChatGPT —the powerful and controversial artificial intelligence chatbot from OpenAI that answers questions, writes essays, generates code, and chats with humans. In celebration of Fair Use Week 2023 , we wanted to share selected strengths and weaknesses of the large language model’s current (Feb. 9 version) ability to generate human-like responses to questions about fair use.

In Scholarly Communication Services at Washington University Libraries, we’re often faced with a patron wanting a definitive yes or no answer to their fair use questions. The legal doctrine of fair use guides the everyday use of copyrighted material in the university. We replicated this approach in our questions to ChatGPT, and the AI assistant would almost always advise consulting a nebulous legal counsel or “otherwise qualified professional,” with the repetitious conclusion that, “ultimately, fair use is determined on a case-by-case basis and can be difficult to assess.”

Fair use is a limitation on copyright where purposes such as criticism, comment, news reporting, teaching, scholarship, or research are not infringements of copyright. The four fair use factors are as follows:

  • the purpose and character of the use, including whether the use is of a commercial nature or is for nonprofit educational purposes;
  • the nature of the work itself, or the degree to which the work that was used relates to the copyright’s purpose of encouraging creative expression;
  • the amount and substantiality of the work used in relation to the copyrighted work as a whole;
  • the effect of the use upon the potential market for or value of the copyrighted work.

Asking Questions of ChatGPT

The first question asked was, “Is uploading a Jacques Lacan essay to my university learning management system a fair use?” In summary, the response correctly and coherently explains that fair use is a legal doctrine that allows for limited and reasonable use of copyrighted material without obtaining permission from the copyright owner.

ChatGPT acknowledges that supporting classroom instruction (factor one) favors fair use, but if it substitutes the sale for the work (factor four), it weighs against fair use. ChatGPT is correct that a yes/no answer is not always possible in response to a fair use question (even if it were to pretend to be a lawyer).

Because this response indicates that some details weigh for and others weigh against fair use, the questioner provides additional clarifying details about each factor, namely that the essay is not creative, and money is not being made from the use. Details about the nature (factor two) and amount and substantiality (factor three) of the work were addressed next. These include information that there are two paragraphs in a seven-page essay for students to read and then discuss in class. After this, it concludes that the use could favor a fair use finding—noting that, while using a small portion of a work can be seen as favoring fair use, even a small portion of a work can be considered substantial if it represents the heart of the work or the most valuable or significant part of the work.

ChatGPT provided a decent answer on how to determine if a portion is valuable or significant, but additional questions in trying to analyze market effect proved to be more complicated, with answers falling short and unsatisfying. Recognizing the importance of balancing all four factors is reassuring, but ChatGPT does not solicit bibliographic details about the essay or how it’s going to be used. This would be helpful to instill confidence in those attempting to make a fair use analysis. ChatGPT can’t replicate a reference interview, in other words, a conversation between a librarian and a library patron.

A second question presented some stumbling blocks: “I want to develop a program that recommends readings based on bibliographies and citations. Can I use indexes, bibliographies, and citation information from various books to populate my program?”

It answers that it’s likely to be considered a fair use based on the purpose, but it reiterates the importance of the remaining factors, considering the amount and substantiality of the portions used and the usage’s impact on the market. Rather than teasing out details for ChatGPT to aid in analysis this time, we attempt to confirm an assumption, “Isn’t bibliographic and citation information considered data and therefore not protected by copyright?”

It correctly answers that this is generally true, but how it is expressed matters. It gives vague examples to demonstrate this idea of how data can be expressed, ultimately concluding that it’s difficult to determine.

The following two questions attempt to gain an understanding of the transformative purpose of the reading recommendation program the questioner would like to develop. ChatGPT spits out definitions and responds both times that it depends, making it clear that ChatGPT “understands” the connection between transformative use and fair use, but it cannot and will not apply these concepts to answer the question.

In the third question, “Can I use an evolutionary biology article in a live workshop demonstration of LaTex?” ChatGPT answers that despite educational purposes, this use would likely require the permission of the copyright owner. It adds that it may be possible to obtain permission from the copyright owner to use the work, either through direct negotiation or through a licensing service. This response notably omits reference to 17 U.S.C. §110(1) , which allows performances and displays of works in the course of face-to-face teaching. This provision is crucial for educational and instructional activities and includes few limitations or conditions (i.e., the material is lawfully acquired, restricted to enrolled students, connected to instruction, and displayed in a classroom or similar place at the institution; this specific exception does not apply to the online delivery of digital media).

It’s interesting that licensing is brought up for articles but not earlier for the Lacan essay, because whether a license applies to a proposed use is almost always a consideration, regardless of the type of the work at issue . Ideally, ChatGPT would make a reference to the potential existence of a license—and the potential implications thereof—for all fair use questions. This interaction also made it clear that ChatGPT cannot clarify or determine whether the author and the rights holder may or may not be the same person—a detail of obvious importance.

ChatGPT manages one question at a time based on the information in the natural language prompt that the questioner provides. Occasionally it will build responses from previous questions—it recognized this third question as another fair use question, for example. ChatGPT might improve its ability to incorporate important details for the questioner to consider during a fair use analysis, as with the “heart of the work” aspect explored in the first question; however, ChatGPT is still behind in its ability to acknowledge licensing market considerations, among other things. ChatGPT cannot “think,” nor can it anticipate all of the possible clues that might matter to a situation undergoing a fair use analysis. ChatGPT can provide helpful information about fair use, but its responses should not be relied upon as a substitute for legal advice or analysis.

What ChatGPT Does and Doesn’t Do Well

Determining what constitutes fair use can be a complex and nuanced process that requires an understanding of legal principles and case law. One strength of ChatGPT’s response to fair use questions is its ability to provide questioners with general information about the principles of fair use. This can be helpful for those who are unfamiliar with the legal doctrine and want to learn more about it. ChatGPT can also provide examples of how fair use has been applied in specific cases. However, there are also limitations to ChatGPT’s ability to provide accurate and reliable information about fair use. The model’s responses may not take into account the specific facts and circumstances of a questioner’s situation, which can be critical in determining whether a particular use is fair. In addition, ChatGPT’s responses may not reflect the most current legal precedents and developments, as the model’s knowledge is limited by its training data and cutoff date.

ChatGPT seems to have found and used some of the best fair use resources, at various points recommending the US Copyright Office, ALA’s fair use evaluator tool, and the Code of Best Practices. Someone new to fair use may be served well by ChatGPT’s ability to pull facts. Those looking for help with analyzing and weighing each of the four factors will be disappointed and may turn to the recommended tools instead. ChatGPT cannot empower questioners to assert fair use as a right, and we must remember that we can lose the rights we don’t use.

Don’t miss Fair Use celebrations happening this week!

Celebrate a Decade of Fair Use Week with Sandra Enimil on Feb 21, 2023, 06:00 pm, Central Time (US and Canada).

Library Futures “Copyright and Fair Use for Libraries” via Zoom on Thursday, February 23, from 12 pm until 1:30 pm ET.

For other events and details, visit Fair Use Week .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • My Account Login
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Open access
  • Published: 13 December 2023

Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training

  • Alfredo Madrid-García   ORCID: orcid.org/0000-0002-1591-0467 1 ,
  • Zulema Rosales-Rosado   ORCID: orcid.org/0000-0002-4244-3139 1   na1 ,
  • Dalifer Freites-Nuñez   ORCID: orcid.org/0000-0002-0966-2778 1   na1 ,
  • Inés Pérez-Sancristóbal   ORCID: orcid.org/0000-0002-2098-4313 1   na1 ,
  • Esperanza Pato-Cour   ORCID: orcid.org/0000-0002-4145-7395 1   na1 ,
  • Chamaida Plasencia-Rodríguez   ORCID: orcid.org/0000-0003-3503-9047 2   na1 ,
  • Luis Cabeza-Osorio 3 , 4   na1 ,
  • Lydia Abasolo-Alcázar 1 ,
  • Leticia León-Mateos   ORCID: orcid.org/0000-0001-7142-0545 1 ,
  • Benjamín Fernández-Gutiérrez   ORCID: orcid.org/0000-0002-6126-8786 1 , 5 &
  • Luis Rodríguez-Rodríguez   ORCID: orcid.org/0000-0002-2869-7861 1  

Scientific Reports volume  13 , Article number:  22129 ( 2023 ) Cite this article

1398 Accesses

1 Altmetric

Metrics details

  • Engineering
  • Rheumatology

The emergence of large language models (LLM) with remarkable performance such as ChatGPT and GPT-4, has led to an unprecedented uptake in the population. One of their most promising and studied applications concerns education due to their ability to understand and generate human-like text, creating a multitude of opportunities for enhancing educational practices and outcomes. The objective of this study is twofold: to assess the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain (MIR), and to evaluate the medical reasoning followed by these LLM to answer those questions. A dataset, RheumaMIR, of 145 rheumatology-related questions, extracted from the exams held between 2010 and 2023, was created for that purpose, used as a prompt for the LLM, and was publicly distributed. Six rheumatologists with clinical and teaching experience evaluated the clinical reasoning of the chatbots using a 5-point Likert scale and their degree of agreement was analyzed. The association between variables that could influence the models’ accuracy (i.e., year of the exam question, disease addressed, type of question and genre) was studied. ChatGPT demonstrated a high level of performance in both accuracy, 66.43%, and clinical reasoning, median (Q1–Q3), 4.5 (2.33–4.67). However, GPT-4 showed better performance with an accuracy score of 93.71% and a median clinical reasoning value of 4.67 (4.5–4.83). These findings suggest that LLM may serve as valuable tools in rheumatology education, aiding in exam preparation and supplementing traditional teaching methods.

Similar content being viewed by others

can chatgpt answer case study questions

ChatGPT-3.5 and Bing Chat in ophthalmology: an updated evaluation of performance, readability, and informative sources

Brendan Ka-Lok Tao, Nicholas Hua, … Jonathan Andrew Micieli

can chatgpt answer case study questions

The model student: GPT-4 performance on graduate biomedical science exams

Daniel Stribling, Yuxing Xia, … Rolf Renne

can chatgpt answer case study questions

Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments

Dana Brin, Vera Sorin, … Eyal Klang

Introduction

The emergence of large language models (LLM) with remarkable performance such as ChatGPT, has led to an unprecedented uptake in the population, being the fastest-growing application in history 1 , and with the potential to transform various domains, including medicine 2 . A clear example of this burgeoning interest is the notable surge in scientific research focusing on ChatGPT’s role within the medical field. From January 1st, 2023 until July 10th, 2023, there have been over 768 publications indexed in PubMed, that contain the string “ChatGPT” and that delve into various aspects of this Generative Pre-trained Transformer (GPT) chatbot. Some of the topics of such publications include the potential impact of ChatGPT on clinical and translational medicine 3 , clinical pharmacology 4 , scientific communication 5 and medical writing 6 , medical evidence summarization 7 , patient question-answering 8 , public health 9 , ethical and legal 10 , health policy making 11 , plastic and colorectal surgery 12 , 13 , doctor-patient communication 9 , or drug-drug interaction and explainability 14 .

In rheumatology, the use of artificial intelligence (AI) and LLM, such chatbots, is gaining relevance 15 . A query conducted on July 10th in PubMed, “Rheumatology AND ChatGPT” produced 12 results. For instance, in Ref. 16 , the authors asked ChatGPT to draft an editorial about AI potentially replacing rheumatologists in editorial writing and discussed the ethical aspects and the future role of rheumatologists. In Ref. 17 , the authors discussed the role of ChatGPT as an author, and the inherent terms associated with the concept of authorship such as accountability, bias, accuracy, and responsibility. Eventually, they considered refusing to recognize ChatGPT as an author, in accordance also to Nature announcement 18 . For their part, the authors in Ref. 19 provided an overview of the potential use of ChatGPT as a rheumatologist, its capabilities, and risks. They concluded that although chatbots will not be able to make clinical decisions for now; they will be able, for example, to design study protocols, streamline information accessibility and contribute to time-saving. A similar correspondence article highlighted five areas in which ChatGPT has the potential to assist in rheumatology care delivery, to note: patient drug education, medical imaging reporting, medical note-keeping for outpatient consultations, medical education and training and clinical audit and research. Authors in Ref. 20 used ChatGPT as an assistant tool to aid in writing a case report of a systemic lupus erythematosus patient. Recently, a study 21 aimed to assess the diagnostic accuracy of GPT-4 in comparison with rheumatologists. For that purpose, this LLM was instructed to name the top five differential diagnoses. Authors concluded that GPT-4 showed a higher accuracy for the top three overall diagnoses compared to the rheumatologist’s assessment.

One of the most promising and studied applications for tools like ChatGPT lies within the realm of education. The ability of these language models to understand and generate human-like text creates a multitude of opportunities for enhancing educational practices and outcomes 22 , 23 , 24 , 25 , 26 . Within this field, one of the most extensively studied applications of ChatGPT is its ability and reasoning in answering medical examination questions. One of the most prominent studies was 27 . This study demonstrated how ChatGPT performed at or near the passing threshold for all three tests of the United States Medical Licensing Exam despite not having undergone any specialized training or reinforcement. However, many similar studies have emerged in recent months.

Given the range of studies that have subjected ChatGPT to different medical exam questions (e.g., multiple-choice, open-ended), the disparity in the results obtained, and the proficiency of ChatGPT in understanding context and generating detailed responses, we hypothesize that these tools could aid in the learning process of medical students, particularly in the study of rheumatology and musculoskeletal diseases.

Therefore, the objective of this study is twofold:

First, we seek to ascertain the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain, Médico Interno Residente (MIR).

Secondly, we aim to evaluate the clinical reasoning followed by ChatGPT/GPT-4 in answering those multiple-choice questions.

ChatGPT and GPT-4 as large language models

ChatGPT, an iteration of the Generative Pre-trained Transformer (GPT) model, is an AI chatbot, which belongs to the category of large language models (LLM), that was developed by OpenAI. These models are language models (i.e., a probabilistic distribution of word sequences) that rely on neural networks with millions of parameters. ChatGPT is trained on diverse and extensive data sources (e.g., books, websites, and so on) and exhibits a groundbreaking ability to generate relevant and context-aware responses, making it a promising tool in the medical education area. The training data cut-off was September 2021. This means that data after this date were not used for training. The model is defined by OpenAI as “Our fastest model, great for most everyday tasks”.

On the other hand, GPT-4, is a large multimodal (i.e., accepts text and image as input) model, also developed by OpenAI, faster, with more parameters and better performance than ChatGPT. More details and a comparison between both LLM used in this work are available in Ref. 28 . The model is defined by OpenAI as “Our most capable model, great for tasks that require creativity and advanced reasoning”. For the purpose of this study, we utilized the version of ChatGPT/GPT-4 that was released on May 3rd 2023, as documented in OpenAI’s release notes 29 . This version was used to answer all the rheumatology questions from the MIR exams in our study.

The MIR is an annual competitive examination required for entry into specialist medical training in Spain. It is comprised of 210 questions from more than 30 competencies (i.e., pediatrics, hematology, ophthalmology), see the Supplementary Material Excel File “ MIR Competencies ”, and follows a multiple-choice format (i.e., since the 2015–2016 academic year, the number of choices decreased from five to four), with only one correct answer. Each question on the exam typically presents a clinical case or a factual query, and the exam also includes image-based questions. The number of questions per competency varies per year, and both the exam and the answers are officially published by the Spanish Ministry of Health 30 . The questions from this examination serve as an invaluable resource for this study, as they represent standardized, expertly crafted questions designed to evaluate a comprehensive understanding of medical subjects, and, therefore, are leveraged to evaluate the accuracy and clinical reasoning of ChatGPT and GPT-4 when exposing them to the rheumatology discipline.

Inclusion criteria

Rheumatology-related questions from MIR exams published from 2009–2010 to 2022–2023 were included in this study. Questions assigned to other specialities, (e.g., pediatrics, orthopedics), that intersected with the field of rheumatology were also included.

On the other hand, questions containing images were excluded from the analysis. This decision was taken because of the current limitations of ChatGPT, as it is primarily a text-based model and does not possess the capability to process or interpret image data. GPT-4 allows images as input data, but this feature was not publicly available at the time of this research. Hence, any questions in the MIR exams that were dependent on visual information, such as graphs, pictures, or clinical image data (e.g., X-ray), were not included. Finally, questions invalidated by the Spanish Ministry of Health were also excluded from the analysis.

Methodology

Questions from the MIR exams were used as prompts. For each question provided to ChatGPT/GPT-4, the sentence “Justify your answer” was added at the end. The responses generated by ChatGPT and GPT-4 to the rheumatology questions were evaluated by six independent rheumatologists. Three of them, ZRR, LCO, and CPR, in addition to being practising clinicians, are MIR training professors. The length of the text generated by the models was not artificially limited.

The medical experts evaluated the clinical reasoning of the chatbots followed in each of the responses. Their evaluation was based on a 1–5 scale, where a score of 5 indicates that the reasoning was entirely correct and flawless, while a score of 1 signifies that the reasoning was inconsistent or contained significant errors. Intermediate scores were used to denote minor errors in reasoning and the severity of these errors was reflected in the score; see Table 1 . The final clinical reasoning score was assigned following a majority vote approach. In case of a tie, the worst score was chosen. The evaluators were also asked to justify the score given to ChatGPT/GPT-4 clinical reasoning when necessary. After that, each question was categorized based on the type of disease being asked about, and classified into factual or clinical case questions.

For this study, we solely relied on the initial responses generated by ChatGPT/GPT-4, without employing the “regenerate response” function. The questions were prompted in Spanish, exactly as they were extracted from the exam. Nonetheless, an English translation of both the question and the clinical reasoning by the LLM was obtained with DeepL . In cases where the answers provided by the models were not singular, a new prompt was used to ask for a single and unique response with the following instruction: “If you had to choose one of the answers, the most relevant one, which would it be?”.

The clinical reasoning of the models was evaluated by the medical experts in Spanish.

A questionnaire was provided to each evaluator to assess the overall performance of the language models and their suitability for use in education and clinical practice. This questionnaire comprises seven free-text questions and can be found in the Supplementary Material “ Questionnaire ”.

Finally, the exam questions and chatbot answers lacked identifiable patient information.

Two main variables were considered for evaluation:

Accuracy, defined as the match between the official MIR question response and the chosen option by ChatGPT/GPT-4.

Score assigned to the clinical reasoning of ChatGPT/GPT-4 by the six evaluators.

Covariables to be considered in this study included: year of the exam question, type of question, patient’s gender, pathology that the question addresses, and chatbot model used.

Statistical analysis

Dichotomous and categorical variables were summarized using proportions. Continuous variables were summarized using the median and the first and third quartiles (Q1–Q3). The distribution of correct answers (i.e., accuracy) among the different covariates was analyzed using chi-square or Fisher’s test, depending on the number of events. Differences between LLM, in terms of accuracy, were evaluated using McNemar’s test.

The degree of agreement in the score assigned to the clinical reasoning by the evaluators was analyzed using Krippendorff’s alpha coefficient, Kendall’s coefficient with and without tie correction coefficients 31 and Gwet’s AC2 coefficient 32 . The Kappa-Fleiss coefficient was not used as raters are considered unique (i.e., all evaluators justify the reasoning of all questions). The final clinical reasoning score given to each question was determined by a majority vote among evaluators. In the event of a tie, the worst score was chosen.

Differences between LLM, in terms of clinical reasoning, were evaluated using Wilcoxon signed-rank test. The effect of covariates on the clinical reasoning score was studied using ordinal logistic regression.

R version 4.3.1 was used to perform the statistical analysis.

Ethics board approval

As suggested in Ref. 10 , the Hospital Clínico San Carlos (HCSC) Ethics Review Board evaluated this project, 23/370-E, and stated that this committee was not competent to evaluate studies of this type, since they do not encompass human subjects, or the use of biological samples, or personal data.

Terms of use of the services used in this work

The use of language models in this study complies with the established terms of service of OpenAI ( https://openai.com/policies/terms-of-use ), Google BARD ( https://support.google.com/bard/answer/13594961?hl=en ) and Antrophic ( https://console.anthropic.com/legal/terms ).

A previous draft of this work was published as a preprint and can be found at: Madrid-García, A. et al . Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training. Preprint at medRxiv, 2023-07 (2023).

The questions evaluated by ChatGPT/GPT-4, the answer by both systems, as well as the official answer, the medical experts’ evaluation of the clinical reasoning, the year, genre, type of question, whether the question was invalidated or not, the type of disease being asked about, the English translation of the questions and the clinical reasoning are shown in the RheumaMIR dataset accessible through Zenodo 33 .

Description

After applying the inclusion criteria, 143 questions from 14 MIR exams remained (i.e., academic years 2009–2010 to 2022–2023). Table 2 shows the dataset characteristics. The median number of questions (Q1-Q3) per year is 11 (9.25–12) and the most prevalent disease being asked about is vasculitis. Most of the questions, 65.73%, were clinical cases, and the sex of the clinical case subjects was evenly distributed.

Out of 143 questions, GPT-4 correctly answered 134 (93.71%), demonstrating a high level of accuracy. On the other hand, ChatGPT accurately answered 95 questions (66.43%), indicating a somewhat lower level of performance in comparison to GPT-4 (i.e., McNemar’s test p-value = 1.17 × 10 −09 ).

ChatGPT did not correctly answer any of the questions that GPT-4 failed to answer correctly. Moreover, out of the nine questions that GPT-4 got wrong, seven of them also matched the answer given by ChatGPT. Table 3 shows the number and percentage of errors per model and covariate. Eventually, none of the covariates were associated with the number of errors, neither in GPT-4 nor in ChatGPT.

Clinical reasoning

The Krippendorff’s alpha coefficient, Kendall’s coefficients with and without tie correction and Gwet’s AC2 coefficient, for the clinical reasoning of GPT-4 considering the six evaluators, were 0.225, 0.452, 0.269 and 0.924, and for ChatGPT, 0.624, 0.783, 0.636 and 0.759. A more detailed analysis of the main differences between evaluators’ scores and an error analysis discussing the questions failed by GPT-4 can be found in the Supplementary Material File ‘ Evaluator agreement ’ and ‘ Error analysis ’ sections and Supplementary Material Excel File Agreement Coefficients.

The median (Q1–Q3) value of the average score for the clinical reasoning of GPT-4 was 4.67 (4.5–4.83), while for ChatGPT was 4.5 (2.33–4.67). There exist statistically significant differences in the clinical reasoning score, after applying the majority vote, of both LLM (i.e., Wilcoxon signed rank test p-value = 4.47 × 10 −09 ).

Regarding the covariates, there were no statistically significant differences in the clinical reasoning score for GPT-4/ChatGPT after applying ordinal logistic regression models. Figure  1 shows the proportion of scores given by the evaluators, grouped by score. Except for one evaluator, Evaluator 6, the most repeated score given was 5 for both models, with a small percentage of low scores (i.e., 1, 2, 3). The comparison of the scores given by the evaluators to the clinical reasoning of ChatGPT and GPT-4, grouped by evaluation, is shown in Supplementary Fig.  1 . For both figures, the majority vote statistics are shown.

figure 1

Distribution of the scores given by the evaluators.

On its behalf, Fig.  2 shows the proportion between the clinical reasoning score and the disease addressed in the questions. Supplementary Figs.  2 – 4 , show the proportion between the clinical reasoning score and the year, the type of question, and the genre. There does not appear to be a clear trend between the reasoning score and the covariates shown in these plots. Finally, the completed questionnaires can be found in the Supplementary Material ‘ Questionnaire ’. The evaluators concur on:

The potential usefulness of this tool, particularly in creating educational content, albeit under expert supervision.

The language used in the responses may lack technical precision and could be suitable for students, but not in other scenarios (i.e., official medical documentation).

ChatGPT/GPT-4 models are unaware of the limitations and scope of their knowledge, sometimes justifying facts in a tortuous manner and potentially misleading the reader.

figure 2

Clinical reasoning score according to the disease addressed in the question after taking the majority vote. In case of a tie, the worst score was chosen.

In this study, we have evaluated the accuracy and clinical reasoning of two LLM in answering rheumatology questions from Spanish official medical exams. To our knowledge, this is the first study to evaluate the usefulness of LLM applied to the training of medical students with a special focus on rheumatology.

The ability of GPT-4 to answer questions with high accuracy and sound clinical reasoning is remarkable. This could make such models valuable learning tools for medical students. However, ChatGPT/GPT-4 LLM are only the first models that have reached the public in the rapidly expanding field of LLM chatbots. At present, a myriad of additional models are under development. Some of these nascent models are not only pre-trained in biomedical texts 34 , 35 , but are also specifically designed for a broad range of tasks (e.g., text summarization, question-answering and so on).

Studies with a similar objective to this one have been conducted. For example, a Spanish study 36 , evaluated ChatGPT’s ability to answer questions from the 2022 MIR exam. In this cross-sectional and descriptive analysis, 210 questions from the exam were entered into the model. ChatGPT correctly answered 51.4% of the questions. This resulted in a 7688 position, slightly below the median of the population tested but above the passing score.

In another research 37 , the proficiency of ChatGPT in answering higher-order thinking questions related to medical biochemistry, including 11 competencies such as basic biochemistry, enzymes, chemistry and metabolism of carbohydrates, lipids and proteins, oncogenesis, and immunity, was studied. Two-hundred questions were randomly chosen from an institution’s question bank and classified according to the Competency-Based Medical Education. The answers were evaluated by two expert biochemistry academicians on a scale of zero to five. ChatGPT obtained a median score of 4 out of 5, with oncogenesis and immunity competition having the lowest score and basic biochemistry the competition with the highest.

Research of a similar nature was conducted in Ref. 38 . In this study, the authors appraised the capability of ChatGPT in answering first- and second-order questions on microbiology (e.g., general microbiology and immunity, musculoskeletal system, skin and soft tissue infections, respiratory tract infections and so on) from the Competency Based Medical Education curriculum. A total of 96 essay questions were reviewed for content validity by an expert microbiologist. Subsequently, ChatGPT responses were evaluated on a scale of 1 to 5, with five being the highest score, by three microbiologists. A median score of 4.04 was achieved.

On the other hand, ChatGPT was tested on the Plastic Surgery In-Service examinations from 2018 to 2022 and its performance was compared to the national average performance of plastic surgery residents 39 . Out of 1129 questions, ChatGPT answered 630 (55.8%) correctly. When compared with the performance of plastic surgery residents in 2022, ChatGPT ranked in the 49th percentile for first-year residents, but its performance fell significantly among residents in higher years of training, dropping to the 0th percentile for 5th and 6th-year residents.

Another study was conducted by researchers in Ref. 40 , who aimed to assess whether ChatGPT could score equivalently to human candidates in a virtual objective structured clinical examination in obstetrics and gynecology. Seven structured examination questions were selected, and the responses of ChatGPT were compared to the responses of two human candidates and evaluated by fourteen qualified examiners. ChatGPT received an average score of 77.2%, while the average historical human score was 73.7%.

Moreover, the authors in Ref. 41 instructed ChatGPT to deliver concise answers to the 24-item diabetes knowledge questionnaire, consisting of a clear “Yes” or “No” response, followed by a concise rationale comprising two sentences for each question. The authors found that ChatGPT successfully answered all the questions.

In Ref. 42 , the researchers were interested in evaluating the performance of ChatGPT on open-ended clinical reasoning questions. Therefore, fourteen multi-part cases were selected from clinical reasoning exams administered to first and second-year medical students and provided to ChatGPT. Each case was comprised of 2–7 open-ended questions and was shown to ChatGPT twice. ChatGPT achieved or surpassed the pre-established passing score of 70% in 43% of the runs (12 out of 28), registering an average score of 69%.

Some studies showed remarkable performance, for instance, a research study evaluated the performance of ChatGPT in medical physiology university examination of phase I MBBS 43 . In this investigation, ChatGPT correctly answered 17 out of 20 multiple-choice questions, while providing a comprehensive explanation for each one. On their side, researchers in Ref. 44 proposed a four-grading system to classify the answers of ChatGPT, to note, comprehensive, correct but inadequate, mixed with correct and incorrect/outdated data, and completely incorrect. ChatGPT showed a 79% and a 74% of accuracy when answering questions related to cirrhosis and hepatocellular carcinoma. However, only the 47% and 41% of the answers were classified as comprehensive.

Conversely, in another research 45 in which ChatGPT was exposed to the family medicine course’s multiple-choice exam of Antwerp University, only 2/125 students performed worse than ChatGPT. Since the questions were prompted in Dutch language, the potential correlation between ChatGPT’s low performance and the proportion of Dutch texts used in its training could be a factor worth considering for this discordant result.

Another study, Ref. 46 , evaluated ChatGPT’s performance on standardized admission tests in the United Kingdom, including the BioMedical Admissions Test (BMAT), Test of Mathematics for University Admission (TMUA), Law National Aptitude Test (LNAT), and Thinking Skills Assessment (TSA). A dataset of 509 multiple-choice questions from these exams, ranging from 2019 to 2022 was used. The results varied among specialities. For BMAT, the percentage of correct answers varied from 5 to 66%, for TMUA varied from 11 to 22%, for LNAT from 36 to 53%; and for TSA from 42 to 60%. The authors concluded that while ChatGPT demonstrated potential as a supplemental tool for areas assessing aptitude, problem-solving, critical thinking, and reading comprehension, it showed limitations in scientific and mathematical knowledge and applications.

The results shown by most of these studies are in line with our results, the average score of ChatGPT is between 4 and 5 (on a scale of five elements) when answering medical-related questions. However, in these studies, GPT-4 performance was not evaluated. Based on our results, we can postulate that there would be an increase in accuracy in comparison to those obtained by ChatGPT. In addition, to solve some of the limitations identified by our evaluators, such as the employment of a language that can lack technical precision by the models, LLM could perform better if trained or fine-tuned with biomedical texts.

A large part of the concerns and doubts that arise from using these models are due to regulatory and ethical issues. Some of the ethical dilemmas have been highlighted in Ref. 47 . For instance, the authors pointed out that LLM reflect any false, outdated, and biased data from which the model was trained, and that they could not reflect the latest guidelines. Some authors have pointed out the risks of perpetuating biases if the model has been trained on biased data 48 , 49 Other authors go further and declare that these types of models should not be used in clinical practice 50 . The motivation behind this statement lies in the presence of biases such as clinician bias, which may exacerbate racial-ethnic disparities, or “hallucinations” meaning that ChatGPT produces high levels of confidence in its output even when insufficient or masked information in the prompt. According to the authors, this phenomenon could lead users to place unwavering trust in the output of chatbots, even if it contains unreliable information. This was also pointed out by our evaluators, as shown in the Supplementary Material ‘ Questionnaire ’. The firmness with which these models justify erroneous reasoning may limit their potential usefulness. Eventually, the authors also claimed that these models are susceptible to “Falsehood Mimicry”, that is, the model will attempt to generate an output that aligns with the user’s assumption rather than clarifying questions. Falsehood mimicry and hallucinations may limit the potential use of these models as diagnostic decision support systems (DDSS). For instance, a clinical trial 51 compared the diagnostic accuracy of medical students regarding the typical rheumatic diseases, with and without the use of a DDSS and concluded that no significant advantage was observed from the use of the DDSS. Moreover, researchers reported that students accepted false DDSS diagnostic suggestions in a substantial number of situations. This phenomenon could be exacerbated when using LLM, and therefore should be study with caution.

In our study, due to the nature of the questions, we were unable to assess racial or ethnic disparities. However, we did not find any gender bias when considering the clinical case questions. Finally, a relevant study that looks in depth at the biases that can arise when using LLMs can be found in Ref. 52 .

Regarding regulation, the arrival of these models has led to greater efforts being made to regulate the use of AI. The EU AI Act 53 is a good example of this. According to our results, in these early stages of LLM, the corpus used for training, as well as the content generated by them, should be carefully analyzed.

During the writing of this manuscript, new articles have emerged. For instance, in Ref. 54 authors studied the performance of ChatGPT when introducing the most searched keywords related to seven rheumatic and musculoskeletal diseases. The content of each answer was evaluated in terms of usefulness for patients with the ChatGPT in a scale from 1 to 7 by two raters.

Finally, further analysis is needed to explore these observations and understand their implications for the development and use of AI in medical education and practice.

Limitations

Two chatbots were primarily used in this study, ChatGPT and GPT-4, both owned by OpenAI. However, other LLM such as BARD or Med-PaLM2 by Google, Claude 2 by Anthropic or LLaMA and LIMA by Meta are in development. Some of them are publicly available. To provide a better overview of other LLM, the accuracy of BARD (60.84%) and Claude 2 (79.72%) was calculated and compared against ChatGPT/GPT-4 in the Supplementary Material File Section ‘ LLM comparison ’ and Supplementary Fig.  5 .

The Krippendorff’s alpha coefficient and Kendall’s coefficient of agreement oscillates between 0.225 and 0.452 for the clinical reasoning of GPT-4, although the most repeated score of five out of six evaluators is 5 (the percentage of four and five scores oscillates between 87.41% and 93.70%), see Fig.  1 . This phenomenon is known as “Kappa paradoxes” (i.e., ‘high agreement, but low reliability’) 55 , 56 , and tends to appear in skewed distributions such as the one presented in this study. More details can be found in Ref. 31 . In this study, since the ChatGPT clinical reasoning score distribution is less skewed, the reliability coefficient values are higher, between 0.624 and 0.783, than the ones obtained with GPT-4. However, when considering the Gwet’s AC2 coefficient, the trend is reversed, 0.924 vs. 0.759, with higher inter-rater agreement in GPT-4 compared to ChatGPT. These large differences between interrater reliability indices have been observed in simulation studies with skewed distributions 57 .

To ensure reproducibility and facilitate the comparison of results, each question could have been submitted twice to ChatGPT/GPT-4, a strategy supported by previous research endeavours 44 , and a default feature of BARD. However, in the tests we conducted, the responses were consistent across iterations and this would have doubled the workload of evaluators, so we chose to include more questions from a single run, rather than fewer questions run multiple times. In addition, according to Ref. 44 , the 90.48% of “regenerated questions” produced two similar responses with similar grading.

The format of each question could have been transformed from multiple choice to open-ended. With this approach, it could have been possible to delve deeper into ChatGPT’s clinical reasoning. As explained in the previous point, this would have doubled the workload. Additionally, there are no open questions in the MIR exams.

When conducting such studies, it is crucial to consider the evolution of knowledge over time. Evaluating older questions with models trained on recent data may reveal disparities compared to previously accepted and conventional beliefs. Therefore, accounting for this temporal aspect is essential to ensure the accuracy and relevance of the study findings. Moreover, although not extensively explored in this research, one of the key concepts when using LLM chatbots is what is known as the prompt or input text that is entered into the model. Depending on how well-defined the prompt is, the results can vary significantly. In this study, we tried to adhere closely to the official question while minimizing any unnecessary additional text.

Another identified limitation of the study is the absence of medical students evaluation for the models’ clinical reasoning. This would have allowed us to determine whether students can recognize the main issues discussed above (e.g., bias, falsehood mimicry, hallucinations), and analyze to what extent these limitations may affect their usefulness.

One of the main criticisms of the evaluators in assessing the LLM response was the use of non-technical language. This could have been remedied, in part, by using prompt engineering, this is, by modifying the initial input of the model and asking for a more technical response.

We have explored the performance of GPT-4/ChatGPT in Spanish questions. However, different authors have suggested that such performance is language-dependent 58 . Hence, special caution should be taken when extrapolating the results to other languages.

The accuracy of ChatGPT and GPT-4 in answering the rheumatology questions of the Spanish access exam to specialized medical training is high, as well as the clinical reasoning. Nevertheless, discerning the veracity of such reasoning can pose a challenge for students in situations where the LLM experiences failures, given the comprehensive and seemingly accurate elaboration present in erroneous responses. Hence, extreme caution should be exercised when using these models as teaching aids and even greater diligence should be taken when using them in clinical practice. However, these kinds of models hold the potential to serve as a valuable asset in the development of pedagogical materials, subject to rigorous expert evaluation.

Data availability

The data that support the findings of this study are openly available in Zenodo at: Alfredo Madrid García, Zulema Rosales Rosado, Dalifer Freites Núñez, Inés Pérez San Cristobal, Esperanza Pato Cour, Chamaida Plasencia Rodríguez, & Luis Cabeza Osorio. (2023). RheumaMIR (3.0.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.10204293 .

Dennean, K., Gantori, S., Limas, D. K., Pu, A. & Gilligan, R. Let’s Chat About ChatGPT . https://www.ubs.com/global/en/wealth-management/our-approach/marketnews/article.1585717.html (2023).

Biswas, S. ChatGPT and the future of medical writing. Radiology 307 , 3312 (2023).

Article   Google Scholar  

Xue, V. W., Lei, P. & Cho, W. C. The potential impact of ChatGPT in clinical and translational medicine. Clin. Transl. Med. 13 , 1216 (2023).

Krumborg, J. R. et al. ChatGPT: First glance from a perspective of clinical pharmacology. Basic Clin. Pharmacol. Toxicol. 133 , 3–5 (2023).

Article   CAS   PubMed   Google Scholar  

Huang, J. & Tan, M. The role of ChatGPT in scientific communication: Writing better scientific review articles. Am. J. Cancer Res. 13 , 1148–1154 (2023).

PubMed   PubMed Central   Google Scholar  

Biswas, S. Passing is great: Can ChatGPT conduct USMLE exams? Ann. Biomed. Eng. 51 , 1885–1886 (2023).

Article   PubMed   Google Scholar  

Tang, L. et al. Evaluating large language models on medical evidence summarization. NPJ Digit. Med. 6 , 158 (2023).

Article   PubMed   PubMed Central   Google Scholar  

Lee, T.-C. et al. ChatGPT answers common patient questions about colonoscopy. Gastroenterology 165 , 509–511 (2023).

He, Y. et al. Can ChatGPT/GPT-4 assist surgeons in confronting patients with Mpox and handling future epidemics? Int. J. Surg. 109 , 2544–2548 (2023).

da Silva, J. A. T. Is institutional review board approval required for studies involving ChatGPT? Am. J. Obstet. Gynecol. MFM 5 , 101005 (2023).

Sifat, R. I. ChatGPT and the future of health policy analysis: Potential and pitfalls of using ChatGPT in policymaking. Ann. Biomed. Eng. 51 , 1357–1359 (2023).

Kang, Y., Xia, Z. & Zhu, L. When ChatGPT meets plastic surgeons. Aesthetic Plast. Surg. 47 , 2190–2193 (2023).

Li, W., Zhang, Y. & Chen, F. ChatGPT in colorectal surgery: A promising tool or a passing fad? Ann. Biomed. Eng. 51 , 1892–1897 (2023).

Juhi, A. et al. The capability of ChatGPT in predicting and explaining common drug–drug interactions. Cureus. https://doi.org/10.7759/cureus.36272 (2023).

Madrid-García, A. et al. Understanding the role and adoption of artificial intelligence techniques in rheumatology research: An in-depth review of the literature. Semin. Arthritis Rheum. 61 , 152213 (2023).

Verhoeven, F., Wendling, D. & Prati, C. ChatGPT: When artificial intelligence replaces the rheumatologist in medical writing. Ann. Rheum. Dis. 82 , 1015–1017 (2023).

Solomon, D. H. et al. Artificial intelligence, authorship, and medical publishing. Arthritis Rheumatol. 75 , 867–868 (2023).

Nature editorial. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. Nature 613 , 612 https://www.nature.com/articles/d41586-023-00191-1 (2023).

Hügle, T. The wide range of opportunities for large language models such as ChatGPT in rheumatology. RMD Open 9 , e003105 (2023).

Jansz, J., Manansala, M. J. & Sweiss, N. J. Treatment of periorbital edema in a patient with systemic lupus erythematosus during pregnancy: A case report written with the assistance of ChatGPT. Cureus. https://doi.org/10.7759/cureus.36302 (2023).

Krusche, M., Callhoff, J., Knitza, J. & Ruffer, N. Diagnostic accuracy of a large language model in rheumatology: Comparison of physician and ChatGPT-4. Rheumatol. Int. https://doi.org/10.1007/s00296-023-05464-6 (2023).

Grabb, D. ChatGPT in medical education: A paradigm shift or a dangerous tool? Acad. Psychiatry 47 , 439–440 (2023).

van de Ridder, J. M. M., Shoja, M. M. & Rajput, V. Finding the place of ChatGPT in medical education. Acad. Med. 98 , 867–867 (2023).

Munaf, U., Ul-Haque, I. & Arif, T. B. ChatGPT: A helpful tool for resident physicians? Acad. Med. 98 , 868–869 (2023).

Feng, S. & Shen, Y. ChatGPT and the future of medical education. Acad. Med. 98 , 867–868 (2023).

Seetharaman, R. Revolutionizing medical education: Can ChatGPT boost subjective learning and expression? J. Med. Syst. 47 , 61 (2023).

Kung, T. H. et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digital Health 2 , e0000198 (2023).

OpenAI. GPT-4. Preprint at (2023).

OpenAI. ChatGPT—Release notes. Preprint at (2023).

de España, M. D. & Sanidad, G. BOE-A-2022-14414. II. Autoridades y personal B. Oposiciones y concursos. Preprint at https://www.boe.es/boe/dias/2022/09/02/pdfs/BOE-A-2022-14414.pdf (2022).

Feng, G. C. Mistakes and how to avoid mistakes in using intercoder reliability indices. Methodology 11 , 13–22 (2015).

Gwet, K. L. Computing inter-rater reliability and its variance in the presence of high agreement. Br. J. Math. Stat. Psychol. 61 , 29–48 (2008).

Article   MathSciNet   PubMed   Google Scholar  

García, A. M. RheumaMIR. Preprint at 10.5281/zenodo.8153291 (2023).

Jin, Q., Yang, Y., Chen, Q. & Lu, Z. GeneGPT: Augmenting large language models with domain tools for improved access to biomedical information. Preprint at (2023).

Wu, C., Zhang, X., Zhang, Y., Wang, Y. & Xie, W. PMC-LLaMA: Further finetuning LLaMA on medical papers. Preprint at (2023).

Carrasco, J. P. et al. ¿Es capaz “ChatGPT” de aprobar el examen MIR de 2022? Implicaciones de la inteligencia artificial en la educación médica en España. Rev. Esp. Educ. Méd. 4 , 1 (2023).

Google Scholar  

Ghosh, A. & Bir, A. Evaluating ChatGPT’s ability to solve higher-order questions on the competency-based medical education curriculum in medical biochemistry. Cureus. https://doi.org/10.7759/cureus.37023 (2023).

Das, D. et al. Assessing the capability of ChatGPT in answering first- and second-order knowledge questions on microbiology as per competency-based medical education curriculum. Cureus. https://doi.org/10.7759/cureus.36034 (2023).

Humar, P., Asaad, M., Bengur, F. B. & Nguyen, V. ChatGPT is equivalent to first-year plastic surgery residents: Evaluation of ChatGPT on the plastic surgery in-service examination. Aesthet. Surg. J. 43 , 1085–1089 (2023).

Li, S. W. et al. ChatGPT outscored human candidates in a virtual objective structured clinical examination in obstetrics and gynecology. Am. J. Obstet. Gynecol. 229 , e1–e12 (2023).

Nakhleh, A., Spitzer, S. & Shehadeh, N. ChatGPT’s response to the diabetes knowledge questionnaire: Implications for diabetes education. Diabetes Technol. Ther. 25 , 571–573 (2023).

Strong, E. et al. Performance of ChatGPT on free-response, clinical reasoning exams. MedRxiv. https://doi.org/10.1101/2023.03.24.23287731 (2023).

Subramani, M., Jaleel, I. & Krishna Mohan, S. Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Adv. Physiol. Educ. 47 , 270–271 (2023).

Yeo, Y. H. et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin. Mol. Hepatol. 29 , 721–732 (2023).

Morreel, S., Mathysen, D. & Verhoeven, V. Aye, AI! ChatGPT passes multiple-choice family medicine exam. Med. Teach. 45 , 665–666 (2023).

Giannos, P. & Delardas, O. Performance of ChatGPT on UK standardized admission tests: Insights from the BMAT, TMUA, LNAT, and TSA examinations. JMIR Med. Educ. 9 , e47737 (2023).

Beltrami, E. J. & Grant-Kels, J. M. Consulting ChatGPT: Ethical dilemmas in language model artificial intelligence. J. Am. Acad. Dermatol. https://doi.org/10.1016/j.jaad.2023.02.052 (2023).

Wang, C. et al. Ethical considerations of using ChatGPT in health care. J. Med. Internet Res. 25 , e48009 (2023).

Ferrara, E. Should ChatGPT be biased? Challenges and risks of bias in large language models. First Monday. https://doi.org/10.5210/fm.v28i11.13346 (2023).

Au Yeung, J. et al. AI chatbots not yet ready for clinical use. Front. Digit. Health 5 , 60 (2023).

Knitza, J. et al. Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: A randomized controlled trial among medical students. Arthritis Res. Ther. 23 , 233 (2021).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Ray, P. P. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber Phys. Syst. 3 , 121–154 (2023).

European Parliament. Proposal for a regulation of the European Parliament and of the Council on harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts. Preprint at https://www.europarl.europa.eu/news/es/press-room/20230505IPR84904/ai-act-a-step-closer-to-the-first-rules-on-artificial-intelligence (2023).

Uz, C. & Umay, E. “Dr ChatGPT”: Is it a reliable and useful source for common rheumatic diseases? Int. J. Rheum. Dis. 26 , 1343–1349 (2023).

Feinstein, A. R. & Cicchetti, D. V. High agreement but low Kappa: I. The problems of two paradoxes. J. Clin. Epidemiol. 43 , 543–549 (1990).

Cicchetti, D. V. & Feinstein, A. R. High agreement but low kappa: II. Resolving the paradoxes. J. Clin. Epidemiol. 43 , 551–558 (1990).

Quarfoot, D. & Levine, R. A. How robust are multirater interrater reliability indices to changes in frequency distribution? Am. Stat. 70 , 373–384 (2016).

Article   MathSciNet   MATH   Google Scholar  

Seghier, M. L. ChatGPT: Not all languages are equal. Nature 615 , 216 (2023).

Article   ADS   CAS   PubMed   Google Scholar  

Download references

This work was supported by the Instituto de Salud Carlos III, Ministry of Health, Madrid, Spain [RD21/002/0001]. The sponsor or funding organization had no role in the design or conduct of this research. The journal’s fee was funded by the institution employing the senior author of the manuscript (Fundación Biomédica del Hospital Clínico San Carlos).

Author information

These authors contributed equally: Zulema Rosales-Rosado, Dalifer Freites-Nuñez, Inés Pérez-Sancristóbal, Esperanza Pato-Cour, Chamaida Plasencia-Rodríguez and Luis Cabeza-Osorio.

Authors and Affiliations

Grupo de Patología Musculoesquelética, Hospital Clínico San Carlos, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), Prof. Martin Lagos S/N, 28040, Madrid, Spain

Alfredo Madrid-García, Zulema Rosales-Rosado, Dalifer Freites-Nuñez, Inés Pérez-Sancristóbal, Esperanza Pato-Cour, Lydia Abasolo-Alcázar, Leticia León-Mateos, Benjamín Fernández-Gutiérrez & Luis Rodríguez-Rodríguez

Reumatología, Hospital Universitario La Paz-IdiPaz, Paseo de La Castellana, 261, 28046, Madrid, Spain

Chamaida Plasencia-Rodríguez

Medicina Interna, Hospital Universitario del Henares, Avenida de Marie Curie, 0, 28822, Madrid, Spain

Luis Cabeza-Osorio

Facultad de Medicina, Universidad Francisco de Vitoria, Carretera Pozuelo, Km 1800, 28223, Madrid, Spain

Facultad de Medicina, Universidad Complutense de Madrid, Madrid, Spain

Benjamín Fernández-Gutiérrez

You can also search for this author in PubMed   Google Scholar

Contributions

A.M.-G.: Conceptualization of this study, methodology, review, writing (original draft preparation). Z.R.-R.: Evaluation. D.F.-N.: Evaluation. I.P.-S.: Evaluation. E.P.-C.: Evaluation. C.P.-R.: Evaluation. L.C.-O.: Evaluation. L.L.-M.: Methodology. L.A.-A.: Methodology. B.F.-G.: Conceptualization of this study. L.R.-R.: Conceptualization of this study, methodology, review. All of the authors were involved in the drafting and/or revising of the manuscript. During the preparation of this work, the author(s) used ChatGPT May 12th version, 2023, (OpenAI, San Francisco, CA, USA) as a writing aid in the composition of this scientific article, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Corresponding author

Correspondence to Alfredo Madrid-García .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information 1., supplementary information 2., supplementary information 3., supplementary information 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Madrid-García, A., Rosales-Rosado, Z., Freites-Nuñez, D. et al. Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training. Sci Rep 13 , 22129 (2023). https://doi.org/10.1038/s41598-023-49483-6

Download citation

Received : 16 October 2023

Accepted : 08 December 2023

Published : 13 December 2023

DOI : https://doi.org/10.1038/s41598-023-49483-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

can chatgpt answer case study questions

ChatGPT: Everything you need to know about the AI-powered chatbot

can chatgpt answer case study questions

ChatGPT, OpenAI’s text-generating AI chatbot, has taken the world by storm. What started as a tool to hyper-charge productivity through writing essays and code with short text prompts has evolved into a behemoth used by more than 92% of Fortune 500 companies for more wide-ranging needs. And that growth has propelled OpenAI itself into becoming one of the most-hyped companies in recent memory, even if CEO and co-founder Sam Altman’s firing and swift return  raised concerns about its direction and opened the door for competitors.

What does that mean for OpenAI, ChatGPT and its other ambitions? The fallout is still settling, but it might empower competitors like Meta and its LLaMA family of large language models , or help other AI startups get attention and funding as the industry watches OpenAI implode and put itself back together.

While there is a more… nefarious side to ChatGPT, it’s clear that AI tools are not going away anytime soon. Since its initial launch nearly a year ago, ChatGPT has hit 100 million weekly active users , and OpenAI is heavily investing in it.

Prior to the leadership chaos, on November 6, OpenAI held its first developer conference: OpenAI DevDay. During the conference, it announced a slew of updates coming to GPT, including GPT-4 Turbo (a super-charged version of GPT-4 , its latest language-writing model) and a multimodal API . OpenAI also unveiled the GPT store , where users could create and monetize their own custom versions of GPT. Though the launch was delayed in December , it officially launched in January.

GPT-4, which can write more naturally and fluently than previous models, remains largely exclusive to paying ChatGPT users. But you can access GPT-4 for free through Microsoft’s Bing Chat in Microsoft Edge, Google Chrome and Safari web browsers. Beyond GPT-4 and OpenAI DevDay announcements, OpenAI recently connected ChatGPT to the internet for all users. And with the integration of DALL-E 3, users are also able to generate both text prompts and images right in ChatGPT. 

Here’s a timeline of ChatGPT product updates and releases, starting with the latest, which we’ve been updating throughout the year. And if you have any other questions, check out our ChatGPT FAQ here .

Timeline of the most recent ChatGPT updates

February 2024, january 2024, december 2023.

  • November 2023 

October 2023

September 2023, august 2023, february 2023, january 2023, december 2022, november 2022.

  • ChatGPT FAQs

The New York Times responds to OpenAI’s claims that it “hacked” ChatGPT for its copyright lawsuit

In a court filing opposing OpenAI’s motion to dismiss The New York Times’ lawsuit alleging copyright infringement, the newspaper asserted that “OpenAI’s attention-grabbing claim that The Times ‘hacked’ its products is as irrelevant as it is false.” The New York Times also claimed that some users of ChatGPT used the tool to bypass its paywalls.

OpenAI VP doesn’t say whether artists should be paid for training data

At a SXSW 2024 panel, Peter Deng, OpenAI’s VP of consumer product dodged a question on whether artists whose work was used to train generative AI models should be compensated . While OpenAI lets artists “opt out” of and remove their work from the datasets that the company uses to train its image-generating models, some artists have described the tool as onerous.

A new report estimates that ChatGPT uses more than half a million kilowatt-hours of electricity per day

ChatGPT’s environmental impact appears to be massive. According to a report from The New Yorker , ChatGPT uses an estimated 17,000 times the amount of electricity than the average U.S. household to respond to roughly 200 million requests each day.

ChatGPT can now read its answers aloud

OpenAI released a new Read Aloud feature for the web version of ChatGPT as well as the iOS and Android apps. The feature allows ChatGPT to read its responses to queries in one of five voice options and can speak 37 languages, according to the company. Read aloud is available on both GPT-4 and GPT-3.5 models.

ChatGPT can now read responses to you. On iOS or Android, tap and hold the message and then tap “Read Aloud”. We’ve also started rolling on web – click the "Read Aloud" button below the message. pic.twitter.com/KevIkgAFbG — OpenAI (@OpenAI) March 4, 2024

OpenAI partners with Dublin City Council to use GPT-4 for tourism

As part of a new partnership with OpenAI, the Dublin City Council will use GPT-4 to craft personalized itineraries for travelers, including recommendations of unique and cultural destinations, in an effort to support tourism across Europe.

A law firm used ChatGPT to justify a six-figure bill for legal services

New York-based law firm Cuddy Law was criticized by a judge for using ChatGPT to calculate their hourly billing rate . The firm submitted a $113,500 bill to the court, which was then halved by District Judge Paul Engelmayer, who called the figure “well above” reasonable demands.

ChatGPT experienced a bizarre bug for several hours

ChatGPT users found that ChatGPT was giving nonsensical answers for several hours , prompting OpenAI to investigate the issue. Incidents varied from repetitive phrases to confusing and incorrect answers to queries. The issue was resolved by OpenAI the following morning.

Match Group announced deal with OpenAI with a press release co-written by ChatGPT

The dating app giant home to Tinder, Match and OkCupid announced an enterprise agreement with OpenAI in an enthusiastic press release written with the help of ChatGPT . The AI tech will be used to help employees with work-related tasks and come as part of Match’s $20 million-plus bet on AI in 2024.

ChatGPT will now remember — and forget — things you tell it to

As part of a test, OpenAI began rolling out new “memory” controls for a small portion of ChatGPT free and paid users, with a broader rollout to follow. The controls let you tell ChatGPT explicitly to remember something, see what it remembers or turn off its memory altogether. Note that deleting a chat from chat history won’t erase ChatGPT’s or a custom GPT’s memories — you must delete the memory itself.

We’re testing ChatGPT's ability to remember things you discuss to make future chats more helpful. This feature is being rolled out to a small portion of Free and Plus users, and it's easy to turn on or off. https://t.co/1Tv355oa7V pic.twitter.com/BsFinBSTbs — OpenAI (@OpenAI) February 13, 2024

OpenAI begins rolling out “Temporary Chat” feature

Initially limited to a small subset of free and subscription users, Temporary Chat lets you have a dialogue with a blank slate. With Temporary Chat, ChatGPT won’t be aware of previous conversations or access memories but will follow custom instructions if they’re enabled.

But, OpenAI says it may keep a copy of Temporary Chat conversations for up to 30 days for “safety reasons.”

Use temporary chat for conversations in which you don’t want to use memory or appear in history. pic.twitter.com/H1U82zoXyC — OpenAI (@OpenAI) February 13, 2024

ChatGPT users can now invoke GPTs directly in chats

Paid users of ChatGPT can now bring GPTs into a conversation by typing “@” and selecting a GPT from the list. The chosen GPT will have an understanding of the full conversation, and different GPTs can be “tagged in” for different use cases and needs.

You can now bring GPTs into any conversation in ChatGPT – simply type @ and select the GPT. This allows you to add relevant GPTs with the full context of the conversation. pic.twitter.com/Pjn5uIy9NF — OpenAI (@OpenAI) January 30, 2024

ChatGPT is reportedly leaking usernames and passwords from users’ private conversations

Screenshots provided to Ars Technica found that ChatGPT is potentially leaking unpublished research papers, login credentials and private information from its users. An OpenAI representative told Ars Technica that the company was investigating the report.

ChatGPT is violating Europe’s privacy laws, Italian DPA tells OpenAI

OpenAI has been told it’s suspected of violating European Union privacy , following a multi-month investigation of ChatGPT by Italy’s data protection authority. Details of the draft findings haven’t been disclosed, but in a response, OpenAI said: “We want our AI to learn about the world, not about private individuals.”

OpenAI partners with Common Sense Media to collaborate on AI guidelines

In an effort to win the trust of parents and policymakers, OpenAI announced it’s partnering with Common Sense Media to collaborate on AI guidelines and education materials for parents, educators and young adults. The organization works to identify and minimize tech harms to young people and previously flagged ChatGPT as lacking in transparency and privacy .

OpenAI responds to Congressional Black Caucus about lack of diversity on its board

After a letter from the Congressional Black Caucus questioned the lack of diversity in OpenAI’s board, the company responded . The response, signed by CEO Sam Altman and Chairman of the Board Bret Taylor, said building a complete and diverse board was one of the company’s top priorities and that it was working with an executive search firm to assist it in finding talent. 

OpenAI drops prices and fixes ‘lazy’ GPT-4 that refused to work

In a blog post , OpenAI announced price drops for GPT-3.5’s API, with input prices dropping to 50% and output by 25%, to $0.0005 per thousand tokens in, and $0.0015 per thousand tokens out. GPT-4 Turbo also got a new preview model for API use, which includes an interesting fix that aims to reduce “laziness” that users have experienced.

Expanding the platform for @OpenAIDevs : new generation of embedding models, updated GPT-4 Turbo, and lower pricing on GPT-3.5 Turbo. https://t.co/7wzCLwB1ax — OpenAI (@OpenAI) January 25, 2024

OpenAI bans developer of a bot impersonating a presidential candidate

OpenAI has suspended AI startup Delphi, which developed a bot impersonating Rep. Dean Phillips (D-Minn.) to help bolster his presidential campaign. The ban comes just weeks after OpenAI published a plan to combat election misinformation, which listed “chatbots impersonating candidates” as against its policy.

OpenAI announces partnership with Arizona State University

Beginning in February, Arizona State University will have full access to ChatGPT’s Enterprise tier , which the university plans to use to build a personalized AI tutor, develop AI avatars, bolster their prompt engineering course and more. It marks OpenAI’s first partnership with a higher education institution.

Winner of a literary prize reveals around 5% her novel was written by ChatGPT

After receiving the prestigious Akutagawa Prize for her novel The Tokyo Tower of Sympathy, author Rie Kudan admitted that around 5% of the book quoted ChatGPT-generated sentences “verbatim.” Interestingly enough, the novel revolves around a futuristic world with a pervasive presence of AI.

Sam Altman teases video capabilities for ChatGPT and the release of GPT-5

In a conversation with Bill Gates on the Unconfuse Me podcast, Sam Altman confirmed an upcoming release of GPT-5 that will be “fully multimodal with speech, image, code, and video support.” Altman said users can expect to see GPT-5 drop sometime in 2024.

OpenAI announces team to build ‘crowdsourced’ governance ideas into its models

OpenAI is forming a Collective Alignment team of researchers and engineers to create a system for collecting and “encoding” public input on its models’ behaviors into OpenAI products and services. This comes as a part of OpenAI’s public program to award grants to fund experiments in setting up a “democratic process” for determining the rules AI systems follow.

OpenAI unveils plan to combat election misinformation

In a blog post, OpenAI announced users will not be allowed to build applications for political campaigning and lobbying until the company works out how effective their tools are for “personalized persuasion.”

Users will also be banned from creating chatbots that impersonate candidates or government institutions, and from using OpenAI tools to misrepresent the voting process or otherwise discourage voting.

The company is also testing out a tool that detects DALL-E generated images and will incorporate access to real-time news, with attribution, in ChatGPT.

Snapshot of how we’re preparing for 2024’s worldwide elections: • Working to prevent abuse, including misleading deepfakes • Providing transparency on AI-generated content • Improving access to authoritative voting information https://t.co/qsysYy5l0L — OpenAI (@OpenAI) January 15, 2024

OpenAI changes policy to allow military applications

In an unannounced update to its usage policy , OpenAI removed language previously prohibiting the use of its products for the purposes of “military and warfare.” In an additional statement, OpenAI confirmed that the language was changed in order to accommodate military customers and projects that do not violate their ban on efforts to use their tools to “harm people, develop weapons, for communications surveillance, or to injure others or destroy property.”

ChatGPT subscription aimed at small teams debuts

Aptly called ChatGPT Team , the new plan provides a dedicated workspace for teams of up to 149 people using ChatGPT as well as admin tools for team management. In addition to gaining access to GPT-4, GPT-4 with Vision and DALL-E3, ChatGPT Team lets teams build and share GPTs for their business needs.

OpenAI’s GPT store officially launches

After some back and forth over the last few months, OpenAI’s GPT Store is finally here . The feature lives in a new tab in the ChatGPT web client, and includes a range of GPTs developed both by OpenAI’s partners and the wider dev community.

To access the GPT Store, users must be subscribed to one of OpenAI’s premium ChatGPT plans — ChatGPT Plus, ChatGPT Enterprise or the newly launched ChatGPT Team.

the GPT store is live! https://t.co/AKg1mjlvo2 fun speculation last night about which GPTs will be doing the best by the end of today. — Sam Altman (@sama) January 10, 2024

Developing AI models would be “impossible” without copyrighted materials, OpenAI claims

Following a proposed ban on using news publications and books to train AI chatbots in the U.K., OpenAI submitted a plea to the House of Lords communications and digital committee. OpenAI argued that it would be “impossible” to train AI models without using copyrighted materials, and that they believe copyright law “does not forbid training.”

OpenAI claims The New York Times’ copyright lawsuit is without merit

OpenAI published a public response to The New York Times’s lawsuit against them and Microsoft for allegedly violating copyright law, claiming that the case is without merit.

In the response , OpenAI reiterates its view that training AI models using publicly available data from the web is fair use. It also makes the case that regurgitation is less likely to occur with training data from a single source and places the onus on users to “act responsibly.”

We build AI to empower people, including journalists. Our position on the @nytimes lawsuit: • Training is fair use, but we provide an opt-out • "Regurgitation" is a rare bug we're driving to zero • The New York Times is not telling the full story https://t.co/S6fSaDsfKb — OpenAI (@OpenAI) January 8, 2024

OpenAI’s app store for GPTs planned to launch next week

After being delayed in December , OpenAI plans to launch its GPT Store sometime in the coming week, according to an email viewed by TechCrunch. OpenAI says developers building GPTs will have to review the company’s updated usage policies and GPT brand guidelines to ensure their GPTs are compliant before they’re eligible for listing in the GPT Store. OpenAI’s update notably didn’t include any information on the expected monetization opportunities for developers listing their apps on the storefront.

GPT Store launching next week – OpenAI pic.twitter.com/I6mkZKtgZG — Manish Singh (@refsrc) January 4, 2024

OpenAI moves to shrink regulatory risk in EU around data privacy

In an email, OpenAI detailed an incoming update to its terms, including changing the OpenAI entity providing services to EEA and Swiss residents to OpenAI Ireland Limited. The move appears to be intended to shrink its regulatory risk in the European Union, where the company has been under scrutiny over ChatGPT’s impact on people’s privacy.

Study finds white-collar workers are uneasy about using ChatGPT

A study conducted by professors from Harvard and MIT , which is still under review, looked at how ChatGPT could affect the productivity of more than 750 white-collar workers, as well as their complicated feelings about using the tool. The study found that while ChatGPT was helpful with creative tasks, workers were led to more mistakes with analytical work.

The New York Times sues OpenAI and Microsoft over alleged copyright infringement

In a lawsuit filed in the Federal District Court in Manhattan , The Times argues that millions of its articles were used to train AI models without its consent. The Times is asking for OpenAI and Microsoft to “destroy” models and training data containing offending material and to be held responsible for “billions of dollars in statutory and actual damages.”

OpenAI re-opens ChatGPT Plus subscriptions

After pausing ChatGPT Plus subscriptions in November due to a “surge of usage,” OpenAI CEO Sam Altman announced they have once again enabled sign-ups. The Plus subscription includes access to GPT-4 and GPT-4 Turbo .

we have re-enabled chatgpt plus subscriptions! 🎄 thanks for your patience while we found more gpus. — Sam Altman (@sama) December 13, 2023

OpenAI and Axel Springer partner up for a “real-time” ChatGPT news deal

OpenAI has struck a new deal with Berlin-based news publisher Axel Springer , which owns Business Insider and Politico, to “help provide people with new ways to access quality, real-time news content through our AI tools.” OpenAI will train its generative AI models on the publisher’s content and add recent Axel Springer-published articles to ChatGPT.

Stanford researchers say ChatGPT didn’t cause an influx in cheating in high schools

New research from Stanford University shows that the popularization of chatbots like ChatGPT has not caused an increase in cheating across U.S. high schools. In a survey of more than 40 U.S. high schools, researchers found that cheating rates are similar across the board this year.

ChatGPT users worry the chatbot is experiencing seasonal depression

Starting in November, ChatGPT users have noticed that the chatbot feels “lazier” than normal, citing instances of simpler answers and refusing to complete requested tasks. OpenAI has confirmed that they are aware of this issue , but aren’t sure why it’s happening.

Some users think it plays into the “winter break hypothesis,” which argues that AI is worse in December because it “learned” to do less work over the holidays , while others wonder if the chatbot is simulating seasonal depression .

we've heard all your feedback about GPT4 getting lazier! we haven't updated the model since Nov 11th, and this certainly isn't intentional. model behavior can be unpredictable, and we're looking into fixing it 🫡 — ChatGPT (@ChatGPTapp) December 8, 2023

Judges in the U.K. are now allowed to use ChatGPT in legal rulings

The U.K. Judicial Office issued guidance that permits judges to use ChatGPT, along with other AI tools, to write legal rulings and perform court duties. The guidance lays out ways to responsibly use AI in the courts, including being aware of potential bias and upholding privacy.

OpenAI makes repeating words “forever” a violation of its terms of service after Google DeepMind test

Following an experiment by Google DeepMind researchers that led ChatGPT to repeat portions of its training data, OpenAI has flagged asking ChatGPT to repeat specific words “forever” as a violation of its terms of service .

Lawmakers in Brazil enact an ordinance written by ChatGPT

City lawmakers in Brazil enacted a piece of legislation written entirely by ChatGPT without even knowing. Weeks after the bill was passed, Porto Alegre councilman Ramiro Rosário admitted that he used ChatGPT to write the proposal, and did not tell fellow council members until after the fact.

OpenAI reportedly delays the launch of its GPT store to 2024

According to a memo seen by Axios , OpenAI plans to delay the launch of its highly anticipated GPT store to early 2024. Custom GPTs and the accompanying store was a major announcement at OpenAI’s DevDay conference , with the store expected to open last month.

November 2023

Chatgpts mobile apps top 110m installs and nearly $30m in revenue.

After launching for iOS and Androidin May and July, ChatGPT’s have topped 110 million combined installs and have reached nearly $30 million in consumer spending, according to a market analysis by data.ai.

ChatGPT celebrates one-year anniversary

OpenAI hit a major milestone: one year of ChatGPT . What began as a “low-key research preview” evolved into a powerhouse that changed the AI industry forever. In a post on X , CEO Sam Altman looked back on the night before its launch: “what a year it’s been…”

a year ago tonight we were probably just sitting around the office putting the finishing touches on chatgpt before the next morning’s launch. what a year it’s been… — Sam Altman (@sama) November 30, 2023

Apple and Google avoid naming ChatGPT as their ‘app of the year’

Neither Apple nor Google chose an AI app as its app of the year for 2023, despite the success of ChatGPT’s mobile app, which became the fastest-growing consumer application in history before the record was broken by Meta’s Threads .

An attack from researchers prompts ChatGPT to reveal training data

A test led by researchers at Google DeepMind found that there is a significant amount of privately identifiable information in OpenAI’s LLMs. The test involved asking ChatGPT to repeat the word “poem” forever, among other words, which over time led the chatbot to churn out private information like email addresses and phone numbers.

ChatGPT and other AI chatbots are fueling an increase in phishing emails

According to a new report by SlashNext , there’s been a 1,265% increase in malicious phishing emails since Q4 of 2022. The report alleges that AI tools like ChatGPT are being prominently used by cybercriminals to write compelling and sophisticated phishing emails .

South Africa officials investigate if President Cyril Ramaphosa used ChatGPT to write a speech

Following speculation, social media users fed portions of Ramaphosa’s November 21 speech in Johannesburg through AI detectors , alleging parts of it may have been written with ChatGPT. South African presidency spokesperson Vincent Magwenya refuted the claims, and local officials are investigating.

ChatGPT Voice can be used to replace Siri

Now that OpenAI’s ChatGPT Voice feature is available to all free users, it can be used to replace Siri on an iPhone 15 Pro and Pro Max by configuring the new Action Button. The new feature lets you ask ChatGPT questions and listen to its responses — like a much smarter version of Siri.

Sam Altman returns as CEO

Altman’s return came swiftly , with an “agreement in principle” announced between him and OpenAI’s board that will reinstate him as CEO and restructure the board to include new members, including former U.S. Treasury Secretary Larry Summers . The biggest takeaway for ChatGPT is that the members of the board more focused on the nonprofit side of OpenAI, with the most concerns over the commercialization of its tools, have been pushed to the side .

ChatGPT Voice rolls out to all free users

Even if its leadership is in flux, OpenAI is still releasing updates to ChatGPT . First announced in September and granted to paid users on a rolling basis, the text-to-speech model can create a voice from text prompts and a few seconds of speech samples. OpenAI worked with voice actors to create the five voice options, and you can give it a shot by heading to the settings in your mobile ChatGPT apps and tapping the “headphones” icon.

Sam Altman might return, but it’s complicated

The only constant within OpenAI right now is change, and in a series of interviews, Nadella hedged on earlier reporting that Altman and Brockman were headed to Microsoft .

“Obviously, we want Sam and Greg to have a fantastic home if they’re not going to be in OpenAI,” Nadella said in an interview with CNBC, saying that we was “open” to them settling at Microsoft or returning to OpenAI should the board and employees support the move.

Confirmation Sam Altman will not return as OpenAI’s CEO

A number of investors and OpenAI employees tried to bring back Altman after his sudden firing by the company’s board, but following a weekend of negotiations, it was confirmed that Altman would not return to OpenAI and new leadership would take hold. What this means for ChatGPT’s future, and for the OpenAI Dev Day announcements , remains to be seen.

Sam Altman ousted as OpenAI’s CEO

Sam Altman has been fired from OpenAI . He will leave the company’s board and step down as CEO, with OpenAI’s chief technology officer Mira Murati stepping in as interim CEO. In a blog post from OpenAI, the company writes that the board “no longer has confidence in [Altman’s] ability to continue leading OpenAI.”

In a statement on X , Altman said working at OpenAI “was transformative” for him and “hopefully the world.”

OpenAI explores how ChatGPT can be used in the classroom

OpenAI COO Brad Lightcap revealed at a San Francisco conference that the company will likely create a team to identify ways AI and ChatGPT can be used in education . This announcement comes at a time when ChatGPT is being criticized by educators for encouraging cheating , resulting in bans in certain school districts .

OpenAI pauses new ChatGPT Plus subscriptions due to a “surge of usage”

Following OpenAI’s Dev Day conference , Sam Altman announced the company is putting a pause on new subscriptions for its premium ChatGPT Plus offering. The temporary hold on sign-ups, as well as the demand for ChatGPT Plus’ new features like making custom GPTS , has led to a slew of resellers on eBay .

ChatGPT gets flagged as potentially unsafe for kids

An independent review from Common Sense Media, a nonprofit advocacy group, found that  ChatGPT could potentially be harmful for younger users. ChatGPT got an overall three-star rating in the report, with its lowest ratings relating to transparency, privacy, trust and safety. 

OpenAI blames DDoS attack for ChatGPT outage

OpenAI confirmed that a DDoS attack was behind outages affecting ChatGPT and its developer tools. ChatGPT experienced sporadic outages for about 24 hours, resulting in users being unable to log into or use the service.

OpenAI debuts GPT-4 Turbo

OpenAI unveiled GPT-4 Turbo at its first-ever OpenAI DevDay conference. GPT-4 Turbo comes in two versions: one that’s strictly text-analyzing and another that understands the context of both text and images.

GPT-4 gets a fine-tuning

As opposed to the fine-tuning program for GPT-3.5, the GPT-4 program will involve more oversight and guidance from OpenAI teams, the company says — largely due to technical hurdles.

OpenAI’s GPT Store lets you build (and monetize) your own GPT

Users and developers will soon be able to make their own GPT , with no coding experience required. Anyone building their own GPT will also be able to list it on OpenAI’s marketplace and monetize it in the future.

ChatGPT has 100 million weekly active users

After being released nearly a year ago, ChatGPT has 100 million weekly active users . OpenAI CEO Sam Altman also revealed that over two million developers use the platform, including more than 92% of Fortune 500 companies.

OpenAI launches DALL-E 3 API, new text-to-speech models

DALL-E 3, OpenAI’s text-to-image model , is now available via an API after first coming to ChatGPT-4 and Bing Chat. OpenAI’s newly released text-to-speech API, Audio API, offers six preset voices to choose from and two generative AI model variants.

OpenAI promises to defend business customers against copyright claims

Bowing to peer pressure, OpenAI it will pay legal costs incurred by customers who face lawsuits over IP claims against work generated by an OpenAI tool. The protections seemingly don’t extend to all OpenAI products, like the free and Plus tiers of ChatGPT.

As OpenAI’s multimodal API launches broadly, research shows it’s still flawed

OpenAI announced that GPT-4 with vision will become available alongside the upcoming launch of GPT-4 Turbo API. But some researchers found that the model remains flawed in several significant and problematic ways.

OpenAI launches API, letting developers build ‘assistants’ into their apps

At its OpenAI DevDay, OpenAI announced the Assistants API to help developers build “agent-like experiences” within their apps. Use cases range from a natural language-based data analysis app to a coding assistant or even an AI-powered vacation planner.

ChatGPT app revenue shows no signs of slowing, but it’s not #1

OpenAI’s chatbot app far outpaces all others on mobile devices in terms of downloads, but it’s surprisingly not the top AI app by revenue . Several other AI chatbots, like  “Chat & Ask AI” and “ChatOn — AI Chat Bot Assistant”, are actually making more money than ChatGPT.

ChatGPT tests the ability to upload and analyze files for Plus users

Subscribers to ChatGPT’s Enterprise Plan have reported new beta features, including the ability to upload PDFs to analyze and and ask questions about them directly. The new rollout also makes it so users no longer have to manually select a mode like DALL-E and browsing when using ChatGPT. Instead, users will automatically be switched to models based on the prompt.

ChatGPT officially gets web search

OpenAI has formally launched its internet-browsing feature to ChatGPT, some three weeks after re-introducing the feature in beta after several months in hiatus. The AI chatbot that has historically been limited to data up to September, 2021.

OpenAI integrates DALL-E 3 into ChatGPT

The integration means users don’t have to think so carefully about their text-prompts when asking DALL-E to create an image. Users will also now be able to receive images as part of their text-based queries without having to switch between apps.

Microsoft-affiliated research finds flaws in GPT-4

A Microsoft-affiliated scientific paper looked at the “trustworthiness” — and toxicity — of LLMs, including GPT-4. Because GPT-4 is more likely to follow the instructions of “jailbreaking” prompts, the co-authors claim that GPT-4 can be more easily prompted than other LLMs to spout toxic, biased text .

ChatGPT’s mobile app hits record $4.58M in revenue in September

OpenAI amassed 15.6 million downloads and nearly $4.6 million in gross revenue across its iOS and Android apps worldwide in September. But revenue growth has now begun to slow , according to new data from market intelligence firm Appfigures — dropping from 30% to 20% in September.

ChatGPT can now browse the internet (again)

OpenAI posted on Twitter/X that ChatGPT can now browse the internet and is no longer limited to data before September 2021. The chatbot had a web browsing capability for Plus subscribers back in July , but the feature was taken away after users exploited it to get around paywalls.

ChatGPT can now browse the internet to provide you with current and authoritative information, complete with direct links to sources. It is no longer limited to data before September 2021. pic.twitter.com/pyj8a9HWkB — OpenAI (@OpenAI) September 27, 2023

ChatGPT now has a voice

OpenAI announced that it’s adding a new voice for verbal conversations and image-based smarts to the AI-powered chatbot.

Poland opens an investigation against OpenAI

The Polish authority publically announced it has opened an investigation regarding ChatGPT — accusing the company of a string of breaches of the EU’s General Data Protection Regulation (GDPR).

OpenAI unveils DALL-E 3

The upgraded text-to-image tool, DALL-E 3, uses ChatGPT to help fill in prompts. Subscribers to OpenAI’s premium ChatGPT plans, ChatGPT Plus  and  ChatGPT Enterprise , can type in a request for an image and hone it through conversations with the chatbot — receiving the results directly within the chat app.

Opera GX integrates ChatGPT-powered AI

Powered by OpenAI’s ChatGPT, the AI browser Aria  launched on Opera in May to give users an easier way to search, ask questions and write code. Today, the company announced it is bringing Aria to Opera GX , a version of the flagship Opera browser that is built for gamers.

The new feature allows Opera GX users to interact directly with a browser AI to find the latest gaming news and tips.

OpenAI releases a guide for teachers using ChatGPT in the classroom

OpenAI wants to rehabilitate the system’s image a bit when it comes to education, as ChatGPT has been controversial in the classroom due to plagiarism. OpenAI has offered up a selection of ways to put the chatbot to work in the classroom.

OpenAI launches ChatGPT Enterprise

ChatGPT Enterprise can perform the same tasks as ChatGPT, such as writing emails, drafting essays and debugging computer code. However, the new offering also adds “enterprise-grade” privacy and data analysis capabilities on top of the vanilla ChatGPT, as well as enhanced performance and customization options.

Survey finds relatively few American use ChatGPT

Recent Pew polling suggests the language model isn’t quite as popular or threatening as some would have you think. Ongoing polling by Pew Research shows that although ChatGPT is gaining mindshare, only about 18% of Americans have ever actually used it .

OpenAI brings fine-tuning to GPT-3.5 Turbo

With fine-tuning, companies using GPT-3.5 Turbo through the company’s API can make the model better follow specific instructions. For example, having the model always respond in a given language. Or improving the model’s ability to consistently format responses, as well as hone the “feel” of the model’s output, like its tone, so that it better fits a brand or voice. Most notably, fine-tuning enables OpenAI customers to shorten text prompts to speed up API calls and cut costs.

OpenAI is partnering with Scale AI to allow companies to fine-tune GPT-3.5 . However, it is unclear whether OpenAI is developing an in-house tuning tool that is meant to complement platforms like Scale AI or serve a different purpose altogether.

Fine-tuning costs:

  • Training: $0.008 / 1K tokens
  • Usage input: $0.012 / 1K tokens
  • Usage output: $0.016 / 1K tokens

OpenAI acquires Global Illumination

In OpenAI’s first public acquisition in its seven-year history, the company announced it has acquired Global Illumination, a New York-based startup leveraging AI to build creative tools, infrastructure and digital experiences.

“We’re very excited for the impact they’ll have here at OpenAI,” OpenAI wrote in a brief  post published to its official blog. “The entire team has joined OpenAI to work on our core products including ChatGPT.”

The ‘custom instructions’ feature is extended to free ChatGPT users

OpenAI announced that it’s expanding custom instructions to all users, including those on the free tier of service. The feature allows users to add various preferences and requirements that they want the AI chatbot to consider when responding.

China requires AI apps to obtain an administrative license

Multiple generative AI apps have been removed from Apple’s China App Store ahead of the country’s latest generative AI regulations that are set to take effect August 15.

“As you may know, the government has been tightening regulations associated with deep synthesis technologies (DST) and generative AI services, including ChatGPT. DST must fulfill permitting requirements to operate in China, including securing a license from the Ministry of Industry and Information Technology (MIIT),” Apple said in a letter to OpenCat, a native ChatGPT client. “Based on our review, your app is associated with ChatGPT, which does not have requisite permits to operate in China.”

ChatGPT for Android is now available in the US, India, Bangladesh and Brazil

A few days after putting up a preorder page on Google Play, OpenAI has flipped the switch and  released ChatGPT for Android . The app is now live in a handful of countries.

ChatGPT is coming to Android

ChatGPT is available to “pre-order” for Android users.

The ChatGPT app on Android  looks to be more or less identical to the iOS one in functionality, meaning it gets most if not all of the web-based version’s features. You should be able to sync your conversations and preferences across devices, too — so if you’re iPhone at home and Android at work, no worries.

OpenAI launches customized instructions for ChatGPT

OpenAI launched custom instructions for ChatGPT users , so they don’t have to write the same instruction prompts to the chatbot every time they interact with it.

The company said this feature lets you “share anything you’d like ChatGPT to consider in its response.” For example, a teacher can say they are teaching fourth-grade math or a developer can specify the code language they prefer when asking for suggestions. A person can also specify their family size, so the text-generating AI can give responses about meals, grocery and vacation planning accordingly.

The FTC is reportedly investigating OpenAI

The FTC is reportedly in at least the exploratory phase of investigation over whether OpenAI’s flagship ChatGPT conversational AI made “false, misleading, disparaging or harmful” statements about people.

TechCrunch Reporter Devin Coldewey reports:

This kind of investigation doesn’t just appear out of thin air — the FTC doesn’t look around and say “That looks suspicious.” Generally a lawsuit or formal complaint is brought to their attention and the practices described by it imply that regulations are being ignored. For example, a person may sue a supplement company because the pills made them sick, and the FTC will launch an investigation on the back of that because there’s evidence the company lied about the side effects.

OpenAI announced the general availability of GPT-4

Starting July 6, all existing OpenAI developers “with a history of successful payments” can access GPT-4 . OpenAI plans to open up access to new developers by the end of July.

In the future, OpenAI says that it’ll allow developers to fine-tune GPT-4 and  GPT-3.5 Turbo , one of the original models powering ChatGPT, with their own data, as has long been possible with several of OpenAI’s other text-generating models. That capability should arrive later this year, according to OpenAI.

ChatGPT app can now search the web only on Bing

OpenAI announced that subscribers to ChatGPT Plus can now use a new feature on the app called Browsing , which allows ChatGPT to search Bing for answers to questions.

The Browsing feature can be enabled by heading to the New Features section of the app settings, selecting “GPT-4” in the model switcher and choosing “Browse with Bing” from the drop-down list. Browsing is available on both the iOS and Android ChatGPT apps.

Mercedes is adding ChatGPT to its infotainment system

U.S. owners of Mercedes models that use MBUX will be able to opt into a beta program starting June 16 activating the ChatGPT functionality . This will enable the highly versatile large language model to augment the car’s conversation skills. You can join up simply by telling your car “Hey Mercedes, I want to join the beta program.”

It’s not really clear what for, though.

ChatGPT app is now available on iPad, adds support for Siri and Shortcuts

The new ChatGPT app version brings native iPad support to the app , as well as support for using the chatbot with Siri and Shortcuts. Drag and drop is also now available, allowing users to drag individual messages from ChatGPT into other apps.

On iPad, ChatGPT now runs in full-screen mode, optimized for the tablet’s interface.

Texas judge orders all AI-generated content must be declared and checked

The Texas federal judge has added a requirement that any attorney appearing in his court must attest that “no portion of the filing was drafted by generative artificial intelligence,” or if it was, that it was checked “by a human being.”

ChatGPT app expanded to more than 30 countries

The list of new countries includes Algeria, Argentina, Azerbaijan, Bolivia, Brazil, Canada, Chile, Costa Rica, Ecuador, Estonia, Ghana, India, Iraq, Israel, Japan, Jordan, Kazakhstan, Kuwait, Lebanon, Lithuania, Mauritania, Mauritius, Mexico, Morocco, Namibia, Nauru, Oman, Pakistan, Peru, Poland, Qatar, Slovenia, Tunisia and the United Arab Emirates.

ChatGPT app is now available in 11 more countries

OpenAI announced in a tweet that the ChatGPT mobile app is now available on iOS in the U.S., Europe, South Korea and New Zealand, and soon more will be able to download the app from the app store. In just six days, the app topped 500,000 downloads .

The ChatGPT app for iOS is now available to users in 11 more countries — Albania, Croatia, France, Germany, Ireland, Jamaica, Korea, New Zealand, Nicaragua, Nigeria, and the UK. More to come soon! — OpenAI (@OpenAI) May 24, 2023

OpenAI launches a ChatGPT app for iOS

ChatGPT is officially going mobile . The new ChatGPT app will be free to use, free from ads and will allow for voice input, the company says, but will initially be limited to U.S. users at launch.

When using the mobile version of ChatGPT, the app will sync your history across devices — meaning it will know what you’ve previously searched for via its web interface, and make that accessible to you. The app is also integrated with  Whisper , OpenAI’s open source speech recognition system, to allow for voice input.

Hackers are using ChatGPT lures to spread malware on Facebook

Meta said in a report on May 3 that malware posing as ChatGPT was on the rise across its platforms . The company said that since March 2023, its security teams have uncovered 10 malware families using ChatGPT (and similar themes) to deliver malicious software to users’ devices.

“In one case, we’ve seen threat actors create malicious browser extensions available in official web stores that claim to offer ChatGPT-based tools,” said Meta security engineers Duc H. Nguyen and Ryan Victory in  a blog post . “They would then promote these malicious extensions on social media and through sponsored search results to trick people into downloading malware.”

ChatGPT parent company OpenAI closes $300M share sale at $27B-29B valuation

VC firms including Sequoia Capital, Andreessen Horowitz, Thrive and K2 Global are picking up new shares, according to documents seen by TechCrunch. A source tells us Founders Fund is also investing. Altogether the VCs have put in just over $300 million at a valuation of $27 billion to $29 billion . This is separate to a big investment from Microsoft announced earlier this year , a person familiar with the development told TechCrunch, which closed in January. The size of Microsoft’s investment is believed to be around $10 billion, a figure we confirmed with our source.

OpenAI previews new subscription tier, ChatGPT Business

Called ChatGPT Business, OpenAI describes the forthcoming offering as “for professionals who need more control over their data as well as enterprises seeking to manage their end users.”

“ChatGPT Business will follow our API’s data usage policies, which means that end users’ data won’t be used to train our models by default,” OpenAI  wrote in a blog post. “We plan to make ChatGPT Business available in the coming months.”

OpenAI wants to trademark “GPT”

OpenAI applied for a trademark for “GPT,” which stands for “Generative Pre-trained Transformer,” last December. Last month, the company petitioned the USPTO to speed up the process, citing the “myriad infringements and counterfeit apps” beginning to spring into existence.

Unfortunately for OpenAI, its petition was  dismissed  last week. According to the agency, OpenAI’s attorneys neglected to pay an associated fee as well as provide “appropriate documentary evidence supporting the justification of special action.”

That means a decision could take up to five more months.

Auto-GPT is Silicon Valley’s latest quest to automate everything

Auto-GPT is an open-source app created by game developer Toran Bruce Richards that uses OpenAI’s latest text-generating models, GPT-3.5 and GPT-4, to interact with software and services online, allowing it to “autonomously” perform tasks.

Depending on what objective the tool’s provided, Auto-GPT can behave in very… unexpected ways. One Reddit  user  claims that, given a budget of $100 to spend within a server instance, Auto-GPT made a wiki page on cats, exploited a flaw in the instance to gain admin-level access and took over the Python environment in which it was running — and then “killed” itself.

FTC warns that AI technology like ChatGPT could ‘turbocharge’ fraud

FTC chair Lina Khan and fellow commissioners warned House representatives of the potential for modern AI technologies, like ChatGPT, to be used to “turbocharge” fraud in a congressional hearing .

“AI presents a whole set of opportunities, but also presents a whole set of risks,” Khan told the House representatives. “And I think we’ve already seen ways in which it could be used to turbocharge fraud and scams. We’ve been putting market participants on notice that instances in which AI tools are effectively being designed to deceive people can place them on the hook for FTC action,” she stated.

Superchat’s new AI chatbot lets you message historical and fictional characters via ChatGPT

The company behind the popular iPhone customization app  Brass , sticker maker  StickerHub  and  others  is out today with a new AI chat app called  SuperChat , which allows iOS users to chat with virtual characters powered by OpenAI’s ChatGPT . However, what makes the app different from the default experience or the dozens of generic AI chat apps now available are the characters offered which you can use to engage with SuperChat’s AI features.

Italy gives OpenAI to-do list for lifting ChatGPT suspension order

Italy’s data protection watchdog has laid out what OpenAI needs to do for it to lift an order against ChatGPT issued at the  end of last month — when it said it suspected the AI chatbot service was in breach of the EU’s GSPR and ordered the U.S.-based company to stop processing locals’ data.

The DPA has given OpenAI a deadline — of April 30 — to get the regulator’s compliance demands done. (The local radio, TV and internet awareness campaign has a slightly more generous timeline of May 15 to be actioned.)

Researchers discover a way to make ChatGPT consistently toxic

A study co-authored by scientists at the Allen Institute for AI shows that assigning ChatGPT a “persona” — for example, “a bad person,” “a horrible person” or “a nasty person” — through the ChatGPT API increases its toxicity sixfold. Even more concerning, the co-authors found having the conversational AI chatbot pose as certain historical figures, gendered people and members of political parties also increased its toxicity — with journalists, men and Republicans in particular causing the machine learning model to say more offensive things than it normally would.

The research was conducted using the latest version, but not the model currently in preview based on OpenAI’s GPT-4 .

Y Combinator-backed startups are trying to build ‘ChatGPT for X’

YC Demo Day’s Winter 2023 batch features no fewer than four startups that claim to be building “ChatGPT for X.” They’re all chasing after a customer service software market that’ll be worth $58.1 billion by 2023, assuming the rather optimistic prediction from Acumen Research comes true.

Here are the YC-backed startups that caught our eye:

  • Yuma , whose customer demographic is primarily Shopify merchants, provides ChatGPT-like AI systems that integrate with help desk software, suggesting drafts of replies to customer tickets.
  • Baselit , which uses one of OpenAI’s text-understanding models to allow businesses to embed chatbot-style analytics for their customers.
  • Lasso customers send descriptions or videos of the processes they’d like to automate and the company combines ChatGPT-like interface with robotic process automation (RPA) and a Chrome extension to build out those automations.
  • BerriAI , whose platform is designed to help developers spin up ChatGPT apps for their organization data through various data connectors.

Italy orders ChatGPT to be blocked

OpenAI has started geoblocking access to its generative AI chatbot, ChatGPT, in Italy .

Italy’s data protection authority has just put out a timely reminder that some countries do have laws that already apply to cutting edge AI: it has  ordered OpenAI to stop processing people’s data locally with immediate effect. The Italian DPA said it’s concerned that the ChatGPT maker is breaching the European Union’s General Data Protection Regulation (GDPR), and is opening an investigation.

1,100+ signatories signed an open letter asking all ‘AI labs to immediately pause for 6 months’

The letter’s signatories include Elon Musk, Steve Wozniak and Tristan Harris of the Center for Humane Technology, among others. The letter calls on “all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.”

The letter reads:

Contemporary AI systems are now becoming human-competitive at general tasks,[3] and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable.

OpenAI connects ChatGPT to the internet

OpenAI launched plugins for ChatGPT, extending the bot’s functionality by granting it access to third-party knowledge sources and databases, including the web. Available in alpha to ChatGPT users and developers on the waitlist , OpenAI says that it’ll initially prioritize a small number of developers and subscribers to its premium ChatGPT Plus plan before rolling out larger-scale and  API  access.

OpenAI launches GPT-4, available through ChatGPT Plus

GPT-4 is a powerful image- and text-understanding AI model from OpenAI. Released March 14, GPT-4 is available for paying ChatGPT Plus users and through a public API. Developers can sign up on a waitlist to access the API.

ChatGPT is available in Azure OpenAI service

ChatGPT is generally available through the Azure OpenAI Service , Microsoft’s fully managed, corporate-focused offering. Customers, who must already be “Microsoft managed customers and partners,” can apply here for special access .

OpenAI launches an API for ChatGPT

OpenAI makes another move toward monetization by launching a paid API for ChatGPT . Instacart, Snap (Snapchat’s parent company) and Quizlet are among its initial customers.

Microsoft launches the new Bing, with ChatGPT built in

At a press event in Redmond, Washington, Microsoft announced its long-rumored integration of OpenAI’s GPT-4 model into Bing , providing a ChatGPT-like experience within the search engine. The announcement spurred a 10x increase in new downloads for Bing globally, indicating a sizable consumer demand for new AI experiences.

Other companies beyond Microsoft joined in on the AI craze by implementing ChatGPT, including OkCupid , Kaito , Snapchat and Discord — putting the pressure on Big Tech’s AI initiatives, like Google .

OpenAI launches ChatGPT Plus, starting at $20 per month

After ChatGPT took the internet by storm, OpenAI launched a new pilot subscription plan for ChatGPT called ChatGPT Plus , aiming to monetize the technology starting at $20 per month. A month prior, OpenAI posted a waitlist for “ChatGPT Professional” as the company began to think about monetizing the chatbot.

OpenAI teases ChatGPT Professional

OpenAI said that it’s “starting to think about how to monetize ChatGPT” in an announcement on the company’s official Discord server. According to a waitlist link OpenAI posted in Discord, the monetized version will be called ChatGPT Professional . The waitlist document includes the benefits of this new paid version of the chatbot which include no “blackout” windows, no throttling and an unlimited number of messages with ChatGPT — “at least 2x the regular daily limit.”

ShareGPT lets you easily share your ChatGPT conversations

A week after ChatGPT was released into the wild , two developers — Steven Tey and Dom Eccleston — made a Chrome extension called ShareGPT to make it easier to capture and share the AI’s answers with the world.

ChatGPT first launched to the public as OpenAI quietly released GPT-3.5

GPT-3.5 broke cover with ChatGPT , a fine-tuned version of GPT-3.5 that’s essentially a general-purpose chatbot. ChatGPT can engage with a range of topics, including programming, TV scripts and scientific concepts. Writers everywhere rolled their eyes at the new technology, much like artists did with OpenAI’s DALL-E model , but the latest chat-style iteration seemingly broadened its appeal and audience.

What is ChatGPT? How does it work?

ChatGPT is a general-purpose chatbot that uses artificial intelligence to generate text after a user enters a prompt, developed by tech startup OpenAI . The chatbot uses GPT-4, a large language model that uses deep learning to produce human-like text.

When did ChatGPT get released?

November 30, 2022 is when ChatGPT was released for public use.

What is the latest version of ChatGPT?

Both the free version of ChatGPT and the paid ChatGPT Plus are regularly updated with new GPT models. The most recent model is GPT-4 .

Can I use ChatGPT for free?

There is a free version of ChatGPT that only requires a sign-in in addition to the paid version, ChatGPT Plus .

Who uses ChatGPT?

Anyone can use ChatGPT! More and more tech companies and search engines are utilizing the chatbot to automate text or quickly answer user questions/concerns.

What companies use ChatGPT?

Multiple enterprises utilize ChatGPT, although others may limit the use of the AI-powered tool .

Most recently, Microsoft announced at it’s 2023 Build conference that it is integrating it ChatGPT-based Bing experience into Windows 11. A Brooklyn-based 3D display startup Looking Glass utilizes ChatGPT to produce holograms you can communicate with by using ChatGPT.  And nonprofit organization Solana officially integrated the chatbot into its network with a ChatGPT plug-in geared toward end users to help onboard into the web3 space.

What does GPT mean in ChatGPT?

GPT stands for Generative Pre-Trained Transformer.

What’s the difference between ChatGPT and Bard?

Much like OpenAI’s ChatGPT, Bard is a chatbot that will answer questions in natural language. Google announced at its 2023 I/O event that it will soon be adding multimodal content to Bard, meaning that it can deliver answers in more than just text, responses can give you rich visuals as well. Rich visuals mean pictures for now, but later can include maps, charts and other items.

ChatGPT’s generative AI has had a longer lifespan and thus has been “learning” for a longer period of time than Bard.

What is the difference between ChatGPT and a chatbot?

A chatbot can be any software/system that holds dialogue with you/a person but doesn’t necessarily have to be AI-powered. For example, there are chatbots that are rules-based in the sense that they’ll give canned responses to questions.

ChatGPT is AI-powered and utilizes LLM technology to generate text after a prompt.

Can ChatGPT write essays?

Can chatgpt commit libel.

Due to the nature of how these models work , they don’t know or care whether something is true, only that it looks true. That’s a problem when you’re using it to do your homework, sure, but when it accuses you of a crime you didn’t commit, that may well at this point be libel.

We will see how handling troubling statements produced by ChatGPT will play out over the next few months as tech and legal experts attempt to tackle the fastest moving target in the industry.

Does ChatGPT have an app?

Yes, there is now a free ChatGPT app that is currently limited to U.S. iOS users at launch. OpenAi says an android version is “coming soon.”

What is the ChatGPT character limit?

It’s not documented anywhere that ChatGPT has a character limit. However, users have noted that there are some character limitations after around 500 words.

Does ChatGPT have an API?

Yes, it was released March 1, 2023.

What are some sample everyday uses for ChatGPT?

Everyday examples include programing, scripts, email replies, listicles, blog ideas, summarization, etc.

What are some advanced uses for ChatGPT?

Advanced use examples include debugging code, programming languages, scientific concepts, complex problem solving, etc.

How good is ChatGPT at writing code?

It depends on the nature of the program. While ChatGPT can write workable Python code, it can’t necessarily program an entire app’s worth of code. That’s because ChatGPT lacks context awareness — in other words, the generated code isn’t always appropriate for the specific context in which it’s being used.

Can you save a ChatGPT chat?

Yes. OpenAI allows users to save chats in the ChatGPT interface, stored in the sidebar of the screen. There are no built-in sharing features yet.

Are there alternatives to ChatGPT?

Yes. There are multiple AI-powered chatbot competitors such as Together , Google’s Bard and Anthropic’s Claude , and developers are creating open source alternatives . But the latter are harder — if not impossible — to run today.

The Google-owned research lab DeepMind claimed that its next LLM, will rival, or even best, OpenAI’s ChatGPT . DeepMind is using techniques from AlphaGo, DeepMind’s AI system that was the first to defeat a professional human player at the board game Go, to make a ChatGPT-rivaling chatbot called Gemini.

Apple is developing AI tools to challenge OpenAI, Google and others. The tech giant created a chatbot that some engineers are internally referring to as “Apple GPT,” but Apple has yet to determine a strategy for releasing the AI to consumers.

How does ChatGPT handle data privacy?

OpenAI has said that individuals in “certain jurisdictions” (such as the EU) can object to the processing of their personal information by its AI models by filling out  this form . This includes the ability to make requests for deletion of AI-generated references about you. Although OpenAI notes it may not grant every request since it must balance privacy requests against freedom of expression “in accordance with applicable laws”.

The web form for making a deletion of data about you request is entitled “ OpenAI Personal Data Removal Request ”.

In its privacy policy, the ChatGPT maker makes a passing acknowledgement of the objection requirements attached to relying on “legitimate interest” (LI), pointing users towards more information about requesting an opt out — when it writes: “See here  for instructions on how you can opt out of our use of your information to train our models.”

What controversies have surrounded ChatGPT?

Recently, Discord announced that it had integrated OpenAI’s technology into its bot named Clyde where two users tricked Clyde into providing them with instructions for making the illegal drug methamphetamine (meth) and the incendiary mixture napalm.

An Australian mayor has publicly announced he may sue OpenAI for defamation due to ChatGPT’s false claims that he had served time in prison for bribery. This would be the first defamation lawsuit against the text-generating service.

CNET found itself in the midst of controversy after Futurism reported the publication was publishing articles under a mysterious byline completely generated by AI. The private equity company that owns CNET, Red Ventures, was accused of using ChatGPT for SEO farming, even if the information was incorrect.

Several major school systems and colleges, including New York City Public Schools , have banned ChatGPT from their networks and devices. They claim that the AI impedes the learning process by promoting plagiarism and misinformation, a claim that not every educator agrees with .

There have also been cases of ChatGPT accusing individuals of false crimes .

Where can I find examples of ChatGPT prompts?

Several marketplaces host and provide ChatGPT prompts, either for free or for a nominal fee. One is PromptBase . Another is ChatX . More launch every day.

Can ChatGPT be detected?

Poorly. Several tools claim to detect ChatGPT-generated text, but in our tests , they’re inconsistent at best.

Are ChatGPT chats public?

No. But OpenAI recently disclosed a bug, since fixed, that exposed the titles of some users’ conversations to other people on the service.

Who owns the copyright on ChatGPT-created content or media?

The user who requested the input from ChatGPT is the copyright owner.

What lawsuits are there surrounding ChatGPT?

None specifically targeting ChatGPT. But OpenAI is involved in at least one lawsuit that has implications for AI systems trained on publicly available data, which would touch on ChatGPT.

Are there issues regarding plagiarism with ChatGPT?

Yes. Text-generating AI models like ChatGPT have a tendency to regurgitate content from their training data.

  • Reference Manager
  • Simple TEXT file

People also looked at

Perspective article, chatgpt in brazilian k-12 science education.

can chatgpt answer case study questions

  • 1 Physics Institute, University of Brasilia, Brasilia, Brazil
  • 2 Federal Institute of Rio de Janeiro, Volta Redonda, Brazil
  • 3 Center for Biological and Natural Sciences, Federal University of Acre, Rio Branco, Brazil
  • 4 Science and Technology Center, Federal University of Ceara, Fortaleza, Brazil
  • 5 Physics Institute, Federal University of Alagoas, Maceió, Brazil
  • 6 Department of Exact Science and Education, Federal University of Santa Catarina, Blumenau, Brazil

This study investigates the perceptions of K-12 science teachers regarding the implications of ChatGPT on school assessments and the quality of students’ education. A questionnaire was administered to K-12 science teachers from public and private schools across the five regions of Brazil: North, Northeast, South, Southeast, and Midwest. More than 400 teachers responded to the questionnaire. The conclusions regarding the opinions of the teachers who responded to the questionnaire were based on the Likert-type scale. The investigation covered various aspects, including the types and frequency of assessment methods used with their students. Additionally, the research delved into their opinions on whether ChatGPT would impact the quality of education and assessment methods, as well as their views on whether ChatGPT use should be deemed plagiarism or a similar infraction. The potential benefits of its use, as well as the challenges arising from it, are discussed in the context of the evolution of teaching and learning. As one of the results, it was found that among K-12 science teachers, there remains some skepticism regarding whether ChatGPT will enhance the quality of students’ education and whether its use constitutes plagiarism or a similar infraction.

1 Introduction

The recently released ChatGPT has captured the attention of academics, sparking their interest in understanding its potential applications and implications ( Chen et al., 2020 ; Firat, 2023a , b ). Integrating Artificial Intelligence (AI) and Natural Language Processing (NLP) technologies in education has the potential to start a transformative era, revolutionizing teaching and learning dynamics, unlike traditional tasks like classification and regression. Among the recent technological tools applied in education, Artificial Intelligence (AI) has emerged prominently, particularly through generative AIs like ChatGPT, Bard, and others. In a study by Tavares et al. (2020) on AI in education, they conducted an exploratory review to identify trends and research in recent years. Their findings revealed that while Intelligent Tutor Systems remain a key application of AI in education, several other AI approaches are also gaining traction in the educational landscape. This new class of generative AI models uses massive datasets and advanced algorithms to simulate human-like conversations and provide contextually relevant responses ( Brown et al., 2020 ; Okonkwo and Ade-Ibijola, 2021 ). Its ability to understand and respond to written input, engage in a conversation, and provide customized interactive assistance, can improve the autodidactic students’ autonomy and independence.

The potential of ChatGPT to provide personalized support, direction, and feedback can enhance autodidactic student motivation and engagement, improve writing, and produce formative assessments and literature reviews, as already mentioned by many authors ( Shawar and Atwell, 2007 ; Sallam, 2023 ; Firat, 2023a , b ). However, new challenges also arise, for example: How can teachers determine whether a student completed a work independently or used a generative tool? What is the acceptable limit for AI-generated content in student work before it is considered plagiarism or a similar issue? How to help students understand when the answer given by a generative AI is inaccurate?

Another important aspect is that the number of Educational Technology companies is increasing. These companies are entering the traditional education market with data-driven teaching and learning solutions. In this case, the question arises: Will some teachers be replaced by these types of generative technologies? Without a doubt, many teachers fear that technologies like GPT Chat could replace them in the future ( Alam, 2021 ; Chan, 2023 ; Chan and Tsi, 2023 ). The future of education around the world will be greatly impacted by this type of technology ( Renz and Hilbig, 2020 ; Ji et al., 2023 ; Mäkitalo et al., 2023 ).

Regarding the Brazilian case, the educational landscape reflects a diverse and multifaceted system comprising public and private schools, each addressing distinct regional characteristics and challenges. Despite the adoption of modern pedagogical approaches, traditional assessment methods have long dominated the evaluation process. In this scenario, as the significance of student assessments remains central to the effectiveness of educational practices, the exploration of innovative AI technologies (such as ChatGPT) presents a compelling avenue for potential advancements in this domain. There is a growing recognition that assessment strategies must evolve in line with contemporary learning paradigms.

Following this trend, the present study analyzes the perceptions of K-12 science teachers about the use of ChatGPT in the educational environment. The general objective is, albeit preliminarily, to outline the impacts and difficulties associated with the AI technologies integration into the educational context.

This text is organized in the following way: First, we present the methodological approach used in this study, including details about the data collection process. Next, the results are presented and discussed. Finally, drawing on the results, some possible implications and challenges for the educational process are presented.

2 Methodology

This study has a qualitative and descriptive nature. An online questionnaire was applied via Google Forms for K-12 science teachers, and over 400 of them responded. These teachers were from both public and private schools in the five regions of Brazil: North, Northeast, South, Southeast, and Midwest. The analyses of these data will be discussed in the next section of this work. At this point, it is important to explain the term “K-12 teachers”. In Brazil, this designation refers to teachers who teach in the first 12 years of basic education, covering students aged 6–18. However, it should be noted that most of the teachers who responded to the questionnaire are physics teachers and primarily worked with students aged 12–18, and teachers who teach science to students in the last two years of elementary school, for these two classes of teachers we are using the term “K-12 science teachers”. It is a crucial stage in the training of new students, preparing them for careers in science, technology, engineering, and mathematics (STEM) fields—essential for the country’s economic development. However, although this study was carried out with teachers who work in K12 science education, it is also crucial to conduct similar research among university teachers, especially those involved in teacher training, such as in undergraduate physics courses. We intend to pursue this in future research. Moreover, while Brazil’s K12 science education has distinct characteristics that may limit the generalizability of the conclusions presented here, we believe that our results can offer insights to teachers in other countries regarding the issues analyzed in this study.

The conclusions regarding the opinions of the teachers who responded to the questionnaire were based on the Likert-type scale. This scale, developed by Rensis Likert in 1932, is used to quantify opinions or attitudes. It involves creating a set of propositions related to a specific construct and then evaluating the degree of agreement among respondents ( Meireles, 2020 ). The Likert scale makes use of semantic differentials that vary qualitatively in degree, ranging from the lowest point of Completely Disagree (CD) to the highest point of Completely Agree (CA). The scale can be symmetrical, with an odd number of points, and Neutrality (N) – neither agree nor disagree—exactly between the two extremes, or asymmetrical, with an even number of points, not offering the option of neutrality and requiring respondents to choose one side ( Joshi et al., 2015 ; Monteiro, 2021 ). In our case, a 5-point Likert-type scale was used. The numerical values assigned to the answers reflect the strength and direction of the opinion of those respondents about the propositions.

The proposition agreement degree (PAD) interpretation related to our results is displayed in Table 1 . The questionnaire explored the following main points: (1) the frequency with which the teachers use homework assignments, exercise lists, oral presentations, online tests, or in-person tests as methods of assessing students, (2) the impact of ChatGPT on the quality of student education and (3) assessment methods, and (4) whether teachers believe that the use of ChatGPT constitutes plagiarism or a similar infraction. The results are displayed in Tables 2 , 3 . Although the items listed in Table 3 are not Likert-type items, similar statistical analyses were conducted, as indicated in Table 1 .

www.frontiersin.org

Table 1 . Interpretation of PAD related to the results presented in Tables 2 , 3 , based on the Likert-type scale with a 5-point format.

www.frontiersin.org

Table 2 . Analysis of the degree of agreement with propositions related to the frequency with which they use homework assignments, exercise lists, oral presentations, online tests, or in-person tests as methods of assessing students, based on the number of responses provided.

www.frontiersin.org

Table 3 . Propositions agreement degree analysis related to the number of responses provided by the teachers in response to questions about the impact of ChatGPT on the quality of student education and assessment methods, and whether teachers believe that the use of ChatGPT constitutes plagiarism or a similar infraction.

3 Results and discussions

We will now analyze the results obtained from the questionnaire applied to teachers. First, we will examine the assessment methods they use and their frequency. Then, we will assess the impact of ChatGPT on student education quality, as well as teachers’ opinions on whether its use constitutes plagiarism or a similar infraction. Furthermore, it is important to highlight that although the sample has more than 400 respondents, the generalization of results based on this number of participants must be analyzed with caution and interpreted as preliminary results. However, it has the potential as an initial working hypothesis for larger studies with more representative samples.

3.1 The assessment methods used by teachers and their frequency

Table 2 shows the teachers’ responses on the frequency of using written assignments, exercise lists, oral presentations, online tests, and in-person tests as student assessment methods. As can be seen in this table, in both public and private schools, the use of exercise lists is the assessment method most used by teachers (weekly), followed by homework assignments (monthly), and oral presentations (bimonthly or quarterly). In addition, very few of them use online tests as an assessment method. This result suggests that in Brazil, for K-12 science teachers, exercise lists stand out as one of the main ways of practicing the content studied, while homework assessments (which help develop study, research habits, and intellectual discipline) come second. However, despite the well-known importance of oral presentation assessment (that leads to the students a deeper reflection on a given topic and problem) K-12 science teachers in Brazil seem to let this type of assessment in the background.

On the other hand, it draws attention to the fact that teachers, both in public and private schools, never use online tests as an assessment method. Although K-12 science teachers in Brazil had to adapt themselves to its use during the COVID-19 pandemic, the results indicate that currently, more than 70% of them seem to have abandoned this approach. Our interpretation is that the school culture contributes significantly to teachers resisting the use of online assessments as much as possible ( AL-Takhayneh et al., 2022 ), even though it can be argued that online tests can be as effective as in-person tests for measuring course proficiency ( Fisher et al., 2022 ).

Regarding the in-person test, the results indicated a slight difference in the frequency with which these tests are applied in public (bimonthly or quarterly) and private (monthly) schools. This is probably related to the fact that Brazilian schools divide the school year into four bimesters. In this situation, private schools generally demand at least two tests for each bimester, while public schools generally demand only one. There is practically a consensus among educators on the importance of assessment tests to identify how well the student understood the content studied. However, despite this, our results indicate that almost 86% of Brazilian K-12 science teachers, who work in public schools, only apply the minimum number of tests currently required. This conduct goes against the trend advocated in several studies related to the importance of the use of formative assessments ( Boström and Palm, 2023 ). Maybe this is due to the low weekly workload that science subjects have had in Brazil in the last few years ( Moreira, 2018 ).

3.2 The impact of ChatGPT on the quality of student education and assessment methods

Another investigation carried out in this study involved collecting teachers’ opinions about the impact of ChatGPT on the quality of student education and assessment methods, as shown in Table 3 . Based on these results, it is evident that teachers have mixed opinions about the impact of ChatGPT on the quality of students’ education. Furthermore, teachers’ opinions about whether ChatGPT will enhance the quality of students’ education are also divided. Approximately half of them believe that ChatGPT will improve students’ education, while the other half think that it will not. This shows that most teachers are skeptical about the potential improvements in education that can occur using this type of generative AI. Probably because they are unaware of the great potential of this type of technology, as reflecting the opacity of AI generative models ( Laupichler et al., 2022 ).

Generative AI holds the potential to greatly benefit teachers by assisting them in various ways, including the creation of educational materials such as lesson plans, worksheets, and custom textbooks. It can also assist students by generating personalized quizzes, assignments, and study recommendations based on their strengths and weaknesses, thereby improving the effectiveness and personalization of learning. Additionally, students are leveraging ChatGPT to swiftly access answers, clarify complex concepts, and explore scientific topics beyond the standard curriculum. Furthermore, Ouyang et al. (2022) suggested that AI can provide immersive learning experiences through virtual reality technology. In addition, it is important to emphasize that the use of AI tools can significantly enhance the teaching-learning process in Professional Education, including fields such as Engineering ( Johri, 2020 ; Qadir, 2023 ), Medicine ( Feng and Shen, 2023 ; Lee, 2023 ), and Architecture ( Ceylan, 2021 ; Zhang et al., 2023 ). This underscores the importance and potential of the theme. However, it is human nature to reject what we do not understand, especially when the unknown can affect others besides ourselves, in this case, students ( Gómez Crespo et al., 2014 ; Ventouris et al., 2021 ).

Considering Brazil’s cultural, social, and economic heterogeneity, we understand that these results, although preliminary, point to the urgent need for continued training for in-service teachers. This training should encompass elements of didactic methodology, assessment, and the use of new technologies, particularly those based on generative AIs. Although this type of training already exists in Brazil, as exemplified by programs such as the Mestrado Nacional Profissional em Ensino de Física and similar initiatives ( Moreira et al., 2016 ; Ferreira et al., 2021 ; De Paulo and Moreira, 2022 ; MNPEF, 2023 ), efforts like these must not only be sustained but also improve. This is crucial to address teachers’ challenges in adapting to changes imposed by technological advances.

In the questionnaire, teachers were also asked their opinions on whether ChatGPT will affect assessment methods. The results in Table 3 showed that, in both public and private schools, most teachers have some reservations about this statement. In their opinion, assessment methods will not be significantly affected. As mentioned in the previous paragraph, generative AI tools will significantly expedite the creation of in-person tests for teachers. However, with homework assessments, such as reports, teachers will need strategies to verify whether students wrote the work themselves or used generative AI to complete it. This issue will be further discussed in the next paragraph. In summary, it is crucial that, despite these challenges, teachers remain open to the potential for developing creative methodologies with the use of ChatGPT ( Yu and Guo, 2023 ).

Another problem is whether the use of ChatGPT by students will reduce their interest in doing research in books and even on the internet since ChatGPT produces responses that closely resemble content from online sources or textbooks. In such cases, students might submit this work as their own without conducting any research or putting in any effort. In addition, it can discourage them from seeking information from a variety of sources and developing their research and critical skills. In that case, the overconfidence of the students using generative AI can lead them to a condition of being a mere spectator, harming interactive learning. Something like this can compromise personal expression and the ability to develop logical thinking, as well as creativity. Other drawbacks of ChatGPT use in schooling include the potential for fostering an overreliance on technology among students and promoting reduced social interaction among them. In light of these trends, educators must recognize and address the potential negative effects. Developing strategies to promote critical thinking, provide personalized feedback, and encourage hands-on learning experiences becomes crucial in mitigating the challenges posed by increased reliance on tools like ChatGPT.

While technologies like ChatGPT are gradually being incorporated into education, teachers must not forget that traditional methods, such as oral presentations, viva-voce examinations, practical training, and hands-on workshops, offer unique benefits. These methods foster essential skills such as public speaking, critical thinking, problem-solving, and the practical application of knowledge. They provide opportunities for real-time interaction, feedback, and experiential learning that technology cannot fully replicate. For example, Delson et al. (2022) discussed the impact of oral exams on student performance and motivation. One of the results they obtained showed that the group with the instructor increased their grade by 14%. Additionally, approximately 70% of the students who participated in the research strongly agreed or agreed that the oral exams increased their motivation to learn. Therefore, by incorporating these traditional methods into education, students develop a well-rounded skill set that prepares them for real-world challenges. While technology has its advantages, the value of in-person communication and practical experience cannot be overstated.

A worrying issue is also related to using ChatGPT as a reference tool for students during exams. How can teachers ensure that students utilize it as a “tutor” rather than simply copying the entire answer provided by ChatGPT? This appears to be a difficult issue to address. Furthermore, teachers must help their students understand that generative AI, such as ChatGPT, can generate stereotypical responses based on the training data they receive. For instance, if it is trained on a dataset with biased or racist content, its generated responses may reflect those biases unless mitigated by careful training practices.

3.3 Is the use of ChatGPT considered plagiarism or a similar infraction?

With the launch of generative AIs accessible to the public, such as GPT-3 (a previous version of the ChatGPT model introduced by OpenAI in June 2020), access to this type of technology has grown exponentially. Thus, the public has access to a technology previously present in only a few sectors of society. ChatGPT was launched very recently, in November last year. In that case, many discussions regarding the ethics of the use of this AI need to be conducted, particularly in education ( Dempere et al., 2023 ). What may be considered an ethical violation or plagiarism for some people will not be considered an ethical violation for others. Some teachers may argue that students will end up using it to obtain undue advantages, expressing knowledge or skills that they do not possess. On the other hand, other teachers may argue that ChatGPT can be an effective “tutor” for their students.

The results of the teachers’ opinions on that question, whether ChatGPT use is plagiarism or a similar infraction, are also shown in Table 3 . The opinion of teachers is quite divided; practically half consider the use of ChatGPT to be plagiarism or a similar infraction, and the other part of teachers think it is not. Particularly, teachers’ concern about the authorship of work done by their students was already present during the COVID-19 pandemic ( Muassomah et al., 2022 ), when classes were held exclusively online for several months and students completed virtually all assessments at home, remotely. Currently, the literature already presents examples of institutions that banned ChatGPT or updated their definitions of plagiarism about its use ( Cassidy, 2023 ; Soper, 2023 ).

In our opinion, generative tools, like ChatGPT, differ from plagiarism because they create “new text” rather than simply copying and pasting. The question then arises: Who is the true author of the work, the student or ChatGPT? The interaction with ChatGPT involves creating prompts, but does that alone make the user the primary author?

In our view, the novelty of this technology leaves many questions unanswered. A similar conundrum arises when using generative AI to create images. Tools like Canva, DALL-E, and Adobe Firefly raise questions about authorship. Is the AI or the person who provided the prompt considered the image’s creator? Furthermore, can such an image be deemed copyrighted? These remain open questions without a clear consensus.

The widespread availability of generative AI tools, particularly among teachers and students, has sparked inquiries that require deliberation within the educational community and society as a whole. Like the emergence of the internet and the subsequent concerns about massive data storage by companies for personalized advertisements, the use of generative AIs prompts questions about ethical boundaries and potential infractions. In the past, regulations were scarce in this regard, but with the evolution of technology, strict laws now govern how companies handle and share customer data.

We anticipate a similar trajectory for generative AIs, with the establishment of regulations in the coming years. This regulatory framework will provide clearer guidelines, helping to define ethical boundaries and delineate when the use of these tools may cross into unethical or infringing territory.

Finally, teachers can check text similarity and identify the source of original content using various online plagiarism detection tools. Some popular plagiarism detection tools include Turnitin, Grammarly, Copyscape, and Plagscan. Furthermore, teachers can also check if a text was generated by a generative AI using specialized AI detection tools designed to identify machine-generated content. Additionally, teachers can look for common indicators of AI-generated text, such as unusual word choices or combinations that are not typical of human writing, incoherent or nonsensical sentences lacking logical flow, a lack of personal voice or individual writing style, and an overuse of repetitive phrases or patterns. While it can be challenging to detect AI-generated text, especially as AI models become more advanced, these methods can help teachers identify content that may have been generated by a machine rather than a human author. In addition to using these tools, teachers can also manually check for text similarity by conducting internet searches or using search engines to find the original source of the content. They can also ask students to provide citations and references for their work to verify the originality of the content.

Although some developers are trying to create tools to find out if a text was done by generative AI and to inhibit its indiscriminate use ( Gao et al., 2022 ; Chaka, 2023 ; Originality, 2023 ), perhaps the problem is deeper and involves a broad discussion of what plagiarism essentially is or is not ( Khalil and Er, 2023 ; King and ChatGPT, 2023 ; Kleebayoon and Wiwanitkit, 2023 ), as discussed before. Furthermore, considering the inevitable integration of AI with various domains, as advocated by numerous authors ( Kelly, 2016 ; Daugeliene and Levinskiene, 2022 ), it appears both urgent and unavoidable to engage in discussions about regulatory and ethical limits for the use of AI technology within an international framework ( Sethu, 2019 ; Tong et al., 2022 ). Furthermore, to discourage students from resorting to cheating with AI tools, some authors propose the establishment of a “new educational pact” with the active participation of students. This approach encourages students to reflect on their roles in learning and society ( Fyfe, 2023 ; Sadeghinejad and Najmaei, 2023 ; Sullivan et al., 2023 ). However, there is still no consensus on the feasibility of implementing this “new educational pact.”

4 Conclusion

The study explored the perspectives of K-12 science teachers in Brazil, investigating various aspects, including the types and frequency of assessment methods used with their students. Additionally, the research delved into their opinions on whether ChatGPT would impact the quality of education and assessment methods, as well as their views on whether ChatGPT should be deemed plagiarism or a similar infraction.

The findings reveal that in both public and private schools, teachers predominantly employ exercise lists as the primary assessment method, followed by homework assignments and oral presentations. Notably, a minimal number of educators incorporate online tests into their assessment practices. The analysis of in-person tests indicates a subtle disparity in their frequency, with public schools conducting them bimonthly or quarterly, and private schools opting for a monthly schedule. A noteworthy observation is that around half of the surveyed teachers believe that ChatGPT will enhance students’ education, while an equal proportion (approximately) holds a contrary view. In addition, there is a consensus among them that assessment methods are unlikely to undergo significant changes with the integration of ChatGPT.

We also discussed the issue concerning students’ use of ChatGPT and its potential impact on their inclination to engage in traditional research methods, such as exploring books and online resources. The concern arises from ChatGPT’s capacity to generate responses closely mirroring content found in online sources or textbooks. In these instances, there is a risk that students may present this content as their own without undertaking genuine research or exerting effort.

Concerning the plagiarism implications associated with ChatGPT use, there is a notable division of opinion among teachers. Approximately half of them view the utilization of ChatGPT as constituting plagiarism or a similar infraction, while the other half disagree. This divergence underscores the need for discussion within the educational community and society at large, given the widespread accessibility of generative AI tools to both teachers and students. From our perspective, this issue prompts a broader conversation about plagiarism’s fundamental definition and boundaries in association with the use of generative AIs.

In conclusion, we hope that the results presented here, along with the accompanying discussions, will aid educators in reflecting on the implications of ChatGPT in education. Specifically, in our opinion, this work contributes to the ongoing dialogue about establishing clear standards. These standards would delineate when the use of ChatGPT by students is considered positive and provide well-defined rules to distinguish instances that may constitute plagiarism or a similar infraction.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

FM: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. PS: Conceptualization, Formal analysis, Investigation, Methodology, Writing – review & editing. MS: Conceptualization, Writing – review & editing. JM: Conceptualization, Investigation, Writing – review & editing. WS: Writing – review & editing. DG: Conceptualization, Data curation, Writing – review & editing.

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. FM acknowledges the financial support from a DPI/DPG/BCE 01/2024 grant 23106.012579/2024-56.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Alam, A. (2021). Possibilities and apprehensions in the landscape of artificial intelligence in education. In 2021 International Conference on Computational Intelligence and Computing Applications (ICCICA), IEEE, Nagpur, India (pp. 1–8).

Google Scholar

Al-Takhayneh, S. K., Karaki, W., Hasan, R. A., Chang, B. L., Shaikh, J. M., and Kanwal, W. (2022). Teachers' psychological resistance to digital innovation in Jordanian entrepreneurship and business schools: moderation of teachers' psychology and attitude toward educational technologies. Front. Psychol. 13:1004078. doi: 10.3389/fpsyg.2022.1004078

PubMed Abstract | Crossref Full Text | Google Scholar

Boström, E., and Palm, T. (2023). The effect of a formative assessment practice on student achievement in mathematics. Front. Educ. 8:1101192. doi: 10.3389/feduc.2023.1101192

Crossref Full Text | Google Scholar

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., et al. (2020). Language models are few-shot learners. Adv. Neural Inf. Proces. Syst. 33, 1877–1901.

Cassidy, C. Australian universities to return to ‘pen and paper’ exams after students caught using AI to write essays (2023). Available at: https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to-pen-and-paper-exams-after-students-caught-using-ai-to-write-essays (Accessed October 09, 2023).

Ceylan, S. (2021). Artificial intelligence in architecture: an educational perspective. In CSEDU , volume 1, 100–107.

Chaka, C. (2023). Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J. Appl. Learn. Teach. 6, 94–104. doi: 10.37074/jalt.2023.6.2.12

Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. Int. J. Educ. Technol. High. Educ. 20, 1–25. doi: 10.1186/s41239-023-00408-3

Chan, C. K. Y., and Tsi, L. H. (2023). The AI revolution in education: will AI replace or assist teachers in higher education? arXiv . 1–18.doi: 10.48550/arXiv.2305.01185

Chen, L., Chen, P., and Lin, Z. (2020). Artificial intelligence in education: a review. IEEE Access 8, 75264–75278. doi: 10.1109/ACCESS.2020.2988510

Daugeliene, R., and Levinskiene, K. (2022). Artificial intelligence in the public sector: mysticism, possibility, or inevitability. New challenges in economic and business development , University of Latvia, Riga, 90–95.

De Paulo, I. J. C., and Moreira, M. A. (2022). Disciplinas do MNPEF: implementação, obstáculos e superações no universo acadêmico. Rev. Prof. Fís. 6, 85–104. doi: 10.26512/rpf.v6i2.44625

Delson, N., Baghdadchi, S., Ghazinejad, M., Lubarda, M., Minnes, M., Phan, A., et al. (2022). Can oral exams increase student performance and motivation? Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. Available at: https://peer.asee.org/41127 (Accessed January 30, 2024).

Dempere, J., Modugu, K., Hesham, A., and Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Front. Educ. 8:1206936. doi: 10.3389/feduc.2023.1206936

Feng, S., and Shen, Y. (2023). ChatGPT and the future of medical education. Acad. Med. 98, 867–868. doi: 10.1097/ACM.0000000000005242

Ferreira, M., Sacerdote, H., Studart, N., and Silva Filho, O. L. D. (2021). Análise de temas, teorias e métodos em dissertações e produtos educacionais no MNPEF. Rev. Bras. Ensino Fís. 43:e20210322. doi: 10.1590/1806-9126-rbef-2021-0322

Firat, M. (2023a). How ChatGPT can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty, Anadolu University .

Firat, M. (2023b). What ChatGPT means for universities: perceptions of scholars and students. J. Appl. Learn. Teach. 6, 1–7. doi: 10.37074/jalt.2023.6.1.22

Fisher, D., Solomons, D., and Makhathini, K. B. (2022). Face-to-face versus online-based lectures: a COVID-19 induced study on assessments. Front. Educ. 7:1045311. doi: 10.3389/feduc.2022.1045311

Fyfe, P. (2023). How to cheat on your final paper: assigning AI for student writing. AI Soc. 38, 1395–1405. doi: 10.1007/s00146-022-01397-z

Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., et al. (2022). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digit. Med . 6, 75. doi: 10.1038/s41746-023-00819-6

Gómez Crespo, M. Á., Cañas Cortázar, A. M., Gutiérrez Julián, M. S., and Martín-Díaz, M. J. (2014). Ordenadores en el aula:¿ estamos preparados los profesores? Enseñ. Cienc. 32, 239–250. doi: 10.5565/rev/ensciencias.939

Ji, F., Zhang, X., Zhao, S., and Fang, Q. (2023). Virtual reality: a promising instrument to promote sail education. Front. Psychol. 14, 1185415. doi: 10.3389/fpsyg.2023.1185415

Johri, A. (2020). Artificial intelligence and engineering education. J. Eng. Educ. 109, 358–361. doi: 10.1002/jee.20326

Joshi, A., Kale, S., Chandel, S., and Pal, D. (2015). Likert scale. Br. J. Appl. Sci. Technol. 7:396. doi: 10.9734/BJAST/2015/14975

Kelly, K. (2016). The inevitable: Understanding the 12 technological forces that will shape our future . Penguin Publishing Group, London.

Khalil, M., and Er, E. (2023). “Will ChatGPT Get You Caught? Rethinking of Plagiarism Detection.” in: Learning and Collaboration Technologies. HCII 2023. Lecture Notes in Computer Science, vol 14040 . Eds. P. Zaphiris and A. Ioannou (Springer, Cham). doi: 10.1007/978-3-031-34411-4_32

King, M. R.ChatGPT (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher education. Cell. Mol. Bioeng. 16, 1–2. doi: 10.1007/s12195-022-00754-8

Kleebayoon, A., and Wiwanitkit, V. (2023). Artificial intelligence, chatbots, plagiarism and basic honesty: comment. Cell. Mol. Bioeng. 16, 173–174. doi: 10.1007/s12195-023-00759-x

Laupichler, M. C., Aster, A., Schirch, J., and Raupach, T. (2022). Artificial intelligence literacy in higher and adult education: a scoping literature review. Comput. Educ.: Artif. Intell. 3:100101. doi: 10.1016/j.caeai.2022.100101

Lee, H. (2023). The rise of ChatGPT: exploring its potential in medical education. Anat. Sci. Educ. 0, 1–6. doi: 10.1002/ase.2270

Mäkitalo, K., Laru, J., and Dagienë, V. (2023). Looking ahead: computational thinking in K12 education. Front. Educ. 8:1248509. doi: 10.3389/feduc.2023.1248509

Meireles, M. (2020). Análise de escala Likert pelo Estocástico de Wilder. Rev. Micro Pequena Empresa 14, 1–2. doi: 10.21714/1982-2537/2020v14n1p12

MNPEF . (2023). Mestrado Nacional Profissional em Ensino de Física. Available at: http://www1.fisica.org.br/mnpef/ (Accessed October 09, 2023).

Monteiro, F. F. (2021). Análise de uma experiência híbrida no ensino de Física 1. Rev. Bras. Ensino Fís. 43, 1–10. doi: 10.1590/1806-9126-rbef-2020-0315

Moreira, M. A. (2018). Uma análise crítica do ensino de Física. Estud. Avançados 32, 73–80. doi: 10.1590/s0103-40142018.3294.0006

Moreira, M. A., Studart, N., and Vianna, D. M. (2016). O mestrado nacional profissional em ensino de física (MNPEF): uma experiência em larga escala no Brasil. Lat. Am. J. Phys. Educ. 10:26.

Muassomah, M., Abdullah, I., Hasanah, U., Dalmeri, D., Sihombing, A. A., and Rodrigo, L. (2022). The academic demoralization of students in online learning during the COVID-19 pandemic. Front. Educ. 7:888393. doi: 10.3389/feduc.2022.888393

Okonkwo, C. W., and Ade-Ibijola, A. (2021). Chatbots applications in education: a systematic review. Comput. Educ.: Artif. Intell. 2:100033. doi: 10.1016/j.caeai.2021.100033

Originality . AI – Plagiarism checker and AI detector (2023). Available at: https://originality.ai/#about (Accessed October 09, 2023).

Ouyang, F., Zheng, L., and Jiao, P. (2022). Artificial intelligence in online higher education: a systematic review of empirical research from 2011 to 2020. Educ. Inf. Technol. 27, 7893–7925. doi: 10.1007/s10639-022-10925-9

Qadir, J. (2023). Engineering education in the era of ChatGPT: promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON) , IEEE, Kuwait, Kuwait, pp. 1–9.

Renz, A., and Hilbig, R. (2020). Prerequisites for artificial intelligence in further education: identification of drivers, barriers, and business models of educational technology companies. Int. J. Educ. Technol. High. Educ. 17, 1–21. doi: 10.1186/s41239-020-00193-3

Sadeghinejad, Z., and Najmaei, A. (2023). From chat GPT to cheat GPT: Call for new AI-empowered teaching models.

Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare 11:887. doi: 10.3390/healthcare11060887

Sethu, S. G. (2019). The inevitability of an international regulatory framework for artificial intelligence. In 2019 international conference on automation, computational and technology management (ICACTM), IEEE, London, UK, pp. 367–372.

Shawar, B. A., and Atwell, E. (2007). Chatbots: are they really useful? J. Lang. Technol. Comput. Linguist. 22, 29–49. doi: 10.21248/jlcl.22.2007.88

Soper, T. (2023). Seattle Public Schools bans ChatGPT; district ‘requires original thought and work from students. GeekWire. Available at: https://www.mentalandhealthawareness.com/featured-articles/seattle-public-schools-is-joining-a-growing-number-of-school-districts-banning-chatgpt#:~:text=SPS%20blocked%20ChatGPT%20on%20all%20school%20devices%20in,original%20thought%20and%20work%20from%20students%2C%E2%80%9D%20he%20said (Accessed October 09, 2023).

Sullivan, M., Kelly, A., and McLaughlan, P. (2023). ChatGPT in higher education: considerations for academic integrity and student learning. J. Appl. Learn. Teach. 6, 1–11. doi: 10.37074/jalt.2023.6.1.17

Tavares, L. A., Meira, M. C., and Amaral, S. F. (2020). Inteligência Artificial na Educação: survey. Braz. J. Dev. 6, 48699–48714. doi: 10.34117/bjdv6n7-496

Tong, L., Yan, W., and Manta, O. (2022). Artificial intelligence influences intelligent automation in tourism: a mediating role of internet of things and environmental, social, and governance investment. Front. Environ. Sci. 10:853302. doi: 10.3389/fenvs.2022.853302

Ventouris, A., Panourgia, C., and Hodge, S. (2021). Teachers’ perceptions of the impact of technology on children and young people's emotions and behaviours. Int. J. Educ. Res. Open 2:100081. doi: 10.1016/j.ijedro.2021.100081

Yu, H., and Guo, Y. (2023). Generative artificial intelligence empowers educational reform: current status, issues, and prospects. Front. Educ. 8:1183162. doi: 10.3389/feduc.2023.1183162

Zhang, Z., Fort, J. M., and Mateu, L. G. (2023). Exploring the potential of artificial intelligence as a tool for architectural design: a perception study using Gaudí’s works. Buildings 13:1863. doi: 10.3390/buildings13071863

Keywords: ChatGPT, K-12 teachers, science education, assessment methods, teaching and learning

Citation: Monteiro FF, Souza PVS, da Silva MC, Maia JR, Silva WF and Girardi D (2024) ChatGPT in Brazilian K-12 science education. Front. Educ . 9:1321547. doi: 10.3389/feduc.2024.1321547

Received: 14 October 2023; Accepted: 05 February 2024; Published: 28 February 2024.

Reviewed by:

Copyright © 2024 Monteiro, Souza, da Silva, Maia, Silva and Girardi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Fábio Ferreira Monteiro, [email protected]

† These authors share first authorship

  • Services & Software

Here's How to Get Started With ChatGPT

Want to use ChatGPT but not quite sure where to start? Check out these tips.

can chatgpt answer case study questions

Learning to use AI chatbots effectively is more important than ever. 

If 2023 was the year of AI, then it's looking like  2024 will carry on the trend . You may not have tried out one of the many AI chatbots that hit the scene over the last year, but you've probably heard a thing or two about them. How could you not?  OpenAI's ChatGPT gained 100 million users in just two months . It took Instagram and TikTok two and a half years and nine months, respectively, to hit that same mark. 

CNET Tech Tips logo

The field of AI chatbots and other generative AI tools is expansive and growing. A short list includes Google Gemini, Microsoft Copilot, Claude.ai, Perplexity, Dall-E and Midjourney, and collectively they're poised to transform how you work, how you get information and how companies do business. But it all got started with ChatGPT.

Learning to use and navigate using AI chatbots will be increasingly important this year, and if one of your goals is to get more comfortable with AI, you're in the right place. Here's what you need to know about getting started with ChatGPT. 

For more, here's an in-depth look at ChatGPT and how AI is changing the way we answer our health questions . 

What is ChatGPT?

ChatGPT is an artificial intelligence service that simulates human conversation to attempt to understand your text-based questions and requests and then respond to them. To accomplish this task, the chatbot is pretrained on large data sets (including public webpages) to look for patterns and create understandable and appropriate responses based on your requests or prompts. (The "GPT" part of the name reflects that and is short for "generative pretraining transformer.")

If you really want to dig into what ChatGPT is, we have a handy ChatGPT glossary that can help you make sense of it all.

Before your first ChatGPT prompt…

You can't start asking ChatGPT for help until you make an account with OpenAI. 

Navigate to ChatGPT's login page , and you'll be met with the options to Log In or Sign Up. Tap the Sign Up button and provide your name, email address and a valid phone number to verify your account. Note you'll have to accept a couple of disclaimers from OpenAI before you can start prompting ChatGPT. 

Tweak some ChatGPT settings

After creating your account and accepting OpenAI's disclaimers, you should do a few more things before asking all of your questions. First, I recommend tinkering with a few ChatGPT settings. There aren't too many settings, but the more familiar you are with the options, the better. 

screenshot of ChatGPT's custom instruction screen

Custom instructions let you give ChatGPT some parameters to work with when answering your prompts -- at least it's supposed to. 

The settings that I suggest adjusting are the Custom instructions , where you can give ChatGPT instructions and parameters for how to answer your prompts. These instructions can include information about you, how long or short responses should be, what your hobbies and interests are, and other specifying information. For instance, I told ChatGPT that I was a big fan of Taylor Swift and Formula 1. But when I would ask what music I should try listening to, or what sports I should try watching, this information did not seem to impact the responses I got. 

Before your first prompt, define your 'why' 

Now that you're set up, you're ready to ask your first question. Before you start typing your prompt, however, stop and think about what you really want out of your query. 

You can ask ChatGPT to do all sorts of things for you, but the more focused you are when you're drafting your prompt, the more focused your answer will be. This practice, commonly referred to as prompt engineering , is important if you want useful answers from ChatGPT. 

Ready, set, prompt

Prompting ChatGPT itself is simple. On ChatGPT's homepage, you will see a bar labeled "Message ChatGPT…" at the bottom of the page. When you have a prompt in mind, simply type it into that bar. 

After you've submitted your prompt, ChatGPT will "think" for a moment, and then return a response to your question. At the bottom of ChatGPT's response, you will see four symbols. A small clipboard, a thumbs up, a thumbs down and a circular arrow. 

The small clipboard allows you to copy ChatGPT's response, the thumbs up and thumbs down allow you to tell ChatGPT if its response is good or bad, and the circular arrow will tell ChatGPT to regenerate its response. Note: Regenerating the response will yield a different response than you got the first time. 

After you prompt…

Even after ChatGPT yields its response, your job isn't quite done. There are still a couple of best practices to consider before putting ChatGPT's response into action. 

Be aware of hallucination : Even if you have crafted a near-perfect prompt, ChatGPT may "hallucinate" in its answer. In the context of AI, hallucination refers to the chatbot returning incorrect and nonfactual information. This is not unique to ChatGPT and can affect other chatbots as well. 

It's wise to double-check any advice you receive, and that's especially true when it's coming from a chatbot such as ChatGPT. 

And that goes for questions around health and self-diagnosis . If you ask ChatGPT if women can use Rogaine (a hair-loss medication with the active ingredient minoxidil), for example, the chatbot may tell you that women can use Rogaine but only the women's formulations. But this isn't necessarily true. A 2017 study found that there is no chemical difference between men's and women's Rogaine . The only difference is the price, which can be 40% more for the women's version. When I was dealing with some hair loss, my dermatologist recommended me to opt for men's Rogaine to save a bit of money. 

Sometimes ChatGPT gets it right. Usually it's close. Once in a while, it's very wrong. Check ChatGPT's answers with a Google search at the very least and, if possible, confirm the answer with professionals. 

The answer is just… bad

I go into this much more in my guide on how to write prompts , but ChatGPT is by no means perfect. It will most likely give you an answer that you're not quite satisfied with. Luckily, it's easy to remedy this issue. 

First, make sure you're asking a detailed question: The more usable information you put in your prompt, the better chance that you get back a usable answer. (You should be willing to go back and refine your prompt if you're not getting the answers you expected.)

Second, ask clarifying follow-up questions. I've found that this produces mixed results. It can sometimes prompt ChatGPT to refine its answer, but it can also just return more garbage. 

Finally, if it feels like you're heading down the wrong path, start a new chat and ask your question again. Note that if you do start over or delete your chat history, your prompts are still on OpenAI's servers.

Does ChatGPT remember what I've asked it?

Your previous prompts will not affect the answers that ChatGPT gives you. I asked ChatGPT to help me brainstorm ideas for a birthday gift for my brother and gave information on his interests in my initial prompt. If I wanted to ask another question about my brother's birthday later, I would need to re-enter the information into ChatGPT. 

This won't be the case for long, though. On Feb. 13, OpenAI announced that it's testing ways for ChatGPT to remember details you've discussed in earlier chats when you create new prompts. OpenAI said it was rolling out the features to a "small portion of ChatGPT free and Plus users this week" and will share plans for a larger rollout soon. If you have access to this new feature and you don't want ChatGPT to remember your history, you can disable it by going to Settings , selecting the Personalization tab and then toggling Memory off. 

For more on AI, here's what to know about ChatGPT's new memory function and whether or not ChatGPT is helpful when it comes to wooing your loved one.

Editors' note: CNET is using an AI engine to help create some stories. For more, see  this post .

Services and Software Guides

  • Best iPhone VPN
  • Best Free VPN
  • Best Android VPN
  • Best Mac VPN
  • Best Mobile VPN
  • Best VPN for Firestick
  • Best VPN for Windows
  • Fastest VPN
  • Best Cheap VPN
  • Best Password Manager
  • Best Antivirus
  • Best Identity Theft Protection
  • Best LastPass Alternative
  • Best Live TV Streaming Service
  • Best Streaming Service
  • Best Free TV Streaming Service
  • Best Music Streaming Services
  • Best Web Hosting
  • Best Minecraft Server Hosting
  • Best Website Builder
  • Best Dating Sites
  • Best Language Learning Apps
  • Best Weather App
  • Best Stargazing Apps
  • Best Cloud Storage
  • Best Resume Writing Services
  • New Coverage on Operating Systems
  • Hostinger Coupon Codes
  • HR&R Block Coupons
  • ShutterStock Coupon Codes
  • FedEx Office Coupons
  • Coursera Coupons
  • Autodesk Coupons
  • Codeacademy Coupon Codes
  • iolo Techologies Coupons
  • Norton Coupon Codes

IMAGES

  1. 7 Biggest Questions About ChatGPT, Answered by Gartner Experts

    can chatgpt answer case study questions

  2. 10 Questions to ChatGPT: How It Can Change Cybersecurity

    can chatgpt answer case study questions

  3. Can ChatGPT answer grammar question ??

    can chatgpt answer case study questions

  4. Chatgpt From Openai Is A Huge Step Toward Usable Answer Engine How Does

    can chatgpt answer case study questions

  5. PDF Document Question Answering with ChatGPT #demo

    can chatgpt answer case study questions

  6. ChatGPT tutorial: ChatGPT

    can chatgpt answer case study questions

VIDEO

  1. Teach Talks Session 39

  2. A Special Lecture on "ChatGPT and Academic Writing"

  3. ChatGPT-4 Takes AP EXAMS 😳

  4. अनुबंध अधिनियम प्रश्न उत्तर केस स्टडी (समस्या) Contract Act Question Answer Case Study

  5. Can ChatGPT replace creative writing?

  6. Simplify journal articles & research papers with these ChatGPT prompts

COMMENTS

  1. Solving for a Harvard Case Competition Using ChatGPT

    The real question is whether ChatGPT can create full-length studies if given enough data, queries, and time to do so. It won't be easy, and the efforts and prompt engineering required to get it ...

  2. Elevate Your Case Prep with ChatGPT

    Elevate Your Case Prep with ChatGPT. 5 Ways AI Can Help You Ask Better Questions and Plan More Thoughtful Class Discussions. by Mitchell Weiss. August 3, 2023. G reat case method teaching is, in many ways, much like prompt engineering for generative AI. You pose a query. You provide some guidance for the type of response you are expecting.

  3. Evaluating Academic Answers Generated Using ChatGPT

    The correct answer is "three", and yet the answers generated by ChatGPT were "one", "four", and "five". In question 1, in the year 2 assessment, the ChatGPT generated responses that outlined organic reaction schemes in text only. Using text only is not the typical format in representing organic chemistry.

  4. How to use ChatGPT in your studies

    ChatGPT result: The text is describing a time period where there are both good and bad aspects, and different beliefs and opinions exist. It's a time of both hope and despair. Multiple-choice questions. You can use ChatGPT to create multiple-choice questions for your study material. This can help you test your level of knowledge.

  5. Using ChatGPT for creating more compelling content for case study outlines

    Additionally, ChatGPT can be used to streamline the content creation process for case studies. By using it to generate text for various sections of the case study, such as the background, methods, results, and conclusion, you can save time and focus on editing and fine-tuning the content. Furthermore, ChatGPT can also assist in summarizing the ...

  6. Empirical assessment of ChatGPT's answering capabilities in ...

    Our study suggests one key takeaway: The answers from ChatGPT are indeed rated sufficiently correct to assist students of Bachelor and Master levels on a wide range of questions in natural science ...

  7. Using ChatGPT for Assignments

    It's important to check that the outline makes sense and covers all the points needed to answer your research question. You should also ensure that the headings and subheadings follow a logical structure. Scribbr's Proofreading & Editing service can help you with this. Example: Creating an outline with ChatGPT.

  8. Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of

    ChatGPT is a powerful large language model (LLM) that covers knowledge resources such as Wikipedia and supports natural language question answering using its own knowledge. Therefore, there is growing interest in exploring whether ChatGPT can replace traditional knowledge-based question answering (KBQA) models. Although there have been some works analyzing the question answering performance of ...

  9. What if the devil is my guardian angel: ChatGPT as a case study of

    To answer the aforementioned research question, this study adopts a qualitative case study approach (Yin, 1984) and benefits from an instrumental case study research design (Stake, 1995).Instrumental research design is helpful when researchers intend to understand a phenomenon in a context (Stake, 1995), which is in our case, ChatGPT which is a fine and recent example of AI-powered chatbots.

  10. Evaluating ChatGPT as a Question Answering System: A Comprehensive

    We give questions that we are sure have no answer to ChatGPT, and if answers are returned for those questions, it indicates a hallucination. If the text contains an answer to the asked question, it should return the answer; otherwise, it should return 'None.' In contrast, all SQuAD 1.1 questions can be found within the paragraphs provided.

  11. Adventures with ChatGPT 4

    My first (of many) prompts was to "write a 500-word customer case study using this interview transcript.". For comparison, you can see the original transcript, as well as the finished product (no formatting, just copy). I learned, when working with different interview transcripts, that GPT 4 has a prompt limit of 2,048 characters.

  12. Study Finds ChatGPT Outperforms Physicians in High-Quality, Empathetic

    The study compared written responses from physicians and those from ChatGPT to real-world health questions. A panel of licensed healthcare professionals preferred ChatGPT's responses 79% of the time and rated ChatGPT's responses as higher quality and more empathetic. "The opportunities for improving healthcare with AI are massive," said ...

  13. How to use ChatGPT to summarize a book, article, or research paper

    1. Find your article, paper, or book to summarize. If you need ChatGPT to help summarize an article or research paper, find the body of text online and keep it open in a separate tab. 2. Open your ...

  14. Using ChatGPT to write exam questions

    ChatGPT can answer exam questions, but can it write them? In this case study, Justin Shaffer explores ChatGPT's ability to write multiple choice exam questions.

  15. ChatGPT and generative AI chatbots: challenges and opportunities for

    Challenges of fraud and plagiarism. The challenges of fraud and plagiarism are particularly concerning in the educational setting. ChatGPT can now write essay answers to exam questions that appear to be well-researched and referenced, leading to some academics declaring the end of essays as a form of assessment due to the potential for plagiarism that can be difficult to detect.3

  16. AI Unreliable Answers: A Case Study on ChatGPT

    This case study's primary objective was to identify its strengths and limitations and to evaluate the effect of ChatGPT's reliability on user satisfaction. The findings of the study revealed that despite the fact that the chatbot occasionally produced incorrect responses, most users still are satisfied of it and enjoyed interacting with it.

  17. The doctor is out, but it's OK. ChatGPT can answer your questions

    The panel of health care professional evaluators preferred ChatGPT responses to physician responses almost 80% of the time. "ChatGPT messages responded with nuanced and accurate information that often addressed more aspects of the patient's questions than physician responses," said study co-author Jessica Kelley, a nurse practitioner with San ...

  18. Can ChatGPT answer your clients' questions?

    These tools are based on enormous datasets. ChatGPT can answer technical tax questions, draft letters and responses, generate contracts, and suggest checklists. It can also answer CPA Exam questions, produce social media posts, and "brainstorm" ideas you can use as a springboard. It can be an amazing time saver.

  19. A Case Study on ChatGPT Question Generation

    The experiments show that appropriate context through the authors' designed prompts makes ChatGPT an appropriate tool for accurately performing the QG task, and demonstrate that ChatGPT can potentially be a strong QG accuracy evaluator comparable to human evaluators. The advent of transformers and the subsequent development of Large Language Models (LLMs) based on these technologies has ...

  20. Can ChatGPT Help in Electronics Research and Development? A Case Study

    To the question about the origins of the suggestions, ChatGPT responded with a sound answer: ChatGPT: "I wrote the code example based on my knowledge and experience with using the OneWire and DallasTemperature libraries to read temperature data from a DS18B20 sensor. However, the code structure and some of the function calls may be similar to ...

  21. How Does ChatGPT Answer Fair Use Questions?

    One strength of ChatGPT's response to fair use questions is its ability to provide questioners with general information about the principles of fair use. This can be helpful for those who are unfamiliar with the legal doctrine and want to learn more about it. ChatGPT can also provide examples of how fair use has been applied in specific cases.

  22. Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions

    For example, a Spanish study 36, evaluated ChatGPT's ability to answer questions from the 2022 MIR exam. In this cross-sectional and descriptive analysis, 210 questions from the exam were ...

  23. PDF Is Stack Overflow Obsolete? An Empirical Study of the Characteristics

    ChatGPT's answers to programming questions. To bridge the gap, we conducted the first in-depth analysis of ChatGPT answers to 517 programming questions on Stack Overflow and examined the correctness, consistency, comprehensiveness, and conciseness of ChatGPT answers. Furthermore, we conducted a large-scale lin-

  24. Automating Academic Document Analysis with ChatGPT: A Mendeley Case

    The management and organization of a large collection of academic documents is an important part of scientific research. This study explores the use of ChatGPT, a large language model from OpenAI, to extract insights from a large collection of academic documents stored in Mendeley. The study found that ChatGPT can be used to generate insightful graphs, concise summaries, and other tasks ...

  25. ChatGPT: Everything you need to know about the AI chatbot

    ChatGPT users can now invoke GPTs directly in chats. Paid users of ChatGPT can now bring GPTs into a conversation by typing "@" and selecting a GPT from the list. The chosen GPT will have an ...

  26. Can ChatGPT Help Manage Gynecologic Cancers?

    Of the 40 questions, ChatGPT provided correct and comprehensive answers to 33 (82.5%), correct but not comprehensive answers to six (15%), and partially incorrect answers to one (2.5%) question.

  27. Frontiers

    This study investigates the perceptions of K-12 science teachers regarding the implications of ChatGPT on school assessments and the quality of students' education. A questionnaire was administered to K-12 science teachers from public and private schools across the five regions of Brazil: North, Northeast, South, Southeast, and Midwest. More than 400 teachers responded to the questionnaire.

  28. Navigating the ethical landscape behind ChatGPT

    In the case of GPT-3, ChatGPT's predecessor, nearly 93% of its training data was in English, followed by major European languages such as French, German, and Spanish, while other widely spoken languages rooted in the Global South, like Chinese and Hindi, are very much underrepresented in the dataset despite their population size (Brown et al., 2020).

  29. ChatGPT struggles to answer medical questions, new research finds

    ChatGPT might not be a cure-all for answers to medical questions, a new study suggests. Researchers at Long Island University posed 39 medication-related queries to the free version of the ...

  30. Here's How to Get Started With ChatGPT

    It can sometimes prompt ChatGPT to refine its answer, but it can also just return more garbage. Finally, if it feels like you're heading down the wrong path, start a new chat and ask your question ...