ux research literature review

Quick Lit Reviews Reduce UX Research Time and Supercharge Your Design

A quick and dirty literature review (Lit Review) is a way to capture and synthesize information about a topic (a design problem, a new technology, an unfamiliar business area, etc.). It’s a simple structure that will allow you to document relevant information in an organized and intentional format. Creating the Lit Review can take a relatively short time compared with formal UX research; but leaves you with a lasting resource that can organize your thoughts, inform your strategy, educate others, and positively influence team behavior and design.

What is a Literature Review?

You may have been exposed to a Lit Review in school as a part of undergraduate or graduate work. Lit Reviews are often performed in preparation for a master’s thesis, doctoral dissertation, or when writing journal articles (“Literature review,” 2019). A Lit Review is a survey of the available published information on a particular topic. A simple review can be composed of just a summary of sources but often includes an overview of the information available and a synthesis of the major findings (The Writing Center, n.d.).

When most people think of a Lit Review they associate it with the highly rigorous, complex, and time consuming Systematic Review and Meta-Analysis. This type is familiar because it is often referenced in journal articles and is performed by graduate students and academic researchers. It includes an exhaustive review of scholarly papers and recent research and an assessment of the search results to offset bias and ensure all relevant research is included. It then uses qualitative and quantitative methods to synthesize findings and has strict rules for the structuring of results (Pare & Kisiou, 2017; Uman, 2011; Venebio, 2017). The average time to conduct a Systematic Review is 1,139 hours (J Med Libr Assoc, 2018)—hardly practical for UX!

What people don’t realize is that the format of the Lit Review can be modified for different fields of study and by purpose. The simple Narrative Review provides a broad perspective on a topic and can be produced quickly and cheaply. It can be performed in mere hours, allows authors to select the material that interests them, ignores selection bias, and permits simple thematic or content analysis (Pare & Kisiou, 2017; The Writing Center, n.d.).

What is a Quick & Dirty Lit Review?

A Quick and Dirty Lit Review (Q&D Lit Review) is a Narrative Review that does not concern itself with formatting for final presentation, liberally uses copy and paste to capture useful information, and —most importantly — leverages qualitative coding techniques to analyze information as it is collected. In business we don’t have the time or budget for deep rigor, long analysis, or well-written prose; but we can still benefit from capturing information from multiple sources for analysis, reuse, and dissemination.

The Q&D Lit Review is also broadened to include non-peer reviewed work and other, non-published work. Often, in business our specific problem may not be supported by an existing body of research so information must be acquired from other sources such as informal, online articles, development forums, social media, talking with colleagues, user interviews, etc. Capturing these other, less reputable sources allows us to consider and incorporate the newest information and trends, while qualitative coding techniques allow us to easily compare themes across sources and quickly compare the value of new ideas against older, tested ones.

When to do a Q&D Lit Review?

A Lit Review can be performed any time you want to quickly get up to speed on a topic. However it is not a replacement for deeper, more rigorous research. Think of it as the first step in your UX research strategy. The Lit Review should bring your UX Research needs into focus. It is ideal when you don’t yet know the questions to ask, or when you want to know what you don’t know. Expect more focused questions to arise out of your initial Lit Review.

How to perform a Q&D Lit Review

A Q&D Lit Review follows the six basic steps of all Lit Reviews (Pare & Kisiou, 2017); but to save time and increase efficiency, steps 3, 4, 5 & 6 are done concurrently:

• Formulate your research question
• Search the literature
• Screen for the material you want to include
• Assess the quality of what you are including
• Extract the data
• Analyze the data

Figure 1 The Quick and Dirty Lit Review is structured for speed and efficiency. The six basic steps of the Narrative Review are condensed to shorten data collection and coding time.

Formulate Your Research Question & Set-Up (15-20 min)

The first step in performing a Q&D Lit Review is to consider what you are researching and formulate a clear research question. This may seem like a trivial step but clearly formulating a research question will keep you focused and guide the rest of your actions (McCombes, 2020). At this stage your research question may be very broad. Some example questions from my own experiences include:

• What should I consider when designing a Log On screen?
• How will the transition to WCAG2.1 affect accessibly testing and accessible design?
• How can I make Tableau as accessible as possible?
• What is the best way to collect user feedback on a Drupal site page?

Often I find that the process of articulating the question yields keywords or additional sub-questions that I will use later. It also gives me a start on my inductive code set.

Note: To get an introduction to developing codes and coding qualitative data read Themes Don’t Just Emerge — Coding the Qualitative Data (Yi, 2018).

At this stage you must also set up your code book (the document where you ‘code’ your data). I like to use a table in Word because it’s easy to copy and paste into, it allows me to add formatting (bold and bullets) to my text, yet still retains a tabular format that makes it easy to sort and filter codes or sources and reorganize data rows. At a minimum, your code book should have three columns: Codes, Data, and source URLs. You may choose additional rows if you want primary and secondary codes, or if you want to easily track source type (i.e. journals, news, social media, interview, etc.), or the keywords you used to find the content.

Search the Literature (30-60 seconds per source)

Information can be acquired from any source: online magazines and journals, informal online posts, online training, development forums, social media, prior usability testing transcripts, impromptu interviews with colleagues or clients, office memos, competitor websites, etc. Printed material is also useful, but you may want to scan it to reduce keyboarding time, or be prepared to summarize the text. I have a shelf with a number of UX and software development books that I like to thumb through and extract ideas from before I begin my online search.

The broader your search is the more comprehensive your review will be; and more comprehensive equals more time. Don’t lose sight of the fact that this is supposed to be quick! If you’re short on time, limit yourself to 30 or 60 minutes. If you have more time, continue searching and reviewing sources until you see the core ideas and guidance repeating.

Screen, Assess, Extract, & Analyze (5-10 min per source)

For each article (or post, interview transcript, etc.) you find, skim for content relevant to your research question. As you see relevant ideas or concepts copy and paste them into your code book. Your codes can be words or phrases, whatever helps you organize the information.

You can also add your own commentary to the cell. I notate the data with my thoughts and questions as they occur. I’ll italicize that text so I can quickly review it later. My notes may lead me to search for additional information, or simply help me interpret the text and recall more valuable information.

Figure 2 Illustration of a code book used to answer the question “What should I consider when designing a Log On screen?” Other codes appearing in the book are also displayed.

Visuals are a major part of UX. If you see a great design pattern or illustration of ideas, take a screenshot and add it to an appendix below the table. Use image captions to briefly summarize its importance and capture the source URL.

As you cut, paste, and organize content you’ll start to see similarities between articles. You may see the same phrase or guidance repeated (sometimes often enough to suspect plagiarism). Occasionally you’ll see content that directly contradicts other guidance. This may cause you to review previous articles and re-examine their statements. You’ll find that you’re reading articles from a more analytical perspective than you would be if you were not coding the data.

As you add sources, continue to organize and re-order the code book so that similar ideas are grouped together. Create theme statements as they occur to you. Merge cells that contain very similar ideas, so that one theme represents ideas repeated by different sources. Combining screening, assessment and extraction with analysis as you read allows you to quickly synthesize and internalize the information.

If a source lacks valuable information, copy the URL to the bottom of your table and provide a short sentence to summarize the article for yourself and why you did not extract information from it. Provide a code like “No Info” so you can sort them out. This will allow you to capture the full breadth of your research effort. It may also prove useful if, as your research develops, you realize that you may have overlooked something valuable and you want to reread a source, or if a source has very basic information that you later realize may be valuable to junior team members. It is also a useful way to keep yourself on task. If you’re not copying valuable information into your code book you may not be reading the articles you should be reading, you may be falling victim to distraction and click-bait. Keeping yourself honest is a good way to conserve and manage your time.

Final Analysis & Report Out (5-60 min)

Once you’ve used the time you have, or once you start to see information repeating, it’s time to stop searching and start reviewing what you’ve collected. At this point themes and high level conclusions will be evident. Skim your entire code book to see if anything new jumps out when you look at the full data set. Occasionally, key guidance is not exciting enough to draw your attention; but when you see it repeated several times you realize its importance. Incorporate these late stage thoughts into your theme statements and conclusions.

Figure 3. This sample code book for a Login page redesign resulted in a list of best practices, design heuristics, and common issues which helped drive requirements and design. It also facilitated a deep partnership with the security team to balance ease of access with data-security concerns. Total sources: 8. Research time: 2 hours.

Review all your themes, conclusions, and notes to ensure that they are written in a manner that is meaningful to others. Create full and complete thoughts that summarize what you’ve learned and relate it to actions, behaviors, or processes that can be performed to solve your research problem. This is important for several reasons. First, it forces you to think reflexively. Reflective thinking is critical to complex problem-solving; it forces you to step back and think about how to solve a problem and how a set of problem-solving strategies can be leveraged to achieve a goal. (University of Hawaii, n.d.) Secondly, much of the value of the Lit Review is in its ability to quickly transfer information to others. If your thoughts are not clear and instructive, you cannot transfer knowledge. Finally, projects may be delayed or compete with other priorities. If you must revisit a project in six months, or if you have to balance multiple projects, you want your research to be meaningful to you.

When you do share your review, you may need to reorganize it so it tells a cohesive story for new readers. Depending on your audience, you can simply add a Table of Images to display the screen shots you’ve assembled. Or, if you plan to share your report with a client, you may want to convert your findings into a more narrative format as well as enter full citations for your sources.

As a beginner, expect to spend at least four hours to a day, on your first Lit Review. Your reading speed will affect your time. (I took a course in speed reading years ago and that allows me to skim many articles and quickly make value judgments. I then slowly re-read the material that I believe has value for my research question.) It takes time to integrate valuable information from various sources and you may need additional time to revisit and compare articles. If you are new to qualitative coding, expect a learning curve. It can be difficult to discern the correct code-set for your research problem if you are not a seasoned coder. Consider learning more about qualitative coding before you begin.

Top 10 Reasons & Tips for a Quick and Dirty Lit Review

You’re likely doing the research already.

To stay abreast of current design trends, technology innovations, and accessibility guidelines it’s likely you already read a great many UX articles, attend conferences or trainings, and network with other UX professionals. In other words, you’re already reviewing the “literature”; you’re just not documenting it in a way that makes it useful to you. If you’ve ever found yourself thinking “where did I see that?” or “what are the best practices?” in response to a design problem or question, then the structure of the Lit Review will help you.

Keep focused when researching online

We’ve all had the experience of reading an article online then getting distracted by click bait. Suddenly you’ve wasted an hour and have nothing to show for it. The Lit Review keeps you focused on drilling into a very specific topic. If you’re not cutting and pasting into the document, then you’re not reading relevant content and you have to move on.

Quickly identify patterns and contradictions

As you cut, paste, and organize content you’ll start to see similarities and contradictions between articles. This will cause you to review previous articles and re-examine their statements. You’ll find that you’re reading articles from a more analytical perspective.

Citations matter

When engaging with a client, design or development team, disagreements are bound to arise. Your research will support your ideas and provide persuasive justifications for design or process decisions. It’s not just you saying how it should be done; it’s coming from numerous well respected professionals. Using citations from reputable sources will add to your own trust and credibility.

Stand on the shoulders of giants

Merriam Webster defines an Expert as “one with the special skill or knowledge representing mastery of a particular subject”. The Lit Review provides a broad understanding of the topic area and equips you with the relevant facts as well as access to the authoritative sources of those facts. That equates to mastery. Congratulations, you are now an expert.

Establish a custom heuristics set to evaluate your design

As you collect and organize your information you will begin to see patterns that define the attributes of good design. You and your team can use these as heuristics to inform your design process and to evaluate and usability test your prototypes.

Avoid the mistakes of others

People are eager to share what works and what doesn’t. With a handful of articles or informal interviews, you can assemble a quick list of potential pitfalls and then establish strategies to avoid them.

Save time in the long run

Uninspired design cycles, falling victim to common mistakes, and late stage rework are all costly and time consuming. Knowledge can be the competitive edge that distinguishes your product’s user experience from that of the competition and shortens overall development time.

Your colleagues will love you

By performing the research and distilling it down to the core themes and issues, you shorten the learning curve of your colleagues. You also increase their confidence in you.

Someone is paying you to learn new things!

The Lit Review is a great excuse to get inspired, expand your knowledge, and create a useful deliverable at the same time.

J Med Libr Assoc. (2018). It takes longer than you think: librarian time spent on systematic review tasks. Journal of the Medical Library Association (JMLA), 198–207. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/29632442

Literature review. (2019). Retrieved January 2, 2019, from https://en.wikipedia.org/wiki/Literature_review

McCombes, S. (2020). https://www.scribbr.com/research-process/research-questions/

Pare, G., & Kisiou, S. (2017). Handbook of eHealth Evaluation: An Evidence-based Approach [Internet Ed.]. Victoria (BC): University of Victoria. Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK481583/

The Writing Center. (n.d.). Literature Reviews. Retrieved from https://writingcenter.unc.edu/tips-and-tools/literature-reviews/

Uman, L. S. (2011). Systematic Reviews and Meta-Analyses. J Can Acad Child Adolesc Psychiatry, 20(1), 57–59.

University of Hawaii. (n.d.). Reflective Thinking: RT. Retrieved from http://www.hawaii.edu/intlrel/pols382/Reflective Thinking – UH/reflection.html

Venebio. (2017). 5 differences between a systematic review and other types of literature review. Retrieved January 2, 2019, from https://venebio.com/news/2017/09/5-differences-between-a-systematic-review-and-other-types-of-literature-review/

Yi, E. (2018). Themes Don’t Just Emerge — Coding the Qualitative Data. Medium, Project UX. Retrieved from https://medium.com/@projectux/themes-dont-just-emerge-coding-the-qualitative-data-95aff874fdce%0D

UX Booth is trusted by over 100,000 user experience professionals. Start your subscription today for free.

• All Services
• UX Strategy
• User Research
• UX/UI Design
• EmTech Strategy
• Our Approach
• Enterprise Clients
• Public Sector
• Competitive Indexes
• White Papers

The Value of Old-School Literature Reviews for Modern UX Research

Chances are if you have spent any amount of time in academia, you have either encountered or been asked to conduct your own literature review. Many folks want to roll their eyes at the idea of having to do a “large book report,” but this discredits the powerful research methodology that is the literature review. Spending 6 years in academia prior to my time in UX research, I have been able to see a tremendous amount of value (and critical need) for the application of literature reviews in modern UX research. 

Often the work, practices, and thought of academia sit in the lofty “ivory tower” - where it is deemed to be used only by other “worthy” academics and rarely makes it to the larger public who could also benefit from this work. This causes methods like the Literature Review to become lost in their application to non-academic research since it is deemed as only being relegated to the world of academia. Additionally, it causes academic and empirical articles to not be utilized by non-academic researchers, since they feel it is not relevant to them and their work (plus the loads of academic jargon certainly don’t help in making these texts accessible). However, once this notion of the “ivory tower” is broken down, and that “academic research methods” are simply  “research methods,” then you realize you can skip over all the jargon to get to the root of the article and the real value of the literature review is able to come through.

A literature review is conducted by referencing published academic papers and other information in a particular subject area (and sometimes a particular time period) to gain an understanding of the work that was conducted prior, as well as where the current research questions fit into this research. It can help to piece together old information in a new way or be used to trace the way a particular research field has progressed. Additionally, a literature review might further evaluate the information presented, and help the reader identify which pieces of information are the most relevant. The goal of the literature review is not to add any new contributions to the body of research, but to summarize and synthesize the work that has already been done. This methodology is critical because it helps you as the researcher determine if the problem you want to solve is one that other researchers and academics agree is worth solving- which is arguably one of the most important aspects of conducting UX research as well.

Being able to determine if the problem you want to solve is worth solving is just one of the critical insights and benefits that literature reviews can provide to UX research. Conducting a literature review in UX will help researchers cover the gaps in their research, speed up time by determining which questions of theirs might have been already answered, and also validate if the work you are doing is going to add something new and valuable in return. A literature review is basically like a guide to a particular topic or research question. Moreover, conducting a literature review for UX allows researchers the chance to draw inspiration and insight from the literature and ensure the research they conduct is grounded in theory and thought, rather than based on assumptions. Furthermore, academic articles are not just theoretical pieces of research - they can provide insights into new and innovative research methods and concrete findings, and even tell the reader what further research the author thinks should be done to help solve this problem. 

Breaking down the idea that literature reviews belong solely in the world of academia helps researchers to be able to see the real-world value and application of this methodology in modern research efforts. I think we have just scratched the surface of the value of literature reviews for UX research!

More by this Author

You might also like.

Add Comment

Key Lime Interactive is a user experience research and service design agency, with a sweet spot for emerging technology. As UX experts, our goal is to make your life easier, optimize user experiences, and make the world a better place.

Post By Topics

• Research Methods
• Emerging Trends
• Events & News
• UX Research
• User Experience
• User Interface
• Conferences & Events
• Competitive Research
• Usability Testing
• Eye tracking & Biometrics
• Qualitative Research
• Journey Mapping
• Remote Testing
• Customer Experience
• User Experience Consultant
• Mobile Banking
• Multicultural UX
• Quantitative Research
• App Development
• Conversational UI
• Mobile Devices
• User Experience Research Studies
• Diversity Equity Inclusion
• Expert Review
• Voice Technology
• Behavioral Personas
• Industries & Market
• Millennials
• Accessibility
• Diary Study & Ethnography
• Global Research
• Inclusivity Index
• Card Sort & Tree Test
• Human-first Approach
• Personalization
• Prototyping
• Diary Study
• Gamification
• Adaptive Design
• Datafication
• Entertainment
• In-Depth Interviews
• Inclusivity
• Information Architecture
• User Testing
• CX Research
• Responsive Design
• Women in UX
• A/B Testing
• Co-Creation
• Customer Journey Mapping
• Food & Beverage
• Framing Effect
• Programming
• Remote Research
• Research Analysis
• Research Report
• Transparency
• Authenticity
• Competitive Benchmarking

Competitive Insights

• Competitive Intelligence
• Confirmation Bias
• Design Thinking
• Ethnography
• Healthcare CX Design
• Healthcare Customer Experience Design
• Internet of Things
• Machine-learning
• Note-taking
• Research Participant Recruitment
• Abstract Thinking
• Algorithmic Disturbance
• Analysis Paralysis
• Anthropology
• Brand Goodwill
• Brand Value
• Change Blindness
• Competitive Intelligence Analysis
• Custom Projects
• Decentralization
• Dedicated Services
• Design Futures
• Electronic Program Guide
• Ethics of AI
• Ethnographic Analysis
• False Memories
• Financial Services
• Front-End Development
• Future State
• Games User Research
• Heuristic Evaluation
• Heuristic Principles
• Inattentional Blindness
• Invisible Link
• Keystroke Level Model
• Love Is Blind
• Low-Fidelity Wireframes
• Mental Health
• Negotiation in UX
• Netnography
• Nonprofit UX
• Patient Experiences
• Playtesting
• Rapid Research
• Research with Children
• Social Desirability Bias
• Speculative Design
• Storyboarding
• Strategic Foresight
• Swiss Cheese Model
• Systemic Design
• Systems Thinking
• UX Templates
• World Usability Day

Miami New York

Subscribe to Our CX Newsletter

Copyright © 2024, Key Lime Interactive. All Rights Reserved . Privacy Policy.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, artificial intelligence (ai) for user experience (ux) design: a systematic literature review and future research agenda.

Information Technology & People

ISSN : 0959-3845

Article publication date: 29 August 2023

The aim of this article is to map the use of AI in the user experience (UX) design process. Disrupting the UX process by introducing novel digital tools such as artificial intelligence (AI) has the potential to improve efficiency and accuracy, while creating more innovative and creative solutions. Thus, understanding how AI can be leveraged for UX has important research and practical implications.

Design/methodology/approach

This article builds on a systematic literature review approach and aims to understand how AI is used in UX design today, as well as uncover some prominent themes for future research. Through a process of selection and filtering, 46 research articles are analysed, with findings synthesized based on a user-centred design and development process.

The authors’ analysis shows how AI is leveraged in the UX design process at different key areas. Namely, these include understanding the context of use, uncovering user requirements, aiding solution design, and evaluating design, and for assisting development of solutions. The authors also highlight the ways in which AI is changing the UX design process through illustrative examples.

Originality/value

While there is increased interest in the use of AI in organizations, there is still limited work on how AI can be introduced into processes that depend heavily on human creativity and input. Thus, the authors show the ways in which AI can enhance such activities and assume tasks that have been typically performed by humans.

• Artificial intelligence
• Machine learning
• User experience
• User interface
• User-centred design process
• Systematic literature review

Acknowledgements

Since acceptance of this article, the following author(s) have updated their affiliation(s): Patrick Mikalef is at the Department of Technology Management, SINTEF Digital, Trondheim, Norway and Efpraxia D. Zamani is at the Durham Business School, Durham University, Durham, UK.

Stige, Å. , Zamani, E.D. , Mikalef, P. and Zhu, Y. (2023), "Artificial intelligence (AI) for user experience (UX) design: a systematic literature review and future research agenda", Information Technology & People , Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/ITP-07-2022-0519

Emerald Publishing Limited

Copyright © 2023, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Skip navigation

• Log in to UX Certification

World Leaders in Research-Based User Experience

Secondary research in ux.

February 20, 2022 2022-02-20

• Email article
• Share on LinkedIn
• Share on Twitter

You don’t have to do all the user-research work yourself. If somebody else already ran a study (and published it), grab it!

Have you ever completed a project only to find out that something very similar has already been done in your organization a couple of years ago? That situation is common, especially with rising employee-churn rates, and fueled the popularity of research repositories (e.g., Microsoft Human Insights System) and the growth of the  research-operations community . It should also inspire practitioners to do more secondary research.

Secondary research,  also known as desk research or, in academic contexts, literature review, refers to the act of gathering prior research findings and other relevant information related to a new project. It is a foundational part of any emerging research project and provides the project with background and context. Secondary research allows us to stand on the shoulders of giants and not to reinvent the wheel every time we initiate a new program or plan a study.

This article provides a step-by-step guide on how to conduct secondary research in UX. The key takeaway is that this type of research is not solely an intellectual exercise, but a way to minimize research costs, win internal stakeholders and get scaffolding for your own projects.

Academic publications include a literature review at the beginning to showcase context or known gaps and to justify the motivation for the research questions. However, the task of incorporating previous results is becoming more and more challenging with a growing number of publications in all fields. Therefore, practitioners across disciplines (for instance in eHealth, business, education, and technology) develop method guidelines for secondary research.  

In This Article:

When to conduct secondary research, types of secondary research, how to conduct secondary research.

Secondary research should be a standard first step in any rigorous research practice, but it’s also often cost-effective in more casual settings. Whether you are just starting a new project, joining an existing one, or planning a primary research effort for your team, it is always good to start with a broad overview of the field and existent resources. That would allow you to synthesize findings and uncover areas where more research is needed. 

Secondary research shows which topics are particularly popular or important for your organization and what problems other researchers are trying to solve. This research method is widely discussed in library and information sciences but is often neglected in UX. Nonetheless, secondary research can be useful to uncover industry trends and to inspire further studies. For example, Jessica Pater and her colleagues looked at the foundational question of participant compensation in user studies. They could have opted for user interviews or a costly large-scale survey, yet through secondary research, they were able to review 2250 unique user studies across 1662 manuscripts published in 2018-2019. They found inconsistencies in participant compensation and suggested changes to the current practices and further research opportunities.

Secondary research can be divided into two main types:  internal  and  external research.

Internal secondary research  involves gathering all relevant research findings already available in your organization. These might include artifacts from the past primary research projects, maps (e.g.,  customer-journey map ,  service blueprint ), deliverables from external consultants, or results from different kinds of  workshops  (e.g., discovery, design thinking, etc.). Hopefully, these will be available in a  research repository . 

External secondary research  is focused on sources outside of your organization, such as academic journals, public libraries, open data repositories, internet searches, and white papers published by reputable organizations. For example, external resources for the field of human-computer interaction (HCI) can be found at the  Association for Computing Machinery (ACM) digital library ,  Journal of Usability Studies (JUS ), or research websites like  ours . University libraries and labs like  UCSD Geisel Library ,  Carnegie Mellon University Libraries ,  MIT D-Lab ,  Stanford d.school , and specialized portals like  Google Scholar  offer another avenue for directed search. 

Our goal is to have the necessary depth, rigor, and usefulness for practitioners. Here are the 4 steps for conducting secondary research:

• Choose the topic of research & write a  problem statement . 

Write a concise description of the problem to be solved. For example, if you are doing a website redesign, you might want to both learn the current standards and look at all the previous design iterations to avoid issues that your team already identified.

• Identi fy external and internal resources.

Peer-reviewed publications (such as those published in academic journals and conferences) are a fairly reliable source. They always include a section describing methods, data-collection techniques, and study limitations. If a study you plan to use does not include such information, that might be a red flag and a reason to further scrutinize that source. Public datasets also often present some challenges because of errors and inclusion criteria, especially if they were collected for another purpose. 

One should be cautious of the seemingly reputable “research” findings published across different websites in a form of blog posts, which could be opinion pieces, not backed up by primary research. If you encounter such a piece, ask yourself — is the conclusion of the writeup based on a real study? If the study was quantitative, was it properly analyzed (e.g., at the very least, are  confidence intervals  reported, and was  statistical significance  evaluated?). For all studies, was the method sound and nonbiased (e.g., did the study have  internal and external validity )?

A more nuanced challenge involves evaluating findings based on a different audience, which might not be always generalizable to your situation, but may form hypotheses worthy of investigating. For example, if a design pattern is found okay to use by young adults, you may still want to know if this finding will also be valid for older generations.

• Collect and analyze data from external and internal resources.

Remember that secondary research involves both the existing data and existing research. Both of those categories become helpful resources when they are critically evaluated for any inherent biases, omissions, and limitations. If you already have some secondary data in your organization, such as customer service logs or search logs, you should include them in secondary research alongside any existent analysis of such logs and previous reports. It is helpful to revisit previous findings, compare how they have or have not been implemented to refresh institutional memory and support future research initiatives.

• Refine your problem statement and determine what still needs to be investigated.

Once you collected the relevant information, write a summary of findings, and discuss them with your team. You might need to refine your problem statement to determine what information you still need to answer your research questions. Next time your team is planning to adopt a trendy new design pattern, it may be a good idea to go back and search the web or an academic database for any evaluations of that pattern.

It is important to note that secondary research is not a substitute for primary research. It is always better to do both. Although secondary research is often cost-effective and quick, its quality depends to a large extent on the quality of your sources. Therefore, before using any secondary sources, you need to identify their validity and limitations. 

Secondary (or desk) research involves gathering existing data from inside and outside of your organization. A literature review should be done more frequently in UX because it is a viable option even for researchers with limited time and budget. The most challenging part is to persuade yourself and your team that the existing data is worth being summarized, compared, and collated to increase the overall effectiveness of your primary research. 

Jessica Pater, Amanda Coupe, Rachel Pfafman, Chanda Phelan, Tammy Toscos, and Maia Jacobs. 2021. Standardizing Reporting of Participant Compensation in HCI: A Systematic Literature Review and Recommendations for the Field. In  Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.  Association for Computing Machinery, New York, NY, USA, Article 141, 1–16. https://doi.org/10.1145/3411764.3445734

Hannah Snyder. 2019. Literature review as a research methodology: An overview and guidelines.  Journal of business research  104, 333-339. DOI: https://doi.org/10.1016/j.jbusres.2019.07.039. 

Related Courses

Discovery: building the right thing.

Conduct successful discovery phases to ensure you build the best solution

User Research Methods: From Strategy to Requirements to Design

How to pick the best UX research method for each stage in the development process

ResearchOps: Scaling User Research

Orchestrate and optimize research to amplify its impact

Related Topics

• Research Methods Research Methods

Learn More:

Always Pilot Test User Research Studies

Kim Salazar · 3 min

Level Up Your Focus Groups

Therese Fessenden · 5 min

Inductively Analyzing Qualitative Data

Tanner Kohler · 3 min

Related Articles:

Open-Ended vs. Closed Questions in User Research

Maria Rosala · 5 min

UX Research Methods: Glossary

Raluca Budiu · 12 min

Recruiting and Screening Candidates for User Research Projects

Therese Fessenden · 10 min

ResearchOps: Study Guide

Kate Kaplan · 5 min

International Usability Testing: Why You Need It

Feifei Liu · 10 min

Triangulation: Get Better Research Results by Using Multiple UX Methods

Kathryn Whitenton · 3 min

• Career Coaching

Literature Review

Ux informational interview.

• UX Resume Review & Critique
• UX Portfolio Review & Critique

UX Practice Job Interview

A literature review is a summary and evaluation of the existing research on a particular topic. In UX, a literature review can help UX researchers and designers understand the current state of knowledge on a topic and to identify gaps or areas for further research.

A literature review typically involves searching for research materials on a specific topic, such as user behavior or design principles. The search can be conducted using databases, search engines, or other sources of research materials. Once the research materials have been identified, they are reviewed and summarized, and their quality and relevance are evaluated.

A literature review can provide several benefits for UX. First, it can help UX teams gain a better understanding of the existing research on a topic and identify key themes, trends, and gaps in the literature. This can be useful for identifying areas where further research is needed or for informing the design of a product or service.

Second, a literature review can help to identify the most relevant and reliable research materials on a topic. This can be useful for UXresearchers and designers looking for evidence or guidance on a specific design problem or who want to avoid repeating research that has already been done.

Third, a literature review can help to contextualize a UX project within the broader field of UX research . It can provide a basis for comparing and contrasting a UX project with other research, and it can help to establish the contribution of the project to the existing body of knowledge.

UX Resume Review & Critique

UX Portfolio Review & Critique

• Reviews / Why join our community?
• For companies
• Frequently asked questions

7 Great, Tried and Tested UX Research Techniques

Thinking about conducting some user research ? Wondering which techniques are most likely to provide useful results? Then look no further. We’ve compiled a list of 7 excellent techniques which are tried and tested and have been proven to deliver real value in UX projects. Let’s take a look at each technique and see what it is and why it works:

Technique Number 1 – Card Sorting

Card sorting was originally a technique used in psychological research long before UX research was a “thing”. It’s a simple concept, you write words or phrases on cards, then you ask the user to categorize them. You might also ask them to label the categories. It’s a great way to determine whether your Information Architecture (IA) is heading in the right direction or to examine IA for new products.

There are all sorts of card sorting techniques and choosing the right one is important. Better still, there are a bunch of online tools that let you do card sorting remotely now – allowing you to use the technique globally and not just locally.

Why is card sorting a good technique?

It’s a very cheap form of research – particularly face-to-face, online tools may be more expensive.

It’s a very easy technique for users to understand and for clients to understand too.

It’s a very easy method to get user input (or even to get user validation) for ideas early on in a UX project.

It requires next to no effort to prepare a card sorting study.

Technique Number 2 – The Expert Review

Expert reviews involve a single “expert” walking through a product via the User Interface ( UI ) and looking for issues with the design, accessibility , and usability of the product. There’s no fixed process to follow and the expert review can vary from professional-to-professional as well from product-to-product. The more expertise the reviewer has in usability and UX design – the more valuable their input (in most cases).

Why is an Expert Review a Good Technique?

It’s quick, easy and cheap. This is doubly so when you compare it to more formal usability testing methods.

It only takes a single professional to conduct an expert review.

It is a great way to inform further UX research and caution should be used in taking an expert review at face value without further user testing .

Technique Number 3 – Eye Movement Tracking

It can be really useful to know where your users are looking when they’re using your system. It helps with UI design and it helps with knowing how to prioritize certain kinds of content. This technique was developed for academic research and has been used extensively in medical research and has now become popular and cost-effective enough to be deployed by the UX team too.

Why is Eye Movement Tracking a Good Technique?

Now that technology is advanced enough, eye movement tracking systems are no longer bulky and invasive and they do not interfere with the results of usability tests.

The technology is reasonably affordable now. It won’t suit every project budget but it often won’t break the bank either.

The technology is now reliable enough for results to be easy to reproduce and for researchers to be able to rely on the outputs.

Clients love eye movement tracking. It’s a great way to demonstrate why they might want to invest in further usability testing.

Technique Number 4 – Field Studies

This is actually a number of techniques under a broad heading. It’s all about going out and observing users “in the wild” so that behaviour can be measured in the context where a product will actually be used. It includes; ethnographic research , interviews and observations, plus contextual enquiry.

Why are Field Studies a Good Technique?

There’s no stronger form of research than observing users behaving as they will when they use your product. Researchers love these techniques and are often passionate about persuading their clients to take them on board.

When conducted well, the outputs of field studies provide the deepest insights into user issues and how they might be solved.

Technique Number 5 – Usability Testing

A firm favourite that has a long and prestigious history in UX research. Usability testing is the observation of users trying to carry out tasks with a product. They can focus on a single process or be much more wide ranging.

Why is Usability Testing a Good Technique?

Can you think of a better way to understand what users do than watching them do stuff? Of course, you have to pick the right users – they need to be a good representation of the user base as a whole but that’s pretty much the only constraint.

Usability tests produce specific results that lead to specific action. Better still, it’s very hard for people to contradict decisions based on these tests; it’s nearly impossible to refute evidence of user behaviour.

You can bring clients into usability testing easily as observers. This increases their enthusiasm for such testing and shows clearly why such testing adds value.

Technique Number 6 – Remote Usability Testing

This is usability testing but without the need to drag users into your laboratory environment. It was once complex and expensive but technology has moved on and now it’s fairly simple to set up and reasonable value for money too.

Why is Remote Usability Testing a Good Technique?

It often saves time and money when compared to lab testing and it allows for a wider range of participants when you don’t have to get them in the lab.

It is closer to field testing in some respects in that the tests are conducted in the user’s environment and not an artificial lab environment. This delivers better results in many cases than a lab environment.

Technique Number 7 – User Personas

User personas are a fictional representation of the ideal user. They focus on the goals of the user, the characteristics that they have and the attitudes that they display. They also examine what the user expects from the product.

User personas are created from other forms of user research and thus offer an in-depth real-life vivid portrait that is easy for the whole team to keep in mind when designing products. User personas have a name and a backstory. They inspire the imagination and keep focus on the user.

Why are User Personas a Good Technique?

They are a step above the old user profile and give a more in depth and specific look at a user.

They are easy for people to relate to and become part of the team as they are constantly spoken about during a project.

They are a lot of fun and they tend to be interesting, easy for people to engage with and more memorable than many other research outputs.

There are many user research techniques but these 7 have shown time and again to offer valuable input into UX projects. Which is your favourite?

Header Image: Author/Copyright holder: Zeke Franco. Copyright terms and licence: CC BY-NC-ND 2.0

User Experience: The Beginner’s Guide

Get Weekly Design Insights

Topics in this article, what you should read next, apple’s product development process – inside the world’s greatest design organization.

• 1.4k shares

How to Change Your Career from Graphic Design to UX Design

What is Interaction Design?

• 1.3k shares
• 3 weeks ago

Shneiderman’s Eight Golden Rules Will Help You Design Better Interfaces

The Principles of Service Design Thinking - Building Better Services

A Simple Introduction to Lean UX

• 3 years ago

Dieter Rams: 10 Timeless Commandments for Good Design

How to Do a Thematic Analysis of User Interviews

• 1.2k shares

The 7 Factors that Influence User Experience

Adaptive vs. Responsive Design

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this article , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share the knowledge!

Share this content on:

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this article.

New to UX Design? We’re giving you a free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

New to UX Design? We’re Giving You a Free ebook!

• Schools & departments

Literature review

A general guide on how to conduct and write a literature review.

Please check course or programme information and materials provided by teaching staff, including your project supervisor, for subject-specific guidance.

What is a literature review?

A literature review is a piece of academic writing demonstrating knowledge and understanding of the academic literature on a specific topic placed in context.  A literature review also includes a critical evaluation of the material; this is why it is called a literature review rather than a literature report. It is a process of reviewing the literature, as well as a form of writing.

To illustrate the difference between reporting and reviewing, think about television or film review articles.  These articles include content such as a brief synopsis or the key points of the film or programme plus the critic’s own evaluation.  Similarly the two main objectives of a literature review are firstly the content covering existing research, theories and evidence, and secondly your own critical evaluation and discussion of this content. 

Usually a literature review forms a section or part of a dissertation, research project or long essay.  However, it can also be set and assessed as a standalone piece of work.

What is the purpose of a literature review?

…your task is to build an argument, not a library. Rudestam, K.E. and Newton, R.R. (1992) Surviving your dissertation: A comprehensive guide to content and process. California: Sage, p49.

In a larger piece of written work, such as a dissertation or project, a literature review is usually one of the first tasks carried out after deciding on a topic.  Reading combined with critical analysis can help to refine a topic and frame research questions.  Conducting a literature review establishes your familiarity with and understanding of current research in a particular field before carrying out a new investigation. After doing a literature review, you should know what research has already been done and be able to identify what is unknown within your topic.

When doing and writing a literature review, it is good practice to:

• summarise and analyse previous research and theories;
• identify areas of controversy and contested claims;
• highlight any gaps that may exist in research to date.

Conducting a literature review

Focusing on different aspects of your literature review can be useful to help plan, develop, refine and write it.  You can use and adapt the prompt questions in our worksheet below at different points in the process of researching and writing your review.  These are suggestions to get you thinking and writing.

Developing and refining your literature review (pdf)

Developing and refining your literature review (Word)

Developing and refining your literature review (Word rtf)

Writing a literature review has a lot in common with other assignment tasks.  There is advice on our other pages about thinking critically, reading strategies and academic writing.  Our literature review top tips suggest some specific things you can do to help you submit a successful review.

Literature review top tips (pdf)

Literature review top tips (Word rtf)

Our reading page includes strategies and advice on using books and articles and a notes record sheet grid you can use.

Reading at university

The Academic writing page suggests ways to organise and structure information from a range of sources and how you can develop your argument as you read and write.

Academic writing

The Critical thinking page has advice on how to be a more critical researcher and a form you can use to help you think and break down the stages of developing your argument.

Critical thinking

As with other forms of academic writing, your literature review needs to demonstrate good academic practice by following the Code of Student Conduct and acknowledging the work of others through citing and referencing your sources.  

Good academic practice

As with any writing task, you will need to review, edit and rewrite sections of your literature review.  The Editing and proofreading page includes tips on how to do this and strategies for standing back and thinking about your structure and checking the flow of your argument.

Editing and proofreading

Guidance on literature searching from the University Library

The Academic Support Librarians have developed LibSmart I and II, Learn courses to help you develop and enhance your digital research skills and capabilities; from getting started with the Library to managing data for your dissertation.

Searching using the library’s DiscoverEd tool: DiscoverEd

Finding resources in your subject: Subject guides

The Academic Support Librarians also provide one-to-one appointments to help you develop your research strategies.

1 to 1 support for literature searching and systematic reviews

Advice to help you optimise use of Google Scholar, Google Books and Google for your research and study: Using Google

Managing and curating your references

A referencing management tool can help you to collect and organise and your source material to produce a bibliography or reference list. 

Referencing and reference management

Information Services provide access to Cite them right online which is a guide to the main referencing systems and tells you how to reference just about any source (EASE log-in may be required).

Cite them right

Published study guides

There are a number of scholarship skills books and guides available which can help with writing a literature review.  Our Resource List of study skills guides includes sections on Referencing, Dissertation and project writing and Literature reviews.

Study skills guides

Empowering education development through AIGC: A systematic literature review

• Published: 29 February 2024

Cite this article

• Xiaojiao Chen 1 ,
• Zhebing Hu 2 &
• Chengliang Wang   ORCID: orcid.org/0000-0003-2208-3508 3  

534 Accesses

Explore all metrics

As an exemplary representative of AIGC products, ChatGPT has ushered in new possibilities for the field of education. Leveraging its robust text generation and comprehension capabilities, it has had a revolutionary impact on pedagogy, learning experiences, personalized education and other aspects. However, to date, there has been no comprehensive review of AIGC technology’s application in education. In light of this gap, this study employs a systematic literature review and selects 134 relevant publications on AIGC’s educational application from 4 databases: EBSCO, EI Compendex, Scopus, and Web of Science. The study aims to explore the macro development status and future trends in AIGC’s educational application. The following findings emerge: 1) In the AIGC’s educational application field, the United States is the most active country. Theoretical research dominates the research types in this domain; 2) Research on AIGC’s educational application is primarily published in journals and academic conferences in the fields of educational technology and medicine; 3) Research topics primarily focus on five themes: AIGC technology performance assessment, AIGC technology instructional application, AIGC technology enhancing learning outcomes, AIGC technology educational application’s Advantages and Disadvantages analysis, and AIGC technology educational application prospects. 4) Through Grounded Theory, the study delves into the core advantages and potential risks of AIGC’s educational application, deconstructing the scenarios and logic of AIGC’s educational application. 5) Based on a review of existing literature, the study provides valuable future agendas from both theoretical and practical application perspectives. Discussing the future research agenda contributes to clarifying key issues related to the integration of AI and education, promoting more intelligent, effective, and sustainable educational methods and tools, which is of great significance for advancing innovation and development in the field of education.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Students’ voices on generative AI: perceptions, benefits, and challenges in higher education

Cecilia Ka Yuk Chan & Wenjie Hu

The AI generation gap: Are Gen Z students more interested in adopting generative AI such as ChatGPT in teaching and learning than their Gen X and millennial generation teachers?

Cecilia Ka Yuk Chan & Katherine K. W. Lee

How should we change teaching and assessment in response to increasingly powerful generative Artificial Intelligence? Outcomes of the ChatGPT teacher survey

Matt Bower, Jodie Torrington, … Mark Alfano

Data availability

The datasets (Coding results) generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Abdelghani, R., Wang, Y. H., Yuan, X., Wang, T., Lucas, P., Sauzéon, H., & Oudeyer, P. Y. (2023). Gpt-3-driven pedagogical agents to train children’s curious question-asking skills. International Journal of Artificial Intelligence in Education , 1–36. https://doi.org/10.1007/s40593-023-00340-7

Abdulai, A. F., & Hung, L. (2023). Will ChatGPT undermine ethical values in nursing education, research, and practice. Nursing Inquiry . e12556–e12556. https://doi.org/10.1111/nin.12556

Ahmed, S. K. (2023). The Impact of ChatGPT on the Nursing Profession: Revolutionizing Patient Care and Education. Annals of Biomedical Engineering , 1–2. https://doi.org/10.1007/s10439-023-03262-6

Albeshri, A., & Thayananthan, V. (2018). Analytical techniques for decision making on information security for big data breaches. International Journal of Information Technology & Decision Making, 17 (02), 527–545. https://doi.org/10.1142/S0219622017500432

Article   Google Scholar  

Allen, B., Dreyer, K., Stibolt Jr, R., Agarwal, S., Coombs, L., Treml, C., ... & Wald, C. (2021). Evaluation and real-world performance monitoring of artificial intelligence models in clinical practice: Try it, buy it, check it. Journal of the American College of Radiology , 18 (11), 1489–1496. https://doi.org/10.1016/j.jacr.2021.08.022

Alnaqbi, N. M., & Fouda, W. (2023). Exploring the role of ChatGPT and social media in enhancing student evaluation of teaching styles in higher education using neutrosophic sets. International Journal of Neutrosophic Science, 20 (4), 181–190. https://doi.org/10.1111/nin.12556

Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., bin Saleh, K., ... & Albekairy, A. M. (2023). The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Research in Social and Administrative Pharmacy. https://doi.org/10.1016/j.sapharm.2023.05.016

Ancillai, C., Sabatini, A., Gatti, M., & Perna, A. (2023). Digital technology and business model innovation: A systematic literature review and future research agenda. Technological Forecasting and Social Change, 188 , 122307. https://doi.org/10.1016/j.techfore.2022.122307

Banić, B., Konecki, M., & Konecki, M. (2023, May). Pair Programming Education Aided by ChatGPT. In 2023 46th MIPRO ICT and Electronics Convention (MIPRO) (pp. 911–915). IEEE.

Busch, F., Adams, L. C., & Bressem, K. K. (2023). Biomedical ethical aspects towards the implementation of artificial intelligence in medical education. Medical Science Educator., 33 , 1007–1012. https://doi.org/10.1007/s40670-023-01815-x

Article   PubMed   PubMed Central   Google Scholar  

Chang, C.-Y., Kuo, S.-Y., & Hwang, G.-H. (2022). Chatbot-facilitated nursing education: Incorporating a knowledge-based Chatbot system into a nursing training program. Educational Technology & Society , 25 (1), 15–27. Retrieved December 19, 2023, from https://www.jstor.org/stable/48647027

Charmaz, K., & Thornberg, R. (2021). The pursuit of quality in grounded theory. Qualitative Research in Psychology, 18 (3), 305–327. https://doi.org/10.1080/14780887.2020.1780357

Choi, E. P. H., Lee, J. J., Ho, M. H., Kwok, J. Y. Y., & Lok, K. Y. W. (2023). Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Education Today, 125 , 105796–105796. https://doi.org/10.1016/j.nedt.2023.105796

Article   PubMed   Google Scholar  

Cooper, G. (2023). Examining science education in chatgpt: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32 (3), 444–452. https://doi.org/10.1007/s10956-023-10039-y

Article   ADS   Google Scholar  

Cross, J., Robinson, R., Devaraju, S., Vaughans, A., Hood, R., Kayalackakom, T., ... & Robinson, R. E. (2023). Transforming medical education: Assessing the integration of ChatGPT into faculty workflows at a Caribbean medical school. Cureus , 15 (7). https://doi.org/10.7759/cureus.41399

Currie, G. M. (2023, May). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? In Seminars in Nuclear Medicine. WB Saunders. https://doi.org/10.1053/j.semnuclmed.2023.04.008

Das, D., Kumar, N., Longjam, L. A., Sinha, R., Roy, A. D., Mondal, H., & Gupta, P. (2023). Assessing the capability of ChatGPT in answering first-and second-order knowledge questions on microbiology as per competency-based medical education curriculum. Cureus , 15 (3). https://doi.org/10.7759/cureus.36034

Deacon, B., Laufer, M., & Schäfer, L. O. (2023). Infusing educational technologies in the heart of the university-a systematic literature review from an organisational perspective. British Journal of Educational Technology, 54 (2), 441–466. https://doi.org/10.1111/bjet.13277

Deeley, S. J. (2018). Using technology to facilitate effective assessment for learning and feedback in higher education. Assessment & Evaluation in Higher Education, 43 (3), 439–448. https://doi.org/10.1080/02602938.2017.1356906

Deng, X., & Yu, Z. (2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15 (4), 2940. https://doi.org/10.3390/su15042940

Diekemper, R. L., Ireland, B. K., & Merz, L. R. (2015). Development of the documentation and appraisal review tool for systematic reviews. World Journal of Meta-Analysis, 3 (3), 142–150. https://doi.org/10.13105/wjma.v3.i3.142

Engel, A., & Coll, C. (2022). Hybrid teaching and learning environments to promote personalized learning. RIED-Revista Iberoamericana de Educacion a Distancia , 225–242. https://doi.org/10.5944/ried.25.1.31489

Escotet, M. Á. (2023). The optimistic future of Artificial Intelligence in higher education. Prospects, 1–10. https://doi.org/10.1007/s11125-023-09642-z

Esplugas, M. (2023). The use of artificial intelligence (AI) to enhance academic communication, education and research: A balanced approach. Journal of Hand Surgery (European Volume) , 48 (8), 819–822.  https://doi.org/10.1177/17531934231185746

Extance, A. (2023). ChatGPT has entered the classroom: How LLMs could transform education. Nature, 623 , 474–477. https://doi.org/10.1038/d41586-023-03507-3

Article   CAS   PubMed   ADS   Google Scholar  

Foroughi, B., Senali, M. G., Iranmanesh, M., Khanfar, A., Ghobakhloo, M., Annamalai, N., & Naghmeh-Abbaspour, B. (2023). Determinants of Intention to Use ChatGPT for Educational Purposes: Findings from PLS-SEM and fsQCA. International Journal of Human-Computer Interaction , 1–20. https://doi.org/10.1080/10447318.2023.2226495

Gaur, A., & Kumar, M. (2018). A systematic approach to conducting review studies: An assessment of content analysis in 25 years of IB research. Journal of World Business, 53 (2), 280–289. https://doi.org/10.1016/j.jwb.2017.11.003

Ghorbani, M., Bahaghighat, M., Xin, Q., & Özen, F. (2020). ConvLSTMConv network: A deep learning approach for sentiment analysis in cloud computing. Journal of Cloud Computing, 9 (1), 1–12. https://doi.org/10.1186/s13677-020-00162-1

Glaser, B., & Strauss, A. (2017). Discovery of grounded theory: Strategies for qualitative research . Routledge https://doi.org/10.1016/j.jwb.2017.11.003

Gough, D., Oliver, S., & Thomas, J. (Eds.). (2017). An introduction to systematic reviews . Sage https://doi.org/10.5124/jkma.2014.57.1.49

Grant, N., & Metz, C. (2022). A new chat bot is a ‘code red’ for Google's search business, The New York Times. Available at: https://www.nytimes.com/2022/12/21/technology/ai-chatgpt-google-search.html . Accessed 19 Dec 2023

Hadi, M. S., & Junor, R. S. (2022). Speaking to devices: Can we use Google assistant to Foster Students' speaking skills? Journal of Languages and Language Teaching, 10 (4), 570–578. https://doi.org/10.33394/jollt.v10i4.5808

Heng, J. J., Teo, D. B., & Tan, L. F. (2023). The impact of Chat Generative Pre-trained Transformer (ChatGPT) on medical education. Postgraduate Medical Journal , qgad058. https://doi.org/10.1093/postmj/qgad058

Ho, W., & Lee, D. (2023). Enhancing engineering education in the roblox metaverse: Utilizing chatgpt for game development for electrical machine course. International Journal on Advanced Science, Engineering & Information Technology , 13 (3). https://doi.org/10.18517/ijaseit.13.3.18458

Holmes, W., & Kay, J. (2023, June). AI in education. Coming of age? The community voice. In International conference on artificial intelligence in education (pp. 85–90). Springer Nature Switzerland.

Google Scholar  

Hsu, Y. C., & Ching, Y. H. (2023). Generative Artificial Intelligence in Education, Part One: the Dynamic Frontier. TechTrends , 1–5. https://doi.org/10.1007/s11528-023-00863-9

Hwang, G. J., & Chang, C. Y. (2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments , 1–14. https://doi.org/10.1080/10494820.2021.1952615

Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023, April). Chatgpt and software testing education: Promises & perils. In 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 4130–4137). IEEE.

Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies , 1–20. https://doi.org/10.1007/s10639-023-11834-1

Jing, Y., Wang, C., Chen, Y., Wang, H., Yu, T., & Shadiev, R. (2023). Bibliometric mapping techniques in educational technology research: A systematic literature review. Education and Information Technologies , 1–29. https://doi.org/10.1007/s10639-023-12178-6

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33 (7), 14–26. https://doi.org/10.3102/0013189X033007014

Karabacak, M., Ozkara, B. B., Margetis, K., Wintermark, M., & Bisdas, S. (2023). The advent of generative language models in medical education. JMIR Medical Education, 9 , e48163. https://doi.org/10.2196/48163

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103 , 102274. https://doi.org/10.1016/j.lindif.2023.102274

Kepuska, V., & Bohouta, G. (2018, January). Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 2018 IEEE 8th annual computing and communication workshop and conference (CCWC) (pp. 99–103). IEEE.

Kerneža, M. (2023). Fundamental And Basic Cognitive Skills Required For Teachers To Effectively Use Chatbots In Education. In Science And Technology Education: New Developments And Innovations (pp. 99–110). Scientia Socialis, UAB.

Kılıçkaya, F. (2020). Using a chatbot, Replika, to practice writing through conversations in L2 English: A Case study. In New Technological applications for foreign and second language learning and teaching (pp. 221–238). IGI Global. https://doi.org/10.4018/978-1-7998-2591-3.ch011

Killian, C. M., Marttinen, R., Howley, D., Sargent, J., & Jones, E. M. (2023). “Knock, Knock... Who’s There?” ChatGPT and Artificial Intelligence-Powered Large Language Models: Reflections on Potential Impacts Within Health and Physical Education Teacher Education. Journal of Teaching in Physical Education , 1 (aop), 1–5. https://doi.org/10.1123/jtpe.2023-0058

Kohnke, L. (2022). A pedagogical Chatbot: A supplemental language learning Tool. RELC Journal , 00336882211067054. https://doi.org/10.1177/00336882211067054

Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., ... & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digital Health, 2 (2), e0000198. https://doi.org/10.1371/journal.pdig.0000198

Lee, D., & Yeo, S. (2022). Developing an AI-based chatbot for practicing responsive teaching in mathematics. Computers & Education, 191 , 104646. https://doi.org/10.1016/j.compedu.2022.104646

Lee, L. W., Dabirian, A., McCarthy, I. P., & Kietzmann, J. (2020). Making sense of text: Artificial intelligence-enabled content analysis. European Journal of Marketing, 54 (3), 615–644. https://doi.org/10.1108/EJM-02-2019-0219

Li, L., Ma, Z., Fan, L., Lee, S., Yu, H., & Hemphill, L. (2023). ChatGPT in education: A discourse analysis of worries and concerns on social media. arXiv preprint arXiv :2305.02201. https://doi.org/10.48550/arXiv.2305.02201

Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21 (2), 100790. https://doi.org/10.1016/j.ijme.2023.100790

Lin, T. J., & Lan, Y. J. (2015). Language learning in virtual reality environments: Past, present, and future. Journal of Educational Technology & Society, 18 (4), 486–497. Retrieved December 19, 2023, from http://www.jstor.org/stable/jeductechsoci.18.4.486

Lodge, J. M., Thompson, K., & Corrin, L. (2023). Mapping out a research agenda for generative artificial intelligence in tertiary education. Australasian Journal of Educational Technology, 39 (1), 1–8. https://doi.org/10.14742/ajet.8695

Luo, H., Li, G., Feng, Q., Yang, Y., & Zuo, M. (2021). Virtual reality in K-12 and higher education: A systematic review of the literature from 2000 to 2019. Journal of Computer Assisted Learning, 37 (3), 887–901. https://doi.org/10.1111/jcal.12538

Mariani, M. M., Hashemi, N., & Wirtz, J. (2023). Artificial intelligence empowered conversational agents: A systematic literature review and research agenda. Journal of Business Research, 161 , 113838. https://doi.org/10.1016/j.jbusres.2023.113838

Mohamed, A. M. (2023). Exploring the potential of an AI-based Chatbot (ChatGPT) in enhancing English as a Foreign Language (EFL) teaching: perceptions of EFL Faculty Members. Education and Information Technologies, 1–23. https://doi.org/10.1007/s10639-023-11917-z

Mohammad, B., Supti, T., Alzubaidi, M., Shah, H., Alam, T., Shah, Z., & Househ, M. (2023). The pros and cons of using ChatGPT in medical education: A scoping review. Student Health Technology Information, 305 , 644–647. https://doi.org/10.3233/SHTI230580

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151 , 264–269. https://doi.org/10.7326/0003-4819-151-4-200908180-00135

Mokmin, N. A. M., & Ibrahim, N. A. (2021). The evaluation of chatbot as a tool for health literacy education among undergraduate students. Education and Information Technologies, 26 (5), 6033–6049. https://doi.org/10.1007/s10639-021-10542-y

Patel, N., Nagpal, P., Shah, T., Sharma, A., Malvi, S., & Lomas, D. (2023). Improving mathematics assessment readability: Do large language models help? Journal of Computer Assisted Learning, 39 (3), 804–822. https://doi.org/10.1111/jcal.12776

Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies, 45 (4), O1–O16. https://doi.org/10.1111/ijcs.12695

Pentina, I., Xie, T., Hancock, T., & Bailey, A. (2023). Consumer–machine relationships in the age of artificial intelligence: Systematic literature review and research directions. Psychology & Marketing, 40 (8), 1593–1614. https://doi.org/10.1002/mar.21853

Pereira, R., Reis, A., Barroso, J., Sousa, J., & Pinto, T. (2022). Virtual assistants applications in education. In International conference on technology and innovation in learning, teaching and education (pp. 468–480). Springer Nature Switzerland.

Pinto, A. S., Abreu, A., Costa, E., & Paiva, J. (2023). How Machine Learning (ML) is transforming higher education: A systematic literature review. Journal of Information Systems Engineering and Management, 8 (2). https://doi.org/10.55267/iadt.07.13227

Prikshat, V., Islam, M., Patel, P., Malik, A., Budhwar, P., & Gupta, S. (2023). AI-augmented HRM: Literature review and a proposed multilevel framework for future research. Technological Forecasting and Social Change, 193 , 122645. https://doi.org/10.1016/j.techfore.2023.122645

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147 , 103778. https://doi.org/10.1016/j.compedu.2019.103778

Rahimzadeh, V., Kostick-Quenet, K., Blumenthal Barby, J., & McGuire, A. L. (2023). Ethics education for healthcare professionals in the era of chatGPT and other large language models: Do we still need it?. The American Journal of Bioethics , 1–11. https://doi.org/10.1080/15265161.2023.2233358

Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13 (9), 5783. https://doi.org/10.3390/app13095783

Article   CAS   Google Scholar  

Rasul, T., Nair, S., Kalendra, D., Robin, M., de Oliveira Santini, F., Ladeira, W. J., ... & Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6 (1). https://doi.org/10.37074/jalt.2023.6.1

Sallam, M. (2023a). ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. In Healthcare (Vol. 11, No. 6, p. 887). MDPI. https://doi.org/10.3390/healthcare11060887

Sallam, M. (2023b). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11 (6), 887. https://doi.org/10.3390/healthcare11060887

Sánchez-Ruiz, L. M., Moll-López, S., Nuñez-Pérez, A., Moraño-Fernández, J. A., & Vega-Fleitas, E. (2023). ChatGPT challenges blended learning methodologies in engineering education: A case study in mathematics. Applied Sciences, 13 (10), 6039. https://doi.org/10.3390/app13106039

Sandu, N., & Gide, E. (2019). Adoption of AI-Chatbots to enhance student learning experience in higher education in India. In 2019 18th International Conference on Information Technology Based Higher Education and Training (ITHET) (pp. 1–5). IEEE.

Schmulian, A., & Coetzee, S. A. (2019). Students’ experience of team assessment with immediate feedback in a large accounting class. Assessment & Evaluation in Higher Education, 44 (4), 516–532. https://doi.org/10.1080/02602938.2018.1522295

Seetharaman, R. (2023). Revolutionizing medical education: Can ChatGPT boost subjective learning and expression? Journal of Medical Systems, 47 (1), 1–4. https://doi.org/10.1007/s10916-023-01957-w

Sharma, M., & Sharma, S. (2023). A holistic approach to remote patient monitoring, fueled by ChatGPT and Metaverse technology: The future of nursing education. Nurse Education Today, 131 , 105972. https://doi.org/10.1016/j.nedt.2023.105972

Shlonsky, A., Noonan, E., Littell, J. H., & Montgomery, P. (2011). The role of systematic reviews and the Campbell collaboration in the realization of evidence-informed practice. Clinical Social Work Journal, 39 , 362–368. https://doi.org/10.1007/s10615-010-0307-0

Shoja, M. M., Van de Ridder, J. M., & Rajput, V. (2023). The emerging role of generative artificial intelligence in medical education, research, and practice. Cureus, 15 (6), e40883. https://doi.org/10.7759/cureus.40883

Siegle, D. (2023). A role for ChatGPT and AI in gifted education. Gifted Child Today, 46 (3), 211–219. https://doi.org/10.1177/10762175231168443

Smith, A., Hachen, S., Schleifer, R., Bhugra, D., Buadze, A., & Liebrenz, M. (2023). Old dog, new tricks? Exploring the potential functionalities of ChatGPT in supporting educational methods in social psychiatry. International Journal of Social Psychiatry . https://doi.org/10.1177/0020764023117845

Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments , 1–14. https://doi.org/10.1080/10494820.2023.2209881

Tam, W., Huynh, T., Tang, A., Luong, S., Khatri, Y., & Zhou, W. (2023). Nursing education in the age of artificial intelligence powered Chatbots (AI-Chatbots): Are we ready yet? Nurse Education Today, 129 , 105917. https://doi.org/10.1016/j.nedt.2023.105917

Teel, Z. A., Wang, T., & Lund, B. (2023). ChatGPT conundrums: Probing plagiarism and parroting problems in higher education practices. College & Research Libraries News, 84 (6), 205. https://doi.org/10.5860/crln.84.6.205

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10 (1), 15. https://doi.org/10.1186/s40561-023-00237-x

Tsang, R. (2023). Practical applications of ChatGPT in undergraduate medical education. Journal of Medical Education and Curricular Development , 10 . https://doi.org/10.1177/23821205231178449

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 1–25. https://doi.org/10.1007/s10639-023-11742-4

Zhang, R., Zou, D., & Cheng, G. (2023a). A review of chatbot-assisted learning: pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments , 1–29. https://doi.org/10.1080/10494820.2023.2202704

Zhang, S., Shan, C., Lee, J. S. Y., Che, S., & Kim, J. H. (2023b). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies , 1–21. https://doi.org/10.1007/s10639-023-11805-6

Zhu, C., Sun, M., Luo, J., Li, T., & Wang, M. (2023). How to harness the potential of ChatGPT in education? Knowledge Management & E-Learning, 15 (2), 133. https://doi.org/10.34105/j.kmel.2023.15.008

Download references

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Hangzhou, China

Xiaojiao Chen

College of Foreign Languages, Zhejiang University of Technology, Hangzhou, China

Department of Education Information Technology, Faculty of Education, East China Normal University, Shanghai, China

Chengliang Wang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Chengliang Wang .

Ethics declarations

Conflict of interest.

During the research, the authors indicate that no commercial or financial ties that may be regarded a possible conflict of interest existed.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Chen, X., Hu, Z. & Wang, C. Empowering education development through AIGC: A systematic literature review. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12549-7

Download citation

Received : 19 October 2023

Accepted : 05 February 2024

Published : 29 February 2024

DOI : https://doi.org/10.1007/s10639-024-12549-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence generated content
• Artificial intelligence
• Systematic literature review
• Educational technology
• Find a journal
• Publish with us
• Track your research

• Ask a Librarian

Artificial Intelligence (AI)

Ai for systematic review.

• How to Cite AI Generated Content
• Prompt Design
• Resources for Educators
• Purdue AI Resources
• AI and Ethics
• Publisher Policies
• Selected Journals in AI

Various AI tools are invaluable throughout the systematic review or evidence synthesis process. While the consensus acknowledges the significant utility of AI tools across different review stages, it's imperative to grasp their inherent biases and weaknesses. Moreover, ethical considerations such as copyright and intellectual property must be at the forefront.

• Application ChatGPT in conducting systematic reviews and meta-analyses
• Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation?
• Artificial intelligence in systematic reviews: promising when appropriately used
• Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions
• In-depth evaluation of machine learning methods for semi-automating article screening in a systematic review of mechanistic
• Tools to support the automation of systematic reviews: a scoping review
• The use of a large language model to create plain language summaries of evidence reviews in healthcare: A feasibility study
• Using artificial intelligence methods for systematic review in health sciences: A systematic review

AI Tools for Systematic Review

• DistillerSR Securely automate every stage of your literature review to produce evidence-based research faster, more accurately, and more transparently at scale.
• Rayyan A web-tool designed to help researchers working on systematic reviews, scoping reviews and other knowledge synthesis projects, by dramatically speeding up the process of screening and selecting studies.
• RobotReviewer A machine learning system aiming which aims to automate evidence synthesis.
• << Previous: AI Tools
• Next: How to Cite AI Generated Content >>
• Last Edited: Mar 21, 2024 1:34 PM
• URL: https://guides.lib.purdue.edu/ai

REVIEW article

This article is part of the research topic.

Recreational forests for co-benefits: conservation, tourism and well-being

An Overview of Community Livelihoods in Biosphere Reserves (BRs): based on the Sustainable Livelihoods Framework for the 21st Century Provisionally Accepted

• 1 Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences (CAS), China

The final, formatted version of the article will be published soon.

Biosphere Reserves (BRs) are the protected areas proposed by the Man and the Biosphere Programme for harmonious coexistence between humanity and nature. Human activities represented by community livelihoods have always been one of the critical issues in the protection and development of BRs. However, the lack of comprehensive research on the status quo and problems of community livelihoods in BRs has caused difficulties in policy formulation and management. In the form of a literature review, this study attempts to summarize and sort out the overview of community livelihoods in BRs by screening the academic literature with the keywords of BRs and livelihoods and using the 21st Century Sustainable Livelihood Framework as the road map. As a result, community livelihoods in BRs highly depend on environmental resources, increasing vulnerability. Although establishing BRs has brought financial and business opportunities to the community, it also provides environmental resources, public services, and geographical areas in the climate-environmental context needed for livelihood maintenance. However, community livelihoods and climate-environmental context show a contradictory relationship of 'mismatch between supply and demand' in environmental resources and public services in BRs. In geographical areas, the conflicts brought by illegal activities, invasive alien species, and wildlife-human conflicts are also gradually increasing. At the same time, unbalanced physical and financial assets and relational power with mixed praise also challenge the sustainable development of community livelihoods in BRs. Therefore, this study believes that through multi-stakeholder joint efforts, BRs Friendly Community Livelihoods other than the initial livelihood with high environmental resource dependence can be sought for communities through livelihood diversification, community participatory management, and community spatial pattern refinement.

Keywords: Sustainable Livelihoods Framework, Man and the biosphere programme, Environmental resources, review, resource management

Received: 23 Jan 2024; Accepted: 25 Mar 2024.

Copyright: © 2024 Yang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Lun Yang, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences (CAS), Beijing, China

People also looked at