field study for user research

• Set up your Design System Workshop
• Tečaj: oblikovalski sistem
• UI tečaj: od ideje do prototipa
• DSG Newsletter
• The Design Strategy Cards
• The Ultimate Design Strategy e-book
• Free Design Strategy Crash Course

No products in the cart.

How to conduct user research: A step-by-step guide

This is part one of a guide to User research.

Continue with part two: How to conduct user research: A Step-by-step guide

Continue with part three: What is exploratory research and why is it so exciting?

What user research did you conduct to reveal your ideal user?

Uh-oh. Not this question again. We both know the most common answer and it’s not great.

“Uhm, we talked to some users and had a brainstorming session with our team. It’s not much, but we don’t have time to do anything more right now. It’s better than nothing.”

Let’s be brutally honest about the meaning of that answer and rephrase it:

“ We don’t have time to get to know our actual user and maximize our chances of success. We’ll just assume that we know what they want and then wonder why the product fails at a later stage.”

If that sounds super bad, it’s because IT IS. You don’t want to end up in this situation. And you won’t.

After reading this guide, you’ll know exactly how to carry out the user research that will become your guiding star during product development.

On this page

Why is user research so important?

Step #1: define research objectives.

Go ahead – create that fake persona

Step #2: Pick your methods

Qualitative methods – the why, quantitative methods – the what, behavioral and attitudinal methods, step #3: find your participants, how to recruit participants, how many participants, step #4: conduct user research.

Focus groups

Competitive analysis

Field studies

What’s next?

User research can be a scary word. It may sound like money you don’t have, time you can’t spare, and expertise you need to find. That’s why some people convince themselves that it’s not that important.

Which is a HUGE mistake.

User research is crucial – without it, you’ll spend your energy, time and money on a product that is based around false assumptions that won’t work in the real world.

Let’s take a look at Segway, a technologically brilliant product with incredible introductory publicity. Although it’s still around, it simply didn’t reach initial expectations. Here are some of the reasons why:

• It brought mockery, not admiration. The user was always “that guy”, who often felt fat or lazy.
• Cities were not prepared for it. Neither users nor policemen knew if it should be used on the road or on the sidewalk.
• A large segment of the target market comprised of postal and security workers. However, postal workers need both hands while walking, and security workers prefer bikes that don’t have a limited range.

Segway mainly fell short because of issues that could’ve been foreseen and solved by better user research.

Tim Brown, the CEO of the innovation and design firm IDEO, sums it up nicely:

“Empathy is at the heart of design. Without the understanding of what others see, feel, and experience, design is a pointless task.”

? Bonus material Download User research checklist and a comparison table

Never forget – you are not your user.

You require proper user research to understand your user’s problems, pain points, needs, desires, feelings and behaviours.

Let’s start with the process!

Before you get in touch with your target users, you need to define why you are doing the research in the first place. 

Establish clear objectives and agree with your team on your exact goals – this will make it much easier to gain valuable insights. Otherwise, your findings will be all over the place.

Here are some sample questions that will help you to define your objectives:

• What do you want to uncover?
• What are the knowledge gaps that you need to fill?
• What is already working and what isn’t?
• Is there a problem that needs to be fixed? What is that problem?
• What will the research bring to the business and/or your customers?

Once you start answering questions like these, it’s time to make a list of objectives. These should be specific and concise .

Let’s say you are making a travel recommendation app. Your research goals could be:

• Understand the end-to-end process of how participants are currently making travel decisions.
• Uncover the different tools that participants are using to make travel decisions.
• Identify problems or barriers that they encounter when making travel decisions.

I suggest that you prioritize your objectives and create an Excel table. It will come in handy later.

Go ahead, create that fake persona

A useful exercise for you to do at this stage is to write down some hypotheses about your target users.

Ask yourself:

What do we think we understand about our users that is relevant to our business or product?

Yes, brainstorm the heck out of this persona, but keep it relevant to the topic at hand.

Here’s my empathy map and empathy map canvas to really help you flesh out your imaginary user.

Once you’re finished, research any and every statement , need and desire with real people.

It’s a simple yet effective way to create questions for some of the research methods that you’ll be using.

However, you need to be prepared to throw some of your assumptions out of the window. If you think this persona may affect your bias, don’t bother with hypotheses and dive straight into research with a completely open mind.

Alright, you have your research goals. Now let’s see how you can reach them.

Here’s the main question you should be asking yourself at this step in the process:

Based on our time and manpower, what methods should we select?

It’s essential to pick the right method at the right time . I’ll delve into more details on specific methods in Step #4. For now, let’s take a quick look at what categories you can choose from.

Qualitative research tells you ‘why’ something occurs. It tells you the reasons behind the behavior, the problem or the desire. It answers questions like: “ Why do you prefer using app X instead of other similar apps?” or “What’s the hardest part about being a sales manager? Why?” .

Qualitative data comes in the form of actual insights and it’s fairly easy to understand.

Most of the methods we’ll look at in Step #4 are qualitative methods.

Quantitative research helps you to understand what is happening by providing different metrics.

It answers questions such as “What percentage of users left their shopping cart without completing the purchase?” or “Is it better to have a big or small subscription button?”.

Most quantitative methods come in handy when testing your product, but not so much when you’re researching your users. This is because they don’t tell you why particular trends or patterns occur.

There is a big difference between “what people do” and “what people say”.

As their names imply, attitudinal research is used to understand or measure attitudes and beliefs, whereas behavioral research is used to measure and observe behaviors.

Here’s a practical landscape that will help you choose the best methods for you. If it doesn’t make sense now, return to it once you’ve finished the guide and you’ll have a much better understanding.

Source: Nielsen Norman Group

I’ll give you my own suggestions and tips about the most common and useful methods in Step #4 – Conducting research.

In general, if your objectives are specific enough, it shouldn’t be too hard to see which methods will help you achieve them.

Remember that Excel table? Choose a method or two that will fulfill each objective and type it in the column beside it.

It won’t always be possible to carry out everything you’ve written down. If this is the case, go with the method(s) that will give you most of the answers. With your table, it will be easy to pick and choose the most effective options for you.

Onto the next step!

This stage is all about channeling your inner Sherlock and finding the people with the secret intel for your product’s success.

Consider your niche, your objectives and your methods – this should give you a general idea of the group or groups you want to talk to and research further.

Here’s my advice for most cases.

If you’re building something from the ground up, the best participants might be:

• People you assume face the problem that your product aims to solve
• Your competitors’ customers

If you are developing something or solving a problem for an existing product, you should also take a look at:

• Advocates and super-users
• Customers who have recently churned
• Users who tried to sign up or buy but decided not to commit

There are plenty of ways to bring on participants, and you can get creative so long as you keep your desired target group in mind.

You can recruit them online – via social media, online forums or niche community sites.

You can publish an ad with requirements and offer some kind of incentive.

You can always use a recruitment agency, too. This can be costly, but it’s also efficient.

If you have a user database and are changing or improving your product, you can find your participants in there. Make sure that you contact plenty of your existing users, as most of them won’t respond.

You can even ask your friends to recommend the right kind of people who you wouldn’t otherwise know.

With that said, you should always be wary of including friends in your research . Sure, they’re the easiest people to reach, but your friendship can (and probably will) get in the way of obtaining honest answers. There are plenty of horror stories about people validating their “brilliant” ideas with their friends, only to lose a fortune in the future. Only consider them if you are 100% sure that they will speak their mind no matter what.

That depends on the method. If you’re not holding a massive online survey, you can usually start with 5 people in each segment . That’s enough to get the most important unique insights. You can then assess the situation and decide whether or not you need to expand your research.

Finally! Let’s go through some of the more common methods you’ll be using, including their pros and cons, some pro tips, and when you should use them.

Engaging in one-on-one discussions with users enables you to acquire detailed information about a user’s attitudes, desires, and experiences. Individual concerns and misunderstandings can be directly addressed and cleared up on the spot.

Interviews are time-consuming, especially on a per participant basis. You have to prepare for them, conduct them, analyze them and sometimes even transcribe them. They also limit your sample size, which can be problematic. The quality of your data will depend on the ability of your interviewer, and hiring an expert can be expensive.

• Prepare questions that stick to your main topics. Include follow-up questions for when you want to dig deeper into certain areas.
• Record the interview . Don’t rely on your notes. You don’t want to interrupt the flow of the interview by furiously scribbling down your answers, and you’ll need the recording for any potential in-depth analysis later on.
• Conduct at least one trial run of the interview to see if everything flows and feels right. Create a “playbook” on how the interview should move along and update it with your findings.
• If you are not comfortable with interviewing people, let someone else do it or hire an expert interviewer. You want to make people feel like they are talking to someone they know, rather than actually being interviewed. In my experience, psychologists are a great choice for an interviewer.

Interviews are not really time-sensitive, as long as you do them before the development process.

However, they can be a great supplement to online surveys and vice-versa. Conducting an interview beforehand helps you to create a more focused and relevant survey, while conducting an interview afterwards helps you to explain the survey answers.

Surveys are generally conducted online, which means that it’s possible to gather a lot of data in a very short time for a very low price . Surveys are usually anonymous, so users are often more honest in their responses.

It’s more difficult to get a representative sample because it’s tough to control who takes part in the survey – especially if you post it across social media channels or general forums. Surveys are quite rigid and if you don’t account for all possible answers, you might be missing out on valuable data. You have to be very careful when choosing your questions – poorly worded or leading ones can negatively influence how users respond. Length can also be an issue, as many people hate taking long surveys.

• Keep your surveys brief , particularly if participants won’t be compensated for their time. Only focus on what is truly important.
• Make sure that the questions can be easily understood. Unclear or ambiguous questions result in data on which you can’t depend. Keep the wording as simple as possible.
• Avoid using leading questions. Don’t ask questions that assume something, such as “What do you dislike about X?”. Replace this with “What’s your experience with X?”.
• Find engaged, niche online communities that fit your user profile. You’ll get more relevant data from these.

Similar to interviews. It depends on whether you want to use the survey as a preliminary method, or if you want a lot of answers to a few, very focused questions.

Focus Groups

Focus groups are moderated discussions with around 5 to 10 participants, the intention of which is to gain insight into the individuals’ attitudes, ideas and desires.

As focus groups include multiple people, they can quickly reveal the desires, experiences, and attitudes of your target audience . They are helpful when you require a lot of specific information in a short amount of time. When conducted correctly, they can act like interviews on steroids.

Focus groups can be tough to schedule and manage. If the moderator isn’t experienced, the discussion can quickly go off-topic. There might be an alpha participant that dictates the general opinion, and because it’s not one-on-one, people won’t always speak their mind.

• Find an experienced moderator who will lead the discussion. Having another person observing and taking notes is also highly recommended, as he or she can emphasize actionable insights and catch non-verbal clues that would otherwise be missed.
• Define the scope of your research . What questions will you ask? How in-depth do you want to go with the answers? How long do you want each discussion to last? This will determine how many people and groups should be tested.
• If possible, recruit potential or existing users who are likely to provide good feedback, yet will still allow others to speak their mind. You won’t know the participants most of the time, so having an experienced moderator is crucial.

Focus groups work best when you have a few clear topics that you want to focus on.

Competitive Analysis

A competitive analysis highlights the strengths and weaknesses of existing products . It explores how successful competitors act on the market. It gives you a solid basis for other user research methods and can also uncover business opportunities. It helps you to define your competitive advantage , as well as identify different user types.

A competitive analysis can tell you what exists, but not why it exists. You may collect a long feature list, but you won’t know which features are valued most by users and which they don’t use at all. In many cases, it’s impossible to tell how well a product is doing, which makes the data less useful. It also has limited use if you’re creating something that’s relatively new to the market.

• Create a list or table of information that you want to gather – market share, prices, features, visual design language, content, etc.
• Don’t let it go stale. Update it as the market changes so that you include new competitors.
• If you find something really interesting but don’t know the reason behind it, conduct research among your competitor’s users .
• After concluding your initial user research, go over the findings of your competitive analysis to see if you’ve discovered anything that’s missing on the market .

It can be a great first method, especially if you’re likely to talk to users of your competitors’ products

Field Studies

Field studies are research activities that take place in the user’s context, rather than at your company or office. Some are purely observational (the researcher is a “fly on the wall”), others are field interviews, and some act as a demonstration of pain points in existing systems.

You really get to see the big picture –  field studies allow you to gain insights that will fundamentally change your product design . You see what people actually do instead of what they say they do. A field study can explain problems and behaviours that you don’t understand better than any other method.

It’s the most time-consuming and expensive method. The results rely on the observer more than any of the other options. It’s not appropriate for products that are used in rare and specific situations.

• Establish clear objectives. Always remember why you are doing the research. Field studies can provide a variety of insights and sometimes it can be hard to stay focused. This is especially true if you are participating in the observed activity.
• Be patient. Observation might take some time. If you rush, you might end up with biased results.
• Keep an open mind and don’t ask leading questions. Be prepared to abandon your preconceptions, assumptions and beliefs. When interviewing people, try to leave any predispositions or biases at the door.
• Be warm but professional. If you conduct interviews or participate in an activity, you won’t want people around you to feel awkward or tense. Instead, you’ll want to observe how they act naturally.

Use a field study when no other method will do or if it becomes clear that you don’t really understand your user. If needed, you should conduct this as soon as possible – it can lead to monumental changes.

We started with a user persona and we’ll finish on this topic, too. But yours will be backed by research 😉

A persona outlines your ideal user in a concise and understandable way. It includes the most important insights that you’ve discovered. It makes it easier to design products around your actual users and speak their language. It’s a great way to familiarize new people on your team with your target market.

A persona is only as good as the user research behind it. Many companies create a “should be” persona instead of an actual one. Not only can such a persona be useless, it can also be misleading.

• Keep personas brief. Avoid adding unnecessary details and omit information that does not aid your decision making. If a persona document is too long, it simply won’t be used.
• Make personas specific and realistic. Avoid exaggerating and include enough detail to help you find real people that represent your ideal user.

Create these after you’ve carried out all of the initial user research. Compile your findings and create a persona that will guide your development process.

Now you know who you are creating your product for – you’ve identified their problems, needs and desires. You’ve laid the groundwork, so now it’s time to design a product that will blow your target user away! But that’s a topic for a whole separate guide, one that will take you through the process of product development and testing 😉

PS. Don’t forget -> Here is your ? User Research Checklist and comparison table

About the author

Oh hey, I’m Romina Kavcic

I am a Design Strategist who holds a Master of Business Administration. I have 14+ years of career experience in design work and consulting across both tech startups and several marquee tech unicorns such as Stellar.org, Outfit7, Databox, Xamarin, Chipolo, Singularity.NET, etc. I currently advise, coach and consult with companies on design strategy & management, visual design and user experience. My work has been published on Forbes, Hackernoon, Blockgeeks, Newsbtc, Bizjournals, and featured on Apple iTunes Store.

More about me  *  Let’s connect on Linkedin   *  Let’s connect on Twitter

Explore more

Username or email address  *

Password  *

Remember me Log in

Lost your password?

Insert/edit link

Enter the destination URL

Or link to existing content

• Reviews / Why join our community?
• For companies
• Frequently asked questions

UX Research

What is ux research.

UX (user experience) research is the systematic study of target users and their requirements, to add realistic contexts and insights to design processes. UX researchers adopt various methods to uncover problems and design opportunities. Doing so, they reveal valuable information which can be fed into the design process.

See why UX research is a critical part of the UX design process.

• Transcript loading…

UX Research is about Finding Insights to Guide Successful Designs

When you do UX research, you’ll be better able to give users the best solutions—because you can discover exactly what they need. You can apply UX research at any stage of the design process. UX researchers often begin with qualitative measures, to determine users’ motivations and needs . Later, they might use quantitative measures to test their results . To do UX research well, you must take a structured approach when you gather data from your users. It’s vital to use methods that 1) are right for the purpose of your research and 2) will give you the clearest information. Then, you can interpret your findings so you can build valuable insights into your design .

“I get very uncomfortable when someone makes a design decision without customer contact.” – Dan Ritzenthaler, Senior Product Designer at HubSpot

We can divide UX research into two subsets:

Qualitative research – Using methods such as interviews and ethnographic field studies, you work to get an in-depth understanding of why users do what they do (e.g., why they missed a call to action, why they feel how they do about a website). For example, you can do user interviews with a small number of users and ask open-ended questions to get personal insights into their exercise habits. Another aspect of qualitative research is usability testing , to monitor (e.g.) users’ stress responses. You should do qualitative research carefully. As it involves collecting non-numerical data (e.g., opinions, motivations), there’s a risk that your personal opinions will influence findings.

Quantitative research – Using more-structured methods (e.g., surveys, analytics), you gather measurable data about what users do and test assumptions you drew from qualitative research. For example, you can give users an online survey to answer questions about their exercise habits (e.g., “How many hours do you work out per week?”). With this data, you can discover patterns among a large user group. If you have a large enough sample of representative test users, you’ll have a more statistically reliable way of assessing the population of target users. Whatever the method, with careful research design you can gather objective data that’s unbiased by your presence, personality or assumptions. However, quantitative data alone can’t reveal deeper human insights.

We can additionally divide UX research into two approaches:

Attitudinal – you listen to what users say—e.g., in interviews.

Behavioral – you see what users do through observational studies.

When you use a mix of both quantitative and qualitative research as well as a mix of attitudinal and behavioral approaches, you can usually get the clearest view of a design problem.

© Interaction Design Foundation, CC BY-SA 4.0

Use UX Research Methods throughout Development

The Nielsen Norman Group—an industry-leading UX consulting organization—identifies appropriate UX research methods which you can use during a project’s four stages . Key methods are:

Discover – Determine what is relevant for users.

Contextual inquiries – Interview suitable users in their own environment to see how they perform the task/s in question.

Diary studies – Have users record their daily interactions with a design or log their performance of activities.

Explore – Examine how to address all users’ needs.

Card sorting – Write words and phrases on cards; then let participants organize them in the most meaningful way and label categories to ensure that your design is structured in a logical way.

Customer journey maps – Create user journeys to expose potential pitfalls and crucial moments.

Test – Evaluate your designs.

Usability testing – Ensure your design is easy to use.

Accessibility evaluations – Test your design to ensure it’s accessible to everyone.

Listen – Put issues in perspective, find any new problems and notice trends.

Surveys/Questionnaires – Use these to track how users’ feel about your product.

Analytics – Collect analytics/metrics to chart (e.g.) website traffic and build reports.

• Copyright holder: Unsplash. Copyright terms and license: CCO Public Domain. Link: https://pixabay.com/en/clay-hands-sculpting-art-69...
• Copyright holder: Unsplash. Copyright terms and license: CCO Public Domain. Link: https://www.pexels.com/photo/man-in-black-shirt-an...
• Copyright holder: Indecent Proposer. Copyright terms and license: CC BY-NC 2.0 Link: https://www.flickr.com/photos/indecent_proposal/14...
• Copyright holder: Anna Langova. Copyright terms and license: CC0 1.0 Link: http://www.publicdomainpictures.net/view-image.php...
• Copyright holder: Conmongt. Copyright terms and license: CC0 Public Domain Link: https://pixabay.com/en/hourglass-time-time-lapse-clock-1623517/

Whichever UX research method you choose, you need to consider the pros and cons of the different techniques . For instance, card sorting is cheap and easy, but you may find it time-consuming when it comes to analysis. Also, it might not give you in-depth contextual meaning. Another constraint is your available resources , which will dictate when, how much and which type of UX research you can do. So, decide carefully on the most relevant method/s for your research . Moreover, involve stakeholders from your organization early on . They can reveal valuable UX insights and help keep your research in line with business goals. Remember, a design team values UX research as a way to validate its assumptions about users in the field , slash the cost of the best deliverables and keep products in high demand —ahead of competitors’.

User research methods have different pros and cons,and vary from observations of users in context to controlled experiments in lab settings.

Learn More about UX Research

For a thorough grasp of UX research, take our course here: User Research – Methods and Best Practices

Read an extensive range of UX research considerations, discussed in Smashing Magazine: A Comprehensive Guide To UX Research

See the Nielsen Norman Group’s list of UX research tips: UX Research Cheat Sheet

Here’s a handy, example-rich catalog of UX research tools: 43 UX research tools for optimizing your product

Questions related to UX Research

UX research is a good career for those who enjoy working with a team and have strong communication skills. As a researcher, you play a crucial role in helping your team understand users and deliver valuable and delightful experiences. You will find a UX research career appealing if you enjoy scientific and creative pursuits. 

Start exploring this career option; see the User Researcher Learning Path .

Studies suggest that companies are also willing to pay well for research roles. The average salary for a UX researcher ranges from $92,000 to $146,000 per year.

In smaller companies, user research may be one of the responsibilities of a generalist UX designer. How much can your salary vary based on your region? Find out in UI & UX Designer Salaries: How Much Can I Earn .

Research is one part of the overall UX design process. UX research helps inform the design strategy and decisions made at every step of the design process. In smaller teams, a generalist designer may end up conducting research.

A UX researcher aims to understand users and their needs. A UX designer seeks to create a product that meets those needs.

A UX researcher gathers information. A UX designer uses that information to create a user-friendly and visually appealing product.

Learn more about the relationship between UX research and UX design in the course:

User Experience: The Beginner’s Guide

If we consider a very broad definition of UX, then all user research is UX research.

However, in practice, there is a subtle difference between user research and UX research. While both involve understanding people, user research can involve users in any kind of research question, and some questions may not be that directly connected to user experience.

For example, you might do user research relating to a customer’s experience in relation to pricing, delivery or the experience across multiple channels.

Common UX research methods are usability testing, A/B testing, surveys, card sorting, user interviews, usage analytics and ethnographic research. Each method has its pros and cons and is useful in different scenarios. Hence, you must select the appropriate research method for the research question and target audience. Learn more about these methods in 7 Great, Tried and Tested UX Research Techniques .

Get started with user research. Download the User Research template bundle .

For a deep dive into usability testing—the most common research method, take the course Conducting Usability Testing .

Having a degree in a related field can give you an advantage. However, you don’t need a specific degree to become a UX researcher. A combination of relevant education, practical experience, and continuous learning can help you pursue a career in UX research. Many UX researchers come from diverse educational backgrounds, including psychology, statistics, human-computer interaction, information systems, design and anthropology.

Some employers may prefer candidates with at least a bachelor’s degree. However, it does not have to be in a UX-related field. There are relatively fewer degrees that focus solely on user research.

Data-Driven Design: Quantitative Research for UX

User Research – Methods and Best Practices

Every research project will vary. However, there are some common steps in conducting research, no matter which method or tool you decide to use: 

Define the research question

Select the appropriate research method

Recruit participants

Conduct the research

Analyze the data

Present the findings

You can choose from various UX research tools . Your choice depends on your research question, how you're researching, the size of your organization, and your project. For instance:

Survey tools such as Typeform and Google Forms.

Card sorting tools such as Maze and UXtweak.

Heatmap tools such as HotJar and CrazyEgg

Usability testing (through first-click testing and tree-testing) tools such as Optimal Workshop and Loop 11

Diagramming applications such as Miro and Whimsical to analyze qualitative data through affinity diagramming.

Spreadsheet tools such as Google Sheets and Microsoft Excel for quantitative data analysis

Interface design and prototyping tools like Figma, Adobe XD, Sketch and Marvel to conduct usability testing.

Presentation tools such as Keynote, Google Slides and Microsoft PowerPoint.

Many of these tools offer additional features you can leverage for multiple purposes. To understand how you can make the most of these tools, we recommend these courses:

There are relatively fewer degrees that focus solely on user research.

While there are no universal research case study formats, here’s one suggested outline: 

An overview of the project: Include the problem statement, goals and objectives.

The research methods and methodology: For example, surveys, interviews, or usability testing).

Research findings

The design process: How the research findings led to design decisions.

Impact of design decisions on users and the business: Include metrics such as conversion and error rates to demonstrate the impact.

Optionally, include notes on what you learned and how you can improve the process in the future.

Learn how to showcase your portfolio to wow your future employer/client in the How to Create a UX Portfolio course.

While AI can help automate tasks and help UX researchers, it will not completely replace them. AI lacks the creativity and empathy that human designers bring to the table.

Human researchers are better at understanding the nuances of human behavior and emotions. They can also think outside the box and develop creative solutions that AI cannot. So, AI can help researchers be more efficient and effective through data analysis, smart suggestions and automation. But it cannot replace them.

Watch AI-Powered UX Design: How to Elevate Your UX Career to learn how you can work with AI.

Agile teams often struggle to incorporate user research in their workflows due to the time pressure of short sprints. However, that doesn’t mean agile teams can’t conduct research. Instead of seeing research as one big project, teams can break it into bite-sized chunks. Researchers regularly conduct research and share their findings in every sprint.

Researchers can involve engineers and other stakeholders in decision-making to give everyone the context they need to make better decisions. When engineers participate in the decision-making process, they can ensure that the design will be technically feasible. There will also be lower chances of errors when the team actually builds the feature. Here’s more on how to make research a team effort .

For more on bite-sized research, see this Master Class: Continuous Product Discovery: The What and Why

For more practical tips and methods to work in an agile environment, take our Agile Methods for UX Design course.

User research is very important in designing products people will want and use. It helps us avoid designing based on what we think instead of what users actually want.

UX research helps designers understand their users’ needs, behaviors, attitudes and how they interact with a product or service. Research helps identify usability problems, gather feedback on design concepts, and validate design decisions. This ultimately benefits businesses by improving the product, brand reputation and loyalty. A good user experience provides a competitive edge and reduces the risk of product failure.

Learn more about the importance of user research in the design process in these courses:

Design Thinking: The Ultimate Guide

Literature on UX Research

Here’s the entire UX literature on UX Research by the Interaction Design Foundation, collated in one place:

Learn more about UX Research

Take a deep dive into UX Research with our course User Research – Methods and Best Practices .

How do you plan to design a product or service that your users will love , if you don't know what they want in the first place? As a user experience designer, you shouldn't leave it to chance to design something outstanding; you should make the effort to understand your users and build on that knowledge from the outset. User research is the way to do this, and it can therefore be thought of as the largest part of user experience design .

In fact, user research is often the first step of a UX design process—after all, you cannot begin to design a product or service without first understanding what your users want! As you gain the skills required, and learn about the best practices in user research, you’ll get first-hand knowledge of your users and be able to design the optimal product—one that’s truly relevant for your users and, subsequently, outperforms your competitors’ .

This course will give you insights into the most essential qualitative research methods around and will teach you how to put them into practice in your design work. You’ll also have the opportunity to embark on three practical projects where you can apply what you’ve learned to carry out user research in the real world . You’ll learn details about how to plan user research projects and fit them into your own work processes in a way that maximizes the impact your research can have on your designs. On top of that, you’ll gain practice with different methods that will help you analyze the results of your research and communicate your findings to your clients and stakeholders—workshops, user journeys and personas, just to name a few!

By the end of the course, you’ll have not only a Course Certificate but also three case studies to add to your portfolio. And remember, a portfolio with engaging case studies is invaluable if you are looking to break into a career in UX design or user research!

We believe you should learn from the best, so we’ve gathered a team of experts to help teach this course alongside our own course instructors. That means you’ll meet a new instructor in each of the lessons on research methods who is an expert in their field—we hope you enjoy what they have in store for you!

All open-source articles on UX Research

7 great, tried and tested ux research techniques.

• 1.2k shares
• 3 years ago

The Ultimate Guide to Understanding UX Roles and Which One You Should Go For

Shadowing in User Research - Do You See What They See?

Contextual Interviews and How to Handle Them

15 Guiding Principles for UX Researchers

Ethnography

Porter’s 5 Forces Model - Design in Context, Understand the Market

• 7 years ago

Ideas for Conducting UX Research with Children

Laddering Questions Drilling Down Deep and Moving Sideways in UX Research

Action Research

4 common pitfalls in usability testing and how to avoid them to get more honest feedback.

Confirmation Bias – It’s Not What We Think We Know That Counts

User Research Methods for Mobile UX

The Top UX Design Books You Need to Read in 2024: Beginner to Expert

6 Tips for Better International UX Research

Team Research

• 2 years ago

Adding Quality to Your Design Research with an SSQS Checklist

• 8 years ago

Common UX Research Interview Questions

How to Fit Quantitative Research into the Project Lifecycle

The Best Free UX Design Courses in 2024

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this page , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share the knowledge!

Share this content on:

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this page.

New to UX Design? We’re Giving You a Free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

Integrations

What's new?

Prototype Testing

Live Website Testing

Feedback Surveys

Interview Studies

Card Sorting

Tree Testing

In-Product Prompts

Participant Management

Automated Reports

Templates Gallery

Choose from our library of pre-built mazes to copy, customize, and share with your own users

Browse all templates

Financial Services

Tech & Software

Product Designers

Product Managers

User Researchers

By use case

Concept & Idea Validation

Wireframe & Usability Test

Content & Copy Testing

Feedback & Satisfaction

Content Hub

Educational resources for product, research and design teams

Explore all resources

Question Bank

Research Maturity Model

Guides & Reports

Help Center

Future of User Research Report

The Optimal Path Podcast

Maze Guides | Resources Hub

What is UX Research: The Ultimate Guide for UX Researchers

0% complete

11 Key UX research methods: How and when to use them

After defining your objectives and planning your research framework, it’s time to choose the research technique that will best serve your project's goals and yield the right insights. While user research is often treated as an afterthought, it should inform every design decision. In this chapter, we walk you through the most common research methods and help you choose the right one for you.

What are UX research methods?

A UX research method is a way of generating insights about your users, their behavior, motivations, and needs. You can use methods like user interviews, surveys, focus groups, card sorting, usability testing to identify user challenges and turn them into opportunities to improve the user experience.

More of a visual learner? Check out this video for a speedy rundown. If you’re ready to get stuck in, jump straight to our full breakdown .

The most common types of user research

First, let’s talk about the types of UX research. Every individual research method falls under these types, which reflect different goals and objectives for conducting research.

Here’s a quick overview:

Qualitative vs. quantitative

All research methods are either quantitative or qualitative . Qualitative research focuses on capturing subjective insights into users' experiences. It aims to understand the underlying reasons, motivations, and behaviors of individuals. Quantitative research, on the other hand, involves collecting and analyzing numerical data to identify patterns, trends, and significance. It aims to quantify user behaviors, preferences, and attitudes, allowing for generalizations and statistical insights.

Qualitative research also typically involves a smaller sample size than quantitative research (40 participants, as recommended by Nielsen Norman Group ).

Attitudinal vs. behavioral

Attitudinal research is about understanding users' attitudes, perceptions, and beliefs. It delves into the 'why' behind user decisions and actions. It often involves surveys or interviews where users are asked about their feelings, preferences, or perceptions towards a product or service. It's subjective in nature, aiming to capture people's emotions and opinions.

Behavioral research is about what users do rather than what they say they do or would do. This kind of research is often based on observation methods like usability testing, eye-tracking, or heat maps to understand user behavior.

Generative vs. evaluative

Generative research is all about generating new ideas, concepts, and insights to fuel the design process. You might run brainstorming sessions with groups of users, card sorting, and co-design sessions to inspire creativity and guide the development of user-centered solutions.

On the other hand, evaluative research focuses on assessing the usability, effectiveness, and overall quality of existing designs or prototypes. Once you’ve developed a prototype of your product, it's time to evaluate its strengths and weaknesses. You can compare different versions of a product design or feature through A/B testing—ensuring your UX design meets user needs and expectations.

Remove the guesswork from product decisions

Collect both quantitative and qualitative insights from your customers and build truly user-centric products with Maze.

11 Best UX research methods and when to use them

There are various UX research techniques—each method serves a specific purpose and can provide unique insights into user behaviors and preferences. In this section, we’ll highlight the most common research techniques you need to know.

Read on for an at-a-glance table, and full breakdown of each method.

User interviews

User interviews are a qualitative research method that involves having open-ended and guided discussions with users to gather in-depth insights about their experiences, needs, motivations, and behaviors.

Typically, you would ask a few questions on a specific topic and analyze participants' answers. The results you get will depend on how well you form and ask questions, as well as follow up on participants’ answers.

“As a researcher, it's our responsibility to drive the user to their actual problems,” says Yuliya Martinavichene , User Experience Researcher at Zinio. She adds, “The narration of incidents can help you analyze a lot of hidden details with regard to user behavior.”

That’s why you should:

• Start with a wide context : Make sure that your questions don’t start with your product
• Ask questions that focus on the tasks that users are trying to complete
• Invest in analysis : Get transcripts done and share the findings with your team

Tanya Nativ , Design Researcher at Sketch recommends defining the goals and assumptions internally. “Our beliefs about our users’ behavior really help to structure good questions and get to the root of the problem and its solution,” she explains.

It's easy to be misunderstood if you don't have experience writing interview questions. You can get someone to review them for you or use our Question Bank of 350+ research questions .

When to conduct user interviews

This method is typically used at the start and end of your project. At the start of a project, you can establish a strong understanding of your target users, their perspectives, and the context in which they’ll interact with your product. By the end of your project, new user interviews—often with a different set of individuals—offer a litmus test for your product's usability and appeal, providing firsthand accounts of experiences, perceived strengths, and potential areas for refinement.

Field studies

Field studies are research activities that take place in the user’s environment rather than in your lab or office. They’re a great method for uncovering context, unknown motivations, or constraints that affect the user experience.

An advantage of field studies is observing people in their natural environment, giving you a glimpse at the context in which your product is used. It’s useful to understand the context in which users complete tasks, learn about their needs, and collect in-depth user stories.

When to conduct field studies

This method can be used at all stages of your project—two key times you may want to conduct field studies are:

• As part of the discovery and exploration stage to define direction and understand the context around when and how users interact with the product
• During usability testing, once you have a prototype, to evaluate the effectiveness of the solution or validate design assumptions in real-world contexts

3. Focus groups

A focus group is a qualitative research method that includes the study of a group of people, their beliefs, and opinions. It’s typically used for market research or gathering feedback on products and messaging.

Focus groups can help you better grasp:

• How users perceive your product
• What users believe are a product’s most important features
• What problems do users experience with the product

As with any qualitative research method, the quality of the data collected through focus groups is only as robust as the preparation. So, it’s important to prepare a UX research plan you can refer to during the discussion.

Here’s some things to consider:

• Write a script to guide the conversation
• Ask clear, open-ended questions focused on the topics you’re trying to learn about
• Include around five to ten participants to keep the sessions focused and organized

When to conduct focus groups

It’s easier to use this research technique when you're still formulating your concept, product, or service—to explore user preferences, gather initial reactions, and generate ideas. This is because, in the early stages, you have flexibility and can make significant changes without incurring high costs.

Another way some researchers employ focus groups is post-launch to gather feedback and identify potential improvements. However, you can also use other methods here which may be more effective for identifying usability issues. For example, a platform like Maze can provide detailed, actionable data about how users interact with your product. These quantitative results are a great accompaniment to the qualitative data gathered from your focus group.

4. Diary studies

Diary studies involve asking users to track their usage and thoughts on your product by keeping logs or diaries, taking photos, explaining their activities, and highlighting things that stood out to them.

“Diary studies are one of the few ways you can get a peek into how users interact with our product in a real-world scenario,” says Tanya.

A diary study helps you tell the story of how products and services fit into people’s daily lives, and the touch-points and channels they choose to complete their tasks.

There’s several key questions to consider before conducting diary research, from what kind of diary you want—freeform or structured, and digital or paper—to how often you want participants to log their thoughts.

• Open, ‘freeform’ diary: Users have more freedom to record what and when they like, but can also lead to missed opportunities to capture data users might overlook
• Closed, ‘structured; diary: Users follow a stricter entry-logging process and answer pre-set questions

Remember to determine the trigger: a signal that lets the participants know when they should log their feedback. Tanya breaks these triggers down into the following:

• Interval-contingent trigger : Participants fill out the diary at specific intervals such as one entry per day, or one entry per week
• Signal-contingent trigger : You tell the participant when to make an entry and how you would prefer them to communicate it to you as well as your preferred type of communication
• Event-contingent trigger : The participant makes an entry whenever a defined event occurs

When to conduct diary studies

Diary studies are often valuable when you need to deeply understand users' behaviors, routines, and pain points in real-life contexts. This could be when you're:

• Conceptualizing a new product or feature: Gain insights into user habits, needs, and frustrations to inspire your design
• Trying to enhance an existing product: Identify areas where users are having difficulties or where there are opportunities for better user engagement

Although surveys are primarily used for quantitative research, they can also provided qualitative data, depending on whether you use closed or open-ended questions:

• Closed-ended questions come with a predefined set of answers to choose from using formats like rating scales, rankings, or multiple choice. This results in quantitative data.
• Open-ended question s are typically open-text questions where test participants give their responses in a free-form style. This results in qualitative data.

Matthieu Dixte , Product Researcher at Maze, explains the benefit of surveys: “With open-ended questions, researchers get insight into respondents' opinions, experiences, and explanations in their own words. This helps explore nuances that quantitative data alone may not capture.”

So, how do you make sure you’re asking the right survey questions? Gregg Bernstein , UX Researcher at Signal, says that when planning online surveys, it’s best to avoid questions that begin with “How likely are you to…?” Instead, Gregg says asking questions that start with “Have you ever… ?” will prompt users to give more specific and measurable answers.

Make sure your questions:

• Are easy to understand
• Don't guide participants towards a particular answer
• Include both closed-ended and open-ended questions
• Respect users and their privacy
• Are consistent in terms of format

To learn more about survey design, check out this guide .

When to conduct surveys

While surveys can be used at all stages of project development, and are ideal for continuous product discovery , the specific timing and purpose may vary depending on the research goals. For example, you can run surveys at:

• Conceptualization phase to gather preliminary data, and identify patterns, trends, or potential user segments
• Post-launch or during iterative design cycles to gather feedback on user satisfaction, feature usage, or suggestions for improvements

6. Card sorting

Card sorting is an important step in creating an intuitive information architecture (IA) and user experience. It’s also a great technique to generate ideas, naming conventions, or simply see how users understand topics.

In this UX research method, participants are presented with cards featuring different topics or information, and tasked with grouping the cards into categories that make sense to them.

There are three types of card sorting:

• Open card sorting: Participants organize topics into categories that make sense to them and name those categories, thus generating new ideas and names
• Hybrid card sorting: Participants can sort cards into predefined categories, but also have the option to create their own categories
• Closed card sorting: Participants are given predefined categories and asked to sort the items into the available groups

You can run a card sorting session using physical index cards or digitally with a UX research tool like Maze to simulate the drag-and-drop activity of dividing cards into groups. Running digital card sorting is ideal for any type of card sort, and moderated or unmoderated sessions.

Read more about card sorting and learn how to run a card sorting session here .

When to conduct card sorting

Card sorting isn’t limited to a single stage of design or development—it can be employed anytime you need to explore how users categorize or perceive information. For example, you may want to use card sorting if you need to:

• Understand how users perceive ideas
• Evaluate and prioritize potential solutions
• Generate name ideas and understand naming conventions
• Learn how users expect navigation to work
• Decide how to group content on a new or existing site
• Restructure information architecture

7. Tree testing

During tree testing a text-only version of the site is given to your participants, who are asked to complete a series of tasks requiring them to locate items on the app or website.

The data collected from a tree test helps you understand where users intuitively navigate first, and is an effective way to assess the findability, labeling, and information architecture of a product.

We recommend keeping these sessions short, ranging from 15 to 20 minutes, and asking participants to complete no more than ten tasks. This helps ensure participants remain focused and engaged, leading to more reliable and accurate data, and avoiding fatigue.

If you’re using a platform like Maze to run remote testing, you can easily recruit participants based on various demographic filters, including industry and country. This way, you can uncover a broader range of user preferences, ensuring a more comprehensive understanding of your target audience.

To learn more about tree testing, check out this chapter .

When to conduct tree testing

Tree testing is often done at an early stage in the design or redesign process. That’s because it’s more cost-effective to address errors at the start of a project—rather than making changes later in the development process or after launch.

However, it can be helpful to employ tree testing as a method when adding new features, particularly alongside card sorting.

While tree testing and card sorting can both help you with categorizing the content on a website, it’s important to note that they each approach this from a different angle and are used at different stages during the research process. Ideally, you should use the two in tandem: card sorting is recommended when defining and testing a new website architecture, while tree testing is meant to help you test how the navigation performs with users.

8. Usability testing

Usability testing evaluates your product with people by getting them to complete tasks while you observe and note their interactions (either during or after the test). The goal of conducting usability testing is to understand if your design is intuitive and easy to use. A sign of success is if users can easily accomplish their goals and complete tasks with your product.

There are various usability testing methods that you can use, such as moderated vs. unmoderated or qualitative vs. quantitative —and selecting the right one depends on your research goals, resources, and timeline.

Usability testing is usually performed with functional mid or hi-fi prototypes . If you have a Figma, InVision, Sketch, or prototype ready, you can import it into a platform like Maze and start testing your design with users immediately.

The tasks you create for usability tests should be:

• Realistic, and describe a scenario
• Actionable, and use action verbs (create, sign up, buy, etc)

Be mindful of using leading words such as ‘click here’ or ‘go to that page’ in your tasks. These instructions bias the results by helping users complete their tasks—something that doesn’t happen in real life.

Product tip ✨

With Maze, you can test your prototype and live website with real users to filter out cognitive biases, and gather actionable insights that fuel product decisions.

When to conduct usability testing

To inform your design decisions, you should do usability testing early and often in the process . Here are some guidelines to help you decide when to do usability testing:

• Before you start designing
• Once you have a wireframe or prototype
• Prior to the launch of the product
• At regular intervals after launch

To learn more about usability testing, check out our complete guide to usability testing .

9. Five-second testing

In five-second testing , participants are (unsurprisingly) given five seconds to view an image like a design or web page, and then they’re asked questions about the design to gauge their first impressions.

Why five seconds? According to data , 55% of visitors spend less than 15 seconds on a website, so it;s essential to grab someone’s attention in the first few seconds of their visit. With a five-second test, you can quickly determine what information users perceive and their impressions during the first five seconds of viewing a design.

Product tip 💡

And if you’re using Maze, you can simply upload an image of the screen you want to test, or browse your prototype and select a screen. Plus, you can star individual comments and automatically add them to your report to share with stakeholders.

When to conduct five-second testing

Five-second testing is typically conducted in the early stages of the design process, specifically during initial concept testing or prototype development. This way, you can evaluate your design's initial impact and make early refinements or adjustments to ensure its effectiveness, before putting design to development.

To learn more, check out our chapter on five-second testing .

10. A/B testing

A/B testing , also known as split testing, compares two or more versions of a webpage, interface, or feature to determine which performs better regarding engagement, conversions, or other predefined metrics.

It involves randomly dividing users into different groups and giving each group a different version of the design element being tested. For example, let's say the primary call-to-action on the page is a button that says ‘buy now’.

You're considering making changes to its design to see if it can lead to higher conversions, so you create two versions:

• Version A : The original design with the ‘buy now’ button positioned below the product description—shown to group A
• Version B : A variation with the ‘buy now’ button now prominently displayed above the product description—shown to group B

Over a planned period, you measure metrics like click-through rates, add-to-cart rates, and actual purchases to assess the performance of each variation. You find that Group B had significantly higher click-through and conversion rates than Group A. This indicates that showing the button above the product description drove higher user engagement and conversions.

Check out our A/B testing guide for more in-depth examples and guidance on how to run these tests.

When to conduct A/B testing

A/B testing can be used at all stages of the design and development process—whenever you want to collect direct, quantitative data and confirm a suspicion, or settle a design debate. This iterative testing approach allows you to continually improve your website's performance and user experience based on data-driven insights.

11. Concept testing

Concept testing is a type of research that evaluates the feasibility, appeal, and potential success of a new product before you build it. It centers the user in the ideation process, using UX research methods like A/B testing, surveys, and customer interviews.

There’s no one way to run a concept test—you can opt for concept testing surveys, interviews, focus groups, or any other method that gets qualitative data on your concept.

*Dive into our complete guide to concept testing for more tips and tricks on getting started. *

When to conduct concept testing

Concept testing helps gauge your audience’s interest, understanding, and likelihood-to-purchase, before committing time and resources to a concept. However, it can also be useful further down the product development line—such as when defining marketing messaging or just before launching.

Which is the best UX research type?

The best research type varies depending on your project; what your objectives are, and what stage you’re in. Ultimately, the ideal type of research is one which provides the insights required, using the available resources.

For example, if you're at the early ideation or product discovery stage, generative research methods can help you generate new ideas, understand user needs, and explore possibilities. As you move to the design and development phase, evaluative research methods and quantitative data become crucial.

Discover the UX research trends shaping the future of the industry and why the best results come from a combination of different research methods.

How to choose the right user experience research method

In an ideal world, a combination of all the insights you gain from multiple types of user research methods would guide every design decision. In practice, this can be hard to execute due to resources.

Sometimes the right methodology is the one you can get buy-in, budget, and time for.

Gregg Bernstein , UX Researcher at Signal

UX research tools can help streamline the research process, making regular testing and application of diverse methods more accessible—so you always keep the user at the center of your design process. Some other key tips to remember when choosing your method are:

Define the goals and problems

A good way to inform your choice of user experience research method is to start by considering your goals. You might want to browse UX research templates or read about examples of research.

Michael Margolis , UX Research Partner at Google Ventures, recommends answering questions like:

• “What do your users need?”
• “What are your users struggling with?”
• “How can you help your users?”

Understand the design process stage

If your team is very early in product development, generative research —like field studies—make sense. If you need to test design mockups or a prototype, evaluative research methods—such as usability testing—will work best.

This is something they’re big on at Sketch, as we heard from Design Researcher, Tanya Nativ. She says, “In the discovery phase, we focus on user interviews and contextual inquiries. The testing phase is more about dogfooding, concept testing, and usability testing. Once a feature has been launched, it’s about ongoing listening.”

Consider the type of insights required

If you're looking for rich, qualitative data that delves into user behaviors, motivations, and emotions, then methods like user interviews or field studies are ideal. They’ll help you uncover the ‘why’ behind user actions.

On the other hand, if you need to gather quantitative data to measure user satisfaction or compare different design variations, methods like surveys or A/B testing are more suitable. These methods will help you get hard numbers and concrete data on preferences and behavior.

*Discover the UX research trends shaping the future of the industry and why the best results come from a combination of different research methods. *

Build a deeper understanding of your users with UX research

Think of UX research methods as building blocks that work together to create a well-rounded understanding of your users. Each method brings its own unique strengths, whether it's human empathy from user interviews or the vast data from surveys.

But it's not just about choosing the right UX research methods; the research platform you use is equally important. You need a platform that empowers your team to collect data, analyze, and collaborate seamlessly.

Simplifying product research is simple with Maze. From tree testing to card sorting, prototype testing to user interview analysis—Maze makes getting actionable insights easy, whatever method you opt for.

Meanwhile, if you want to know more about testing methods, head on to the next chapter all about tree testing .

Get valuable insights from real users

Conduct impactful UX research with Maze and improve your product experience and customer satisfaction.

Frequently asked questions

How do you choose the right UX research method?

Choosing the right research method depends on your goals. Some key things to consider are:

• The feature/product you’re testing
• The type of data you’re looking for
• The design stage
• The time and resources you have available

What is the best UX research method?

The best research method is the one you have the time, resources, and budget for that meets your specific needs and goals. Most research tools, like Maze, will accommodate a variety of UX research and testing techniques.

When to use which user experience research method?

Selecting which user research method to use—if budget and resources aren’t a factor—depends on your goals. UX research methods provide different types of data:

• Qualitative vs quantitative
• Attitudinal vs behavioral
• Generative vs evaluative

Identify your goals, then choose a research method that gathers the user data you need.

Tree Testing: Your Guide to Improve Navigation and UX

Join thousands of product people at Insight Out Conf on April 11. Register free.

Insights hub solutions

Analyze data

Uncover deep customer insights with fast, powerful features, store insights, curate and manage insights in one searchable platform, scale research, unlock the potential of customer insights at enterprise scale.

Featured reads

Tips and tricks

Four ways Dovetail helps Product Managers master continuous product discovery

Product updates

Dovetail retro: our biggest releases from the past year

How to affinity map using the canvas

Events and videos

© Dovetail Research Pty. Ltd.

A guide to field studies

Last updated

18 April 2023

Reviewed by

Cathy Heath

Field studies allow researchers to observe and collect data in real-world settings. Unlike laboratory-based or traditional research methods, field studies enable researchers to investigate complex phenomena within their environment, providing a deeper understanding of the research context.

Researchers can use field studies to investigate a wide range of subjects, from the behavior of animals to the practices of businesses or the experiences of individuals in a particular setting.

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• What is a field study?

A field study is a research method that involves conducting observations and collecting data in a natural setting. This method includes observing, interviewing, and interacting with participants in their environment, such as a workplace, community, or natural habitat.

Field studies can take many forms, from ethnographic studies involving extended periods of observation and using an anthropological lens to shorter-term studies focusing on specific behaviors or events. Regardless of its form, a successful field study requires careful planning, preparation, and execution to ensure the data collected is valid and reliable.

• How to plan a field study

Planning a field study is a critical first step in ensuring successful research. Here are some steps to follow when preparing your field study:

1. Define your research question

When developing a good research question , you should make it clear, concise, and specific. It should also be open-ended, allowing for various possible answers rather than a simple yes or no response. Your research question should also be relevant to the broader field of study and contribute new knowledge to the existing literature.

Once you have a defined research question, identify the key variables you need to study and the data you need to collect. It might involve developing a hypothesis or research framework outlining the relationships between different variables and how you’ll measure them in your study.

2. Identify your research site

A research site is a location where you’ll conduct your study and collect data. Here are the types of research sites to consider when planning a field study:

Natural habitats: For environmental or ecological research, you may need to conduct your study in a natural habitat, such as a forest, wetland, or coral reef.

Communities : If your research relates to social or cultural factors, you may need to study a particular community, such as a neighborhood, village, or city.

Organizations : For questions relating to organizational behavior or management, your location will be in a business environment, like a nonprofit or government agency.

Events : If your research question relates to a particular event, you may need to conduct your study at that event, such as, at a protest, festival, or natural disaster.

Ensure your research site represents the population you're studying. For example, if you're exploring cultural beliefs, ensure the community represents the larger population and you have access to a diverse group of participants.

3. Determine your data collection methods

Choosing a suitable method will depend on the research question, the type of data needed, and the characteristics of the participants. Here are some commonly used data collection methods in field studies:

Interviews : You can collect data on people's experiences, perspectives, and attitudes. In some instances, you can use phone or online interviews.

Observations : This method involves watching and recording behaviors and interactions in a specific setting. 

Surveys : By using a survey , you can easily standardize and tailor the questions to provide answers for your research. Respondents can complete the survey in person, by mail, or online.

Document analysis : Organizational reports, letters, diaries, public records, policies, or social media posts can be analyzed to gain context. 

When selecting data collection methods, consider factors such as the availability of participants, the ethical considerations involved, and the resources needed to carry out each method. For example, conducting interviews may require more time and resources than administering a survey.

4. Obtain necessary permissions

Depending on the research location and the nature of the study, you may require permission from local authorities, organizations, or individuals before conducting your research. 

This process is vital when working with human or animal subjects and conducting research in sensitive or protected environments.

Here are some steps you can take to obtain the necessary permissions:

Identify the relevant authorities , including local governments, regulatory bodies, research institutions, or private organizations, to obtain permission for your research.

Reach out to the relevant authorities to explain the nature of your study. Be ready to hand out detailed information about your research. 

If you're conducting research with human participants, you must have their consent . You'll also need to ensure the participants have the right to withdraw from the study at any time.

Obtain necessary permits from regulatory bodies or local authorities. For example, if you're conducting research in a protected area, you may need a research permit from the relevant government agency.

The process of obtaining permissions can be time-consuming, and failure to obtain the necessary permits can lead to legal and ethical issues.

• Examples of field research

Researchers can apply field research to a wide range of disciplines and phenomena. Here are some examples of field research in different fields:

Anthropology : Anthropologists use field research methods to study different communities' social and cultural practices. For instance, an anthropologist might conduct participant observation in a remote community to understand their customs, beliefs, and practices.

Ecology : Ecologists use field research methods to learn the behavior of organisms and their interactions with the environment. For example, an ecologist might conduct field research on the behavior of birds in their natural habitat to understand their feeding habits, nesting patterns, and migration.

Sociology : Sociologists may use field research methods to study social behavior and interactions. For instance, a sociologist might conduct participant observation in a workplace to understand organizational culture and communication dynamics.

Geography : Geographers use field research methods to study different regions’ physical and human contexts. For example, a geographer might conduct field research on the impact of climate change on a particular ecosystem, such as a forest or wetland.

Psychology : Psychologists use field research methods to study human behavior in natural settings. For instance, a psychologist might conduct field research on the effects of stress on students in a school setting.

Education : Researchers studying education may use field research methods to study teaching and learning in real-world settings. For example, you could use field research to test the effectiveness of a new teaching method in a classroom setting.

By using field research methods, researchers can gain a deeper understanding of the complexities of the natural world, human behavior, and social interaction theory and how they affect each other.

• Advantages of field research

Field research has several advantages over other research methods, including:

Authenticity : Field research conducted in natural settings allows researchers to observe and study real-life phenomena as it happens. This authenticity enhances the validity and accuracy of the data collected.

Flexibility : Field research methods are flexible and adaptable to different research contexts. Researchers can adjust their strategies to meet the specific needs of their research questions and participants and uncover new insights as the research unfolds.

Rich data : Field research provides rich and detailed data, often including contextual information that’s difficult to capture through other research methods. This depth of knowledge allows for a more comprehensive and nuanced understanding of the research topic.

Novel insights : Field research can lead to discoveries that may not be possible with other research methods. Observing and studying phenomena in natural settings can provide unique perspectives and new understandings of complex issues.

Field research methods can enhance the quality and validity of research findings and lead to new insights and discoveries that may not be possible with other research methods.

• Disadvantages of field research

While field research has several advantages, there are also some disadvantages that researchers need to consider, including:

Time-consuming : Researchers need to spend time in the field, possibly weeks or months, which can be challenging, especially if the research site is remote or requires travel.

Cost : Conducting field research can be costly, especially if the research site is remote or requires specialized equipment or materials.

Reliance on participants : It may be challenging to recruit participants, and various factors, such as personal circumstances, attitudes, and beliefs, may influence their participation.

Ethical considerations : Field research may raise ethical concerns, mainly if the research involves vulnerable populations or sensitive topics. 

Causality: Researchers may have little control over the environmental or contextual variables they are studying. This can make it difficult to establish causality and then generalize their results with previous research. 

Researchers must carefully weigh the advantages and disadvantages of field research and select the most appropriate research method based on their research question, participants, and context.

What is another name for field study?

Field study is also known as field research or fieldwork. These terms are often used interchangeably and refer to research methods that involve observing and collecting data in natural settings.

What is the difference between a field study and a case study?

Why is field study important.

Field study is critical because it allows researchers to study real-world phenomena in natural settings. This study can also lead to novel insights that may not be possible with other research methods.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 17 February 2024

Last updated: 19 November 2023

Last updated: 5 March 2024

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

• Trending Now
• Foundational Courses
• Data Science
• Practice Problem
• Machine Learning
• System Design
• DevOps Tutorial
• Web Browser
• Internationalisation and Localisation in UX Design
• 10 Ways to Tackle Creative Block as a UX Designer
• 10 Red Flags to Watch Out During UX Interview
• 100 Days of UI/UX: A Complete Guide For Beginners
• What is Agile UX - A Complete Guide For Beginners
• Top 20 Skills For UX Designers: Technical and Interpersonal
• Usability Testing Guide For UX Designers
• Beginner’s Guide to Create Design Strategy
• Crack UX Design Interview: Most Asked UX Design Questions, Tips and Tricks
• UX Design Interview: A Complete Guide For Beginners
• UX Design Process: A Complete Guide
• Top 15 Web Crawling Tools to Scrape Websites
• 20 Best VSCode Themes in 2024
• Top 10 Microservices Frameworks [2024 Updated]
• Top 10 Visual Studio Code Alternatives
• 7 Hacks to use YouTube Premium
• Best Automation Courses and Certifications [2024]
• Top 10 free Websites to find Remote Job
• Top Web3 Startups in 2023

UX Research Field Study: A Complete Guide

Understanding users’ needs, preferences, and pain points has never been more crucial for businesses aiming to create successful and impactful products and services. UX research, includes user interviews , usability testing , persona creation , data analysis , and the integration of emerging technologies.

As technology continues to advance, so does the complexity of users’ needs and expectations, making it essential to adapt and refine our research methodologies to meet these demands effectively. In this article, we’ll have a look at a complete guide to understand and get the latest insights, methodologies, and best practices in the field of UX research. 

What is UX Research?

The goal of UX (User Experience) research is to understand users’ behaviors, wants, and preferences when dealing with goods, services, or digital interfaces . It is a systematic and multidisciplinary approach. By providing information to the design and development process, its main objective is to enhance the overall user experience.

UX researchers learn important information about users , including their motivations, problems , and objectives, through a combination of qualitative and quantitative methodologies. User interviews, surveys, usability testing, card sorting, and data analysis are a few of these techniques.

After a quick understanding of What UX research is , let’s now move toward understanding what is user research and user observation. 

User Research & User Observation 

Both user research and user observation are separate concepts, which are carried out based on the context of the problem designer or researchers are trying to solve.

1. User Research: Understanding Users for Better Design

A key component of the user experience design process is user research, which focuses on learning about the needs, wants, and preferences of the target audience. It entails gathering information using a range of qualitative and quantitative techniques to guide the creation of products, services, or digital user interfaces.

Methods of User Research:

• User Interviews: Conducting one-on-one interviews with users helps researchers gain in-depth insights into their thoughts, motivations, and pain points. It allows for a personalized understanding of individual experiences and preferences.
• Surveys and Questionnaires: Surveys are useful for collecting data from a large number of users quickly. They provide quantitative data that can reveal trends and patterns.
• Usability Testing: This involves observing users as they interact with a prototype or an existing product. Researchers can identify usability issues and areas for improvement based on users’ actions and feedback.
• Persona Creation: Personas are fictional representations of target users based on research data. They help designers and developers better empathize with users and make informed decisions.
• Card Sorting: Card sorting is a method where users categorize information or features to help designers understand how users mentally organize content.
• Data Analysis: Analyzing the collected data helps researchers identify patterns, trends, and key insights that inform design decisions.

2. User Observation: Observing Users to know how they perform a certain action

A qualitative research technique called “ user observation ” allows researchers to watch users as they engage with a product or service in real time. In contrast to surveys and interviews, which rely on self-reported information, observation offers more unbiased and behavior-based insights. 

Key Aspects of User Observation:

• Natural Context: Observing users in their natural environment provides a better understanding of their real-world interactions and challenges.
• Behaviour Patterns: Observations reveal patterns in user behavior, highlighting pain points, difficulties, and moments of delight.
• Non-Verbal Cues: Users may not always articulate their experiences accurately, but non-verbal cues like facial expressions and body language can offer valuable clues.
• User Empathy: Directly observing users fosters empathy, helping designers connect with users on a deeper level.
• Real-Time Feedback: Observations provide immediate feedback, allowing designers to make quick adjustments and iterate designs rapidly.

In order to produce products and services that actually resonate with users, increase satisfaction, and promote economic success, user research and observation are essential parts of the UX design process.

Data Gathering Method in UX Research

In order to obtain important data regarding users’ behaviors, preferences, and experiences, data gathering methods in UX research use a variety of methodologies and approaches. By using these techniques, researchers can develop user-centric designs by gaining insights into the wants, problems, and motivations of users. 

There are 3 dimensions of data gathering methods, using which designer or researchers conduct research sessions. 

• Direct & Indirect data gathering methods
• Individual & Group data gathering methods
• Performance & Discussion data gathering methods

Let’s have a detailed look at each of the dimensions of data gathering methods.

1. Direct & Indirect Data Gathering Methods

The two primary categories of data collection techniques used in UX research are direct and indirect techniques. Both strategies have different goals and provide distinctive insights on user actions and experiences. Let’s examine each in more depth:

Direct Data Gathering Methods:

• User Interviews: User interviews are one of the most popular direct approaches and involve in-person or virtual encounters with people. To acquire a thorough understanding of consumers’ thoughts, feelings, motives, and pain spots, researchers pose open-ended questions.
• Usability Testing: Usability testing involves watching actual consumers as they engage with a product or prototype. Researchers can spot usability problems, examine navigational patterns, and gather user experience input.
• Focus Groups: Focus groups involve gathering a small group of users to talk about particular subjects or ideas connected to a product. The dynamics of the group can encourage conversation and idea creation while offering insightful information about shared viewpoints and group dynamics. 
• Field Studies and Contextual Inquiry: Field studies entail monitoring people while they utilize a product or service in their natural contexts. This approach gives users context from the real world and reveals user behaviors that may not be seen in a controlled lab environment.

Indirect Data Gathering Methods:

• Surveys and Questionnaires: One of the popular indirect methods for gathering information from a bigger population of people. Surveys are important for determining user demographics, sentiments, and preferences.
• Analytics and Clickstream Analysis: Examining a website’s or an app’s analytics can provide you hints about how users behave by measuring things like time spent on a page, click-through rates , and conversion rates .
• A/B Testing: Following conversations and emotions about a product or brand on social media can provide indirect feedback from a larger audience. The public’s impressions and user sentiment can be better understood with this strategy.
• Social Media Listening: Comparing two or more iterations of a design to see which performs better is known as A/B testing. Researchers can decide on design changes using data-driven decisions by gathering information on user interactions with various versions.
• Heat-maps and Eye Tracking: These tools provide us a hazy understanding of how users interact with our products and what they are paying attention to. Eye tracking shows where users’ attention is focused on a screen, whereas heat maps show areas of strong user interaction.

Both direct and indirect data collection techniques have advantages and disadvantages. Direct techniques may take more time and only work with a smaller number of users, but they offer deep qualitative insights and a greater knowledge of user emotions and motives. On the other hand, indirect methods enable data collecting from a bigger audience and provide quantifiable data, but they could not provide the same level of knowledge as direct contacts.

2. Individual & Group Data Gathering Methods

Based on the number of participants, data collection techniques can be divided into two broad categories: individual and group data collection techniques. Each strategy has unique benefits and is appropriate for various research goals. Let’s examine each type in greater detail:

2.1. Individual Data Gathering Methods

Focuses on Depth of insights, Privacy and Personalisation.

Note: Individual methods might not capture the influence of group dynamics on user behavior.

• Diary Studies: In a diary study, participants maintain a journal or diary detailing their interactions with a good or service over time. This longitudinal approach offers deep insights into the experiences and behaviors of people in everyday life.
• Contextual Inquiry: Contextual inquiry entails observing and speaking with specific users while they carry out specific tasks associated with a product or service in their natural surroundings.

2.2. Group Data Gathering Methods

Focuses on Efficiency, Interaction insights and Stimulating Discussion. 

Note: Due to the dynamics of the group, some members may dominate conversations while others sit back and listen.

• Workshops and Design Sprints: During workshops and design sprints, participants collaborate to brainstorm, rank features, or give input on design concepts.
• Card Sorting (Group-Based): Group-based card sorting entails several individuals cooperating to classify data or qualities, offering insights into societal mental models and organizational preferences.
• Online Discussion Forums and Communities: Online forums and communities allow researchers to gather insights from a larger group of users asynchronously. Participants can share experiences, discuss ideas, and provide feedback at their convenience.

It would be right to say that both individual and group data collection techniques are essential because they provide distinctive insights into user experiences and preferences.

3. Performance & Discussion Data Gathering Methods

3.1. performance data gathering methods.

Focus on observing users’ actions and interactions to measure their task success, efficiency, and effectiveness. These methods provide quantitative data, allowing researchers to identify usability issues and assess the overall performance of a product or interface. 

Note: Performance data might not reveal the “why” behind users’ actions, requiring additional discussion data to gain a comprehensive understanding.

• Usability Testing
• Clickstream Analysis
• Eye Tracking
• Analytics and Heatmaps
• Objective Metrics: Performance data offers objective metrics to evaluate product usability and efficiency.
• Identifying Usability Issues: Performance data helps identify pain points and areas for improvement in the user experience.
• Data-driven Design Decisions: Quantitative data supports data-driven design decisions and can be useful for justifying design changes to stakeholders.

3.2. Discussion Data Gathering Methods

Focus on gathering qualitative data through interactions and discussions with users. These methods provide insights into users’ opinions, perceptions, and subjective experiences, offering a deeper understanding of their motivations and preferences. 

Note: Social desirability bias, where participants may give responses they think researchers want to hear, can have an impact on discussion data.

• User Interviews
• Focus Groups
• Online Discussion Forums and Surveys
• Contextual Inquiry
• User Perspectives: Discussion data provides rich qualitative insights into users’ perspectives, motivations, and emotional responses.
• Understanding Context: Discussion methods offer contextual information and provide a deeper understanding of user needs in real-life settings.
• Ideation and Innovation: Focus groups and discussions foster idea generation and encourage participants to suggest innovative solutions.

It is always a great idea to combine both quantitative performance data and qualitative discussion data to lead to a more comprehensive and informed understanding of users and their interactions with products and services.

Must Check: UX Design Process: A Complete Guide UX Design | Key Process, Flow and Principles 7 Tips to Create an Impressive UX Design

In this detailed article we explored what field studies are, the diverse data gathering techniques employed. We also had a look at the ethical considerations in conducting field studies, ensuring that users’ privacy and consent are respected throughout the research process. Field studies help researchers to find unexpected insights and validate assumptions by stepping outside of the boundaries of controlled lab settings, paving the way for user-centric design decisions. Field studies provide the rich qualitative data that supports the quantitative results from conventional usability testing, whether it be through anthropological observations, contextual queries, or user shadowing. In the end I hope this article serves the purpose as a complete guide to field study for beginners. 

Please Login to comment...

• 10 Best Zoho Vault Alternatives in 2024 (Free)
• 10 Best Twitch Alternatives in 2024 (Free)
• Top 10 Best Cartoons of All Time
• 10 Best AI Instagram Post Generator in 2024
• 30 OOPs Interview Questions and Answers (2024)

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

The Complete Guide to UX Research Methods

UX research provides invaluable insight into product users and what they need and value. Not only will research reduce the risk of a miscalculated guess, it will uncover new opportunities for innovation.

By Miklos Philips

Miklos is a UX designer, product design strategist, author, and speaker with more than 18 years of experience in the design field.

PREVIOUSLY AT

“Empathy is at the heart of design. Without the understanding of what others see, feel, and experience, design is a pointless task.” —Tim Brown, CEO of the innovation and design firm IDEO

User experience (UX) design is the process of designing products that are useful, easy to use, and a pleasure to engage. It’s about enhancing the entire experience people have while interacting with a product and making sure they find value, satisfaction, and delight. If a mountain peak represents that goal, employing various types of UX research is the path UX designers use to get to the top of the mountain.

User experience research is one of the most misunderstood yet critical steps in UX design. Sometimes treated as an afterthought or an unaffordable luxury, UX research, and user testing should inform every design decision.

Every product, service, or user interface designers create in the safety and comfort of their workplaces has to survive and prosper in the real world. Countless people will engage our creations in an unpredictable environment over which designers have no control. UX research is the key to grounding ideas in reality and improving the odds of success, but research can be a scary word. It may sound like money we don’t have, time we can’t spare, and expertise we have to seek.

In order to do UX research effectively—to get a clear picture of what users think and why they do what they do—e.g., to “walk a mile in the user’s shoes” as a favorite UX maxim goes, it is essential that user experience designers and product teams conduct user research often and regularly. Contingent upon time, resources, and budget, the deeper they can dive the better.

What Is UX Research?

There is a long, comprehensive list of UX design research methods employed by user researchers , but at its center is the user and how they think and behave —their needs and motivations. Typically, UX research does this through observation techniques, task analysis, and other feedback methodologies.

There are two main types of user research: quantitative (statistics: can be calculated and computed; focuses on numbers and mathematical calculations) and qualitative (insights: concerned with descriptions, which can be observed but cannot be computed).

Quantitative research is primarily exploratory research and is used to quantify the problem by way of generating numerical data or data that can be transformed into usable statistics. Some common data collection methods include various forms of surveys – online surveys , paper surveys , mobile surveys and kiosk surveys , longitudinal studies, website interceptors, online polls, and systematic observations.

This user research method may also include analytics, such as Google Analytics .

Google Analytics is part of a suite of interconnected tools that help interpret data on your site’s visitors including Data Studio , a powerful data-visualization tool, and Google Optimize, for running and analyzing dynamic A/B testing.

Quantitative data from analytics platforms should ideally be balanced with qualitative insights gathered from other UX testing methods , such as focus groups or usability testing. The analytical data will show patterns that may be useful for deciding what assumptions to test further.

Qualitative user research is a direct assessment of behavior based on observation. It’s about understanding people’s beliefs and practices on their terms. It can involve several different methods including contextual observation, ethnographic studies, interviews, field studies, and moderated usability tests.

Jakob Nielsen of the Nielsen Norman Group feels that in the case of UX research, it is better to emphasize insights (qualitative research) and that although quant has some advantages, qualitative research breaks down complicated information so it’s easy to understand, and overall delivers better results more cost effectively—in other words, it is much cheaper to find and fix problems during the design phase before you start to build. Often the most important information is not quantifiable, and he goes on to suggest that “quantitative studies are often too narrow to be useful and are sometimes directly misleading.”

Not everything that can be counted counts, and not everything that counts can be counted. William Bruce Cameron

Design research is not typical of traditional science with ethnography being its closest equivalent—effective usability is contextual and depends on a broad understanding of human behavior if it is going to work.

Nevertheless, the types of user research you can or should perform will depend on the type of site, system or app you are developing, your timeline, and your environment.

Top UX Research Methods and When to Use Them

Here are some examples of the types of user research performed at each phase of a project.

Card Sorting : Allows users to group and sort a site’s information into a logical structure that will typically drive navigation and the site’s information architecture. This helps ensure that the site structure matches the way users think.

Contextual Interviews : Enables the observation of users in their natural environment, giving you a better understanding of the way users work.

First Click Testing : A testing method focused on navigation, which can be performed on a functioning website, a prototype, or a wireframe.

Focus Groups : Moderated discussion with a group of users, allowing insight into user attitudes, ideas, and desires.

Heuristic Evaluation/Expert Review : A group of usability experts evaluating a website against a list of established guidelines .

Interviews : One-on-one discussions with users show how a particular user works. They enable you to get detailed information about a user’s attitudes, desires, and experiences.

Parallel Design : A design methodology that involves several designers pursuing the same effort simultaneously but independently, with the intention to combine the best aspects of each for the ultimate solution.

Personas : The creation of a representative user based on available data and user interviews. Though the personal details of the persona may be fictional, the information used to create the user type is not.

Prototyping : Allows the design team to explore ideas before implementing them by creating a mock-up of the site. A prototype can range from a paper mock-up to interactive HTML pages.

Surveys : A series of questions asked to multiple users of your website that help you learn about the people who visit your site.

System Usability Scale (SUS) : SUS is a technology-independent ten-item scale for subjective evaluation of the usability.

Task Analysis : Involves learning about user goals, including what users want to do on your website, and helps you understand the tasks that users will perform on your site.

Usability Testing : Identifies user frustrations and problems with a site through one-on-one sessions where a “real-life” user performs tasks on the site being studied.

Use Cases : Provide a description of how users use a particular feature of your website. They provide a detailed look at how users interact with the site, including the steps users take to accomplish each task.

You can do user research at all stages or whatever stage you are in currently. However, the Nielsen Norman Group advises that most of it be done during the earlier phases when it will have the biggest impact. They also suggest it’s a good idea to save some of your budget for additional research that may become necessary (or helpful) later in the project.

Here is a diagram listing recommended options that can be done as a project moves through the design stages. The process will vary, and may only include a few things on the list during each phase. The most frequently used methods are shown in bold.

Reasons for Doing UX Research

Here are three great reasons for doing user research :

To create a product that is truly relevant to users

• If you don’t have a clear understanding of your users and their mental models, you have no way of knowing whether your design will be relevant. A design that is not relevant to its target audience will never be a success.

To create a product that is easy and pleasurable to use

• A favorite quote from Steve Jobs: “ If the user is having a problem, it’s our problem .” If your user experience is not optimal, chances are that people will move on to another product.

To have the return on investment (ROI) of user experience design validated and be able to show:

• An improvement in performance and credibility
• Increased exposure and sales—growth in customer base
• A reduced burden on resources—more efficient work processes

Aside from the reasons mentioned above, doing user research gives insight into which features to prioritize, and in general, helps develop clarity around a project.

What Results Can I Expect from UX Research?

In the words of Mike Kuniaysky, user research is “ the process of understanding the impact of design on an audience. ”

User research has been essential to the success of behemoths like USAA and Amazon ; Joe Gebbia, CEO of Airbnb is an enthusiastic proponent, testifying that its implementation helped turn things around for the company when it was floundering as an early startup.

Some of the results generated through UX research confirm that improving the usability of a site or app will:

• Increase conversion rates
• Increase sign-ups
• Increase NPS (net promoter score)
• Increase customer satisfaction
• Increase purchase rates
• Boost loyalty to the brand
• Reduce customer service calls

Additionally, and aside from benefiting the overall user experience, the integration of UX research into the development process can:

• Minimize development time
• Reduce production costs
• Uncover valuable insights about your audience
• Give an in-depth view into users’ mental models, pain points, and goals

User research is at the core of every exceptional user experience. As the name suggests, UX is subjective—the experience that a person goes through while using a product. Therefore, it is necessary to understand the needs and goals of potential users, the context, and their tasks which are unique for each product. By selecting appropriate UX research methods and applying them rigorously, designers can shape a product’s design and can come up with products that serve both customers and businesses more effectively.

Further Reading on the Toptal Blog:

• How to Conduct Effective UX Research: A Guide
• The Value of User Research
• UX Research Methods and the Path to User Empathy
• Design Talks: Research in Action with UX Researcher Caitria O'Neill
• Swipe Right: 3 Ways to Boost Safety in Dating App Design
• How to Avoid 5 Types of Cognitive Bias in User Research

Understanding the basics

How do you do user research in ux.

UX research includes two main types: quantitative (statistical data) and qualitative (insights that can be observed but not computed), done through observation techniques, task analysis, and other feedback methodologies. The UX research methods used depend on the type of site, system, or app being developed.

What are UX methods?

There is a long list of methods employed by user research, but at its center is the user and how they think, behave—their needs and motivations. Typically, UX research does this through observation techniques, task analysis, and other UX methodologies.

What is the best research methodology for user experience design?

The type of UX methodology depends on the type of site, system or app being developed, its timeline, and environment. There are 2 main types: quantitative (statistics) and qualitative (insights).

What does a UX researcher do?

A user researcher removes the need for false assumptions and guesswork by using observation techniques, task analysis, and other feedback methodologies to understand a user’s motivation, behavior, and needs.

Why is UX research important?

UX research will help create a product that is relevant to users and is easy and pleasurable to use while boosting a product’s ROI. Aside from these reasons, user research gives insight into which features to prioritize, and in general, helps develop clarity around a project.

• UserResearch

Miklos Philips

London, United Kingdom

Member since May 20, 2016

About the author

World-class articles, delivered weekly.

Subscription implies consent to our privacy policy

Toptal Designers

• Adobe Creative Suite Experts
• Agile Designers
• AI Designers
• Art Direction Experts
• Augmented Reality Designers
• Axure Experts
• Brand Designers
• Creative Directors
• Dashboard Designers
• Digital Product Designers
• E-commerce Website Designers
• Full-Stack Designers
• Information Architecture Experts
• Interactive Designers
• Mobile App Designers
• Mockup Designers
• Presentation Designers
• Prototype Designers
• SaaS Designers
• Sketch Experts
• Squarespace Designers
• User Flow Designers
• User Research Designers
• Virtual Reality Designers
• Visual Designers
• Wireframing Experts
• View More Freelance Designers

Join the Toptal ® community.

How to Conduct User Experience Research Like a Professional

Whether you’re looking to develop a broad UX design skillset, or you’re exploring UX research as a design specialization , here’s your complete introduction to conducting user research like a pro. 

Hello, I’m Raven, a mentor for aspiring UX designers enrolled in the CareerFoundry UX Design Course . I also work as a UX Research Assistant at IBM and studied behavioral science at the University of Texas. I have 10 years of experience studying and analyzing human behavior—user research is definitely my thing.

During the past few years, I’ve worked with major companies, academic institutions, and non-profit organizations to develop and improve impactful products and applications. I’ve moderated focus groups, designed and administered surveys, carried out usability testing, and conducted user interviews. I also know a thing or two about creating a good persona!

In this guide, we’re going to cover the basics of UX research. We’ll start with exactly what it is, and then move on to discuss the various steps and associated terminology of UX research , as well as its role and value within the broader design process . We’ll then review the most common UX research methods, diving into how they’re conducted and a few best practices.

If you’re particularly interested in one of these topics, simply select it from the list below to jump straight to it. I’ve also added videos throughout the guide for those of you who prefer to learn with both eyes and ears—and I recommend you save this set of free UX research tutorials for later, too. Sound good? Let’s get started!

Introduction To User Experience (UX) Research

• What is UX research?
• What’s the difference between good and bad UX research?
• What are the five steps of UX research?
• What’s the role of research in the UX design process?
• Whats the value of UX research?

Introduction To User Experience Research Methods

• User Groups
• Usability Testing
• User Interviews
• Online Surveys
• User Personas
• What Next? User Research Analysis

1. What is UX research?

You read my bio in the introduction. Using only this information, could you explain why I recently switched from one time management app to another? Probably not. In order to answer this question, you need more context. UX research provides that context.  So, what is UX research and what is its purpose ?

“User research is how you will know your product or service will work in the real world, with real people. It’s where you will uncover or validate the user needs which should form the basis of what you are designing.”

— Chris Mears, UXr

According to Design Modo , UX research is; “The process of understanding user behaviors, needs, and attitudes using different observation and feedback collection methods.” One of the other benefits of user experience research is that it helps us understand how people live their lives so that we can respond to their needs with informed design solutions. Good UX research involves using the right method at the right time during the development of a product.

Maria Arvidsson, Head of Product and UX at Usabilla , describes UX research as:

“The means through which you try to understand your users’ needs, behaviors and motivations and validate your assumptions and solutions.”

2. What’s the difference between good and bad UX research?

The biggest sign of an amateur UX designer is excluding end users from the design process. At the very start of my career I held the attitude that I could test any app, website, or product on myself, replacing the act of speaking with users. Never a good idea. It took time for me to learn a more professional approach, which is to start the design process by listening to the end user. Overall, UX research helps us avoid our biases since we are required to design solutions for people who are not like us.

“Insights that are received directly from user experience research are like muscle memory; the more you do research, the more insights you build up. But just like muscle memory, YOU have to be a part of the hard work in order to enjoy the lasting benefits of it that are specific to you. While it may be tempting to outsource research to a specialized team (and sometimes you can’t avoid it), you should try your utmost best to engage in at least a little bit of the research so that the insights grow under your skin instead of being handed to you from someone else who has sweated it.” 

—UX designer Ali Rushdan Tariq from ARTariq

A quick plug before we continue: If you’re looking to become a professional in this subdomain of UX, be sure to take a look at our guide to becoming a UX researcher

3. What are the five steps of UX research?

Created by Erin Sanders , the Research Learning Spiral provides five main steps for conducting UX research. The first two steps are about forming questions and hypotheses, and the last three steps are about gathering knowledge through selected UX research methods.

• Objectives: What are the knowledge gaps we need to fill?
• Hypotheses: What do we think we understand about our users?
• Methods: Based on time and manpower, what methods should we select?
• Conduct: Gather data through the selected methods.
• Synthesize: Fill in the knowledge gaps, prove or disprove our hypotheses, and discover opportunities for our design efforts.

4. What’s the role of research in the UX design process?

UX research is the starting point for a project . Research helps us learn about the users and their behavior, goals, motivations, and needs. It also shows us how they currently navigate a system, where they have problems and, most importantly, how they feel when interacting with our product.

UX research comes first in the UX design process because without it, our work can only be based on our own experiences and assumptions, which is not objective. As Neil Turner, founder of UX for the Masses told us, a good foundation is key to successful design:

“Good user research is key to designing a great user experience. Designing without good user research is like building a house without solid foundations—your design will soon start to crumble and eventually fall apart.”

5. What’s the value of UX research?

In the current digital product landscape, the real value of UX research is its ability to reduce uncertainty in terms of what users want and need , which yields benefits for the product, the business, and, of course, the users themselves.

1. Product Benefits

UX research provides data about the end user of the product, how and when the user will use the product, and the main problems the product will solve. UX research is also helpful when UX designers and the rest of the team (and stakeholders) have to decide between multiple design solutions.

2. Business Benefits

UX research brings a lot of a value to businesses. By knowing the end users and incorporating design requirements upfront, businesses can speed up the product development process, eliminate redesign costs, and increase user satisfaction.

3. User Benefits

One of the greatest values of user experience research is that it’s unbiased user feedback. Simply put, UX research speaks the user’s thoughts—without any influence from outside authority. It also serves as a bridge between users and the company.

“User experience research provides powerful insights that allow companies to humanize their customers and insert their needs, intentions, and behaviors into the design and development process. In turn, these insights enable companies to create experiences that meet—and sometimes exceed—customer needs and expectations. User experience research should be conducted well before the first sketch is drawn and integrated throughout the concept, iterative design, and launch phases of a product.”

—Janelle Estes, Director of Research Strategy at UserTesting

UX research is based on observation, understanding,  and analysis.  With the help of various UX research techniques, you will:

• O bserve your users , keeping an eye out for non-verbal clues as to how they are feeling;
• Develop an understanding of the user’s mental model : what does the user anticipate when using a certain product? Based on their previous experience, how do they expect this particular product to work?
• A nalyze  the insights you’ve gathered and try to identify patterns and trends. Eventually, these insights will inform the decisions you make about the product and how it is designed.

With that in mind, let’s consider some of the most valuable user research techniques.

1. User Groups

User groups—also called “focus group discussions” or “focus groups”—are structured interviews that quickly and inexpensively reveal the desires, experiences, and attitudes of a target audience. User groups are a helpful user experience research method when a company needs a lot of insight in a short amount of time. If you are unsure when to use a user experience research method, user groups can be a good one to start with.

Why Do We Conduct User Groups?

User groups can help your company better understand:

1) How users perceive a product

2) What users believe are a product’s most important features

3) What problems users experience with the product

4) Where users feel the product fails to meet expectations

User groups can also be used to generate ideas of what users want to see in the future.

What people say and what people do are often very different, therefore user groups do not provide an accurate measurement of behavior . And because user groups are conducted with more than one user at a time, participants may influence each other’s opinions and preferences (aka “groupthink”), thus introducing bias and producing inaccurate data.

Best Practices For User Groups

Getting the most out of your user group is straightforward if you consider the following best practices when conducting this particular user research technique.

• Ask good questions: Make sure your questions are clear, open-ended, and focused on the topics you’re investigating.
• Choose a few topics: On average, plan to discuss 3-5 topics during a 90-minute focus group.
• Include the right amount of people: A good focus group should include 3-6 users—large enough to include a variety of perspectives, but small enough so everyone has a chance to speak.

“Conducting user research allows you to dive deep beneath the surface of what your users say they want, to instead uncover what they actually need. It’s the key to ensuring that your products and features will actually solve the problems that your clients face on a day to day basis. User research is imperative if you want to create a successful, habit forming product.”

— Jennifer Aldrich, UX and Content Strategist at InVisionApp

How To Conduct User Experience Research With User Groups

Conducting user groups can be broken down into a few major steps:

• Create a schedule that provides enough time for recruiting, testing, analyzing, and integrating results.
• Assemble your team, and establish roles: choose a moderator, note-taker, and discussion leader.
• Define the scope of your research: what questions will you ask? And how in-depth do you want to explore the answers? This will determine the number of people and the number of groups that need to be tested.
• Create a discussion guide that includes 3-5 topics for discussion.
• Recruit potential or existing users who are likely to provide good feedback.
• Conduct user group testing, and record data.
• Analyze and report findings.

“It’s really hard to design products by user groups. A lot of times, people don’t know what they want until you show it to them.”

—Steve Jobs

2. Usability Testing

According to the usability.gov website, usability testing refers to “evaluating a product or service by testing it with representative users.” During a test, participants will be asked to complete specific tasks while one or more observers watch, listen, and record notes. The main goal of this user experience testing method is to identify usability problems, collect qualitative data, and determine participants’ overall satisfaction with the product.

Why Do We Perform Usability Testing?

Usability testing helps identify problems before they are coded. When development issues are identified early on, it is typically less expensive to fix them. Usability testing also reveals how satisfied users are with the product , as well as what changes are required to improve user satisfaction and performance .

Unfortunately, usability testing is not 100% representative of the real life scenario in which a user will engage with your product. Also, because the data is qualitative, this kind of UX testing method doesn’t provide the large samples of feedback a questionnaire might. The good news it that the qualitative feedback you receive can be far more accurate and insightful.

Best Practices For Usability Testing

• Test with five users: Testing five users is typically enough to identify a design’s most important usability problems.
• Invite your team to the testing sessions: Anyone who is involved with how fast and how well problems are addressed should be invited to the usability testing sessions. These stakeholders may include the executive team, and lead developers or designers.
• Keep the findings brief and to-the-point: When you report the findings of a usability test, limit the comments to the ones that are really important. One good rule of thumb is to include the top three positive comments and the top three problems. The overall report should be no more than approximately 50 comments and 30 pages.

How to Conduct UX Research with Usability Testing

Usability testing can be broken down into a few major steps:

• Identify what needs to be tested and why (e.g. a new product, feature, etc.)
• Identify the target audience (or your desired customers).
• Create a list of tasks for the participants to work through.
• Recruit the right participants for the test.
• Involve the right stakeholders.
• Apply what you learn.

“One of usability’s most hard-earned lessons is that ‘you are not the user.’ If you work on a development project, you’re atypical by definition. Design to optimize the user experience for outsiders, not insiders.”

– Jakob Nielsen

3. User Interviews

A well-known user experience methodology is an interview. An interview is a user experience research method used to discover the attitudes, beliefs, and experiences of users (and potential users) of a product. Interviews are typically conducted by one interviewer speaking to one user at a time for 30 minutes to an hour. Interviews can take place face-to-face, over the phone, or via video streaming.

Why Do We Conduct Interviews?

Of all the user experience design methods, interviews are typically conducted at the beginning of the product development cycle when reviewing product goals. Because of the one-to-one nature of the interview, individual concerns and misunderstandings can be directly addressed and cleared up.

Face-to-face interviews also allow you to capture verbal and nonverbal cues, such as emotions and body language, which may identify enthusiasm for the product or discomfort with the questions.

When thinking about what research methodology to use, bear in mind that interviews are also a good supplement to online surveys: conducting an interview beforehand helps you refine questions for the survey, while conducting an interview afterwards allows you to gain explanations for survey answers.

There are a few drawbacks, however. First, because interviews require a team of people to conduct them, personnel costs are usually difficult to keep low. Sample size is also limited to the size of the interviewing staff.

Best Practices For User Interviews

• Hire a skilled interviewer: A skilled interviewer asks questions in a neutral manner, listens well, makes users feel comfortable, and knows when and how to probe for more details.
• Create a discussion guide: Write up a discussion guide (or an interview protocol) for all interviewers to follow. This guide should include questions and follow-up questions.
• Get informed consent: Before conducting the interview, make sure to get permission or consent to record the session. It’s also good to have one or two note takers on hand.

How To Conduct User Experience Research With User Interviews

Conducting an interview can be broken down into a few major steps:

• Prepare a discussion guide, or a list of questions to ask participants.
• Select a recording method (e.g. written notes, tape recorder, video).
• Conduct at least one trial run of the interview.
• Recruit the right participants for the interview.
• Conduct the interview.
• Analyze and report the results.

“Curiosity is a natural outcome of caring, and it is the single greatest contributor to effective user research … Caring and curiosity engender personal investment, and investment motivates a researcher to develop a deep understanding of users.”

– Demetrius Madrigal

4. Online Surveys

A survey is a research tool that typically includes a set of questions used to find out the preferences, attitudes, and opinions of your users on a given topic. Today, surveys are generally conducted online and in various lengths and formats. Data collected from surveys is received automatically, and the survey tool selected generally provides some level of analysis, the data from which can then be used for user experience studies further down the line to inform your product.

“It is so important to avoid using leading questions when it comes to surveys. It’s a common mistake that many people make. For example phrasing a question like “What do you dislike about Uber?” assumes the user has a negative preference for the service off the bat. A more neutral phrase would be “Tell us about your experience getting around town.” – this elicits more natural user feedback and behavior instead of forcing them down a funnel.”

– Top tip from UXBeginner

Why Do We Conduct Online Surveys?

Unlike traditional surveys, online surveys enable companies to quickly collect data from a broad (and sometimes remote) audience for free—or a low price. Surveys also help you discover who your users are , what your users want to accomplish, and what information your users are looking for.

Unfortunately, what users say versus what they do are two different things and can often yield inaccurate results. Furthermore, poorly worded questions can negatively influence how users respond. Length can also be an issue—many people hate taking long surveys. This is why it’s important to create short surveys so users are more likely to complete them and participate in future research efforts.

Best Practices For Online Surveys

• Keep it short: Keep your surveys brief, especially if participants will be compensated little or not at all. Only focus on what is truly important.
• Keep it simple: Make sure questions can be easily understood: ambiguous or complex wording can make questions more difficult to understand, which can bring the data into question.
• Keep it engaging: Include a mix of both multiple choice questions and open-ended questions (or questions in which users complete the answer).

How To Conduct User Experience Research With Online Surveys

Conducting an online survey can be broken down into a few major steps:

• Identify goals and objectives of the survey.
• Create survey questions.

Note: Consider collecting information about how satisfied users are with your product, what users like/dislike, and if they have suggestions for improvement.

• Select an online survey tool (e.g. SurveyMonkey, Qualtrics).
• Recruit participants.
• Conduct the survey.

“We have to arm ourselves with data, research … and a clear understanding of our users so our decisions are not made out of fear but out of real, actionable information. Although our clients may not have articulated reasons for why they want what they want, it is our responsibility to have an ironclad rationale to support our design decisions.”

– Debra Levin Gelman

5. User Personas

A user persona is a fictional representation of your ideal customer. A persona is generally based on user research and includes the needs, goals, and observed behavior patterns of your target audience. You can find out how to create a user persona in this detailed guide .

Why Do We Create User Personas?

Whether you’re developing a smartphone app or a mobile-responsive website, any user experience research job will require you to understand who will be using the product. Knowing your audience will help influence the features and design elements you choose, thus making your product more useful. A persona clarifies who is in your target audience by answering the following questions:

• Who is my ideal customer?
• What are the current behavior patterns of my users?
• What are the needs and goals of my users?

Understanding the needs of your users is vital to developing a successful product. Well-defined personas will enable you to efficiently identify and communicate user needs. Personas will also help you describe the individuals who use your product, which is essential to your overall value proposition.

Unfortunately, creating personas can be expensive — it all depends on how deep into user research your organization is willing to go. There is also no real “scientific logic” behind persona building, which makes some people a little more hesitant to accept them.

Best Practices For User Personas

• Create a well-defined user persona: A great persona contains four key pieces of information: header, demographic profile, end goal(s), scenario.
• Keep personas brief: As a rule of thumb, avoid adding extra details that cannot be used to influence the design. If it does not affect the final design or help make any decisions easier: omit it.
• Make personas specific and realistic: Avoid exaggerated caricatures, and include enough detail to help you find real-life representation.

How To Conduct User Experience Research By Creating Personas

Creating user personas can be broken down into these main steps:

• Discuss and identify who your target users are with stakeholders (e.g. UX team, marketing team, product manager).
• Survey and/or interview real users to get their demographic information, pain points, and preferences.
• Condense the research, and look for themes to define your groups.
• Organize your groups into personas.
• Test your personas.

“Be someone else. It takes great empathy to create a good experience. To create relevant experiences, you have to forget everything you know and design for others. Align with the expected patience, level of interest, and depth of knowledge of your users. Talk in the user’s language.”

– Niko Nyman

Which User Experience Research Method Should You Use?

Now that you know more about the various user experience research methods, which one do you choose? Well, it all depends on your overall research goals.

You’ll also need to consider what stage you’re at in the design process. If you’re just starting out, you’ll want to focus on understanding your users and the underlying problem . What are you trying to solve? Who are you trying to solve it for? At this early stage in the design process, you’ll typically use a mixture of both qualitative and quantitative methods such as field studies, diary studies, surveys, and data mining.

Once you’ve established a direction for your design, you’ll start to think about actually building your product. Your UX research will now focus on evaluating your designs and making sure that they adequately address your users’ needs . So, you’ll choose research methods that can help you to optimize your designs and improve usability—such as card sorting and usability testing.

Eventually, you’ll have finalized your design and developed a working product—but this doesn’t mean your research is done! This is the ideal time to investigate how well the product performs in the real world. At this point, you’ll focus mainly on quantitative research methods , such as usability benchmarking, surveys, and A/B testing.

To help you with the task of choosing your research methods, let’s explore some important distinctions between the various techniques.

Behavioral vs. Attitudinal Research

As mentioned before, there is a big difference between “what people do” versus “what people say.” Attitudinal research is used to understand or measure attitudes and beliefs, whereas behavioral research is used to measure behaviors. For example, usability testing is a behavioral user research method that focuses on action and performance. By contrast, user research methods like user groups, interviews, and persona creation focus on how people think about a product.

UX designers often conduct task analysis to see not how users say they complete tasks in a user flow, but how they actually do.

Quantitative vs. Qualitative Research

When conducting UX research and choosing a suitable method, it’s important to understand the difference between quantitative and qualitative research.

Quantitative research   gathers data that is measurable. It gives you clear-cut figures to work with, such as how many users purchased an item via your e-commerce app, or what percentage of visitors added an item to their wishlist. “Quant methods”, as they’re sometimes called in the industry, help you to put a number on the usability of your product. They also allow you to compare different designs and determine if one version performs significantly better than another.

Qualitative research   explores the reasons or motivations behind these actions. Why did the user bounce from your website? What made them “wishlist” an item instead of purchasing it? While quantitative data is fixed, qualitative data is more descriptive and open-ended. You can learn all about qualitative research in the video guide below, in which CareerFoundry graduate and professional UX designer Maureen Herben takes you through the most common qualitative user research processes and tools.

A further distinction to make is between how qualitative and quantitative studies go about collecting data. Studies that are qualitative in nature are based on direct observation. For example, you’ll gather data about the user’s behaviours or attitudes by observing them directly in action. Quantitative studies gather this data indirectly—through an online survey, for example.

Qualitative research methods (e.g. usability testing, user groups, interviews) are better for answering questions about why or how to fix a problem, whereas quantitative methods (e.g. online surveys) are great for answering questions about how many and how much.

Ideally, you’ll use a mixture of both qualitative and quantitative methods throughout your user research, and work hard to ensure that the UX research you conduct is inclusive !

6. What Next? Conducting User Research Analysis

Once you’ve conducted extensive user research, you’ll move on to the analysis phase. This is where you’ll turn the raw data you’ve gathered into valuable insights. The purpose of UX research analysis is to interpret what the data means; what does it tell you about the product you’re designing, and the people you’re designing it for? How can you use the data you’ve gathered to inform the design process?

Watch this video to learn how to conduct user research analysis in five simple steps:

Final thoughts

“User experience research is the work that uncovers and articulates the needs of individuals and/or groups in order to inform the design of products and services in a structured manner.”

—Nick Remis, Adaptive Path

Overall, the purpose of user experience research is simple: to discover patterns and reveal unknown insights and preferences from the people who use your product. It basically provides the context for our design. Research also helps us fight the tendency to design for ourselves (or our stakeholders)—and returns the focus on designing for the user.

If you’d like to learn more about UX research, check out these articles:

• What Does a UX Researcher Actually Do? The Ultimate Career Guide
• The Ultimate Guide to UX Research Bootcamps
• Top 5 Mistakes to Avoid in Your UX Research Portfolio
• Interview Toolkit: Top 5 UX Research Questions to Prepare For

And to get inspired, check out these 15 quotes from influential designers in the industry.

Skip navigation

• Log in to UX Certification

World Leaders in Research-Based User Experience

4 steps to field studies with users.

Summary:  Customer visits and other field studies to observe users in their natural habitat are one of the most important user research methods. This video covers the 4 basic steps to prepare and carry out ethnographic-style research, preferably early in the UX design process.

3 minute video by 2019-06-14 3

• Kim Salazar
• Research Methods Research Methods

Share this article:

• Share this video:

You must have javascript and cookies enabled in order to display videos.

Related Article

Field Studies

Field research is conducted in the user’s context and location. Learn the unexpected by leaving the office and observing people in their natural environment.

Video Author

Kim Salazar is a Senior User Experience Specialist with Nielsen Norman Group. Salazar combines her background as a developer and education in Computer Science with her user experience expertise, particularly around complex applications, to bring well-rounded insights to her work.

• Share: 

Subscribe to the weekly newsletter to get notified about future articles.

UX Researchers, We Like to Watch (UX Slogan #16)

4 minute video

Field Studies vs. Diary Studies

What Are Contextual Inquiries?

3 minute video

Field Studies vs. Ethnographic Studies vs. Contextual Inquiry

• Setup of an Eyetracking Study
• Field Studies Should Inform Intranet Redesign
• The Science of Silence: Intentional Silence as a Moderation Technique
• 27 Tips and Tricks for Conducting Successful User Research in the Field
• Field Studies Done Right: Fast and Observational

Research Reports

• Marketing Email UX - User Research Methodology
• User Experience Careers

UX Conference Training Course

• Discovery: Building the Right Thing
• Measuring UX and ROI
• ResearchOps: Scaling User Research
• How to Interpret UX Numbers: Statistics for UX
• Survey Design and Execution

• UX & Design
• User Research
• Project Stories

User Research Field Study: What about the Field?

Recently, our cross-functional team planned to conduct a field study during the Validation, Discovery, and Design (VDP) phase of a project for a client. In that study, we intended to shadow people while they used their current software systems at a job site. We wanted to learn how they did their work today, what pain points they ran into, and understand their day-to-day life so we could design an improved system.   

The weather in Texas during winter can be unpredictable, and the weather forecast called for cold and rainy conditions when we had planned on doing our study. This wouldn’t usually be an issue, but the work site for the field study was outdoors, off paved roads, and required special safety training certification just to get through the gate. Adding inclement and uncomfortable weather to the mix made me question if this was the right thing to do.  

However, a colleague on the project pointed out that the client mentioned having to work in all weather conditions – even in 3 feet of snow at one of their northern locations – which was a good point. What better way to gain empathy for our users than to truly spend a day in their shoes? Or, in this case, muddy steel-toed boots.  

Developing a Plan

Any time we’re planning user research, it’s important to sketch out a plan. Key parts of that plan include objectives, success criteria, participants, methods, and reporting. There are some great resources for building a research plan .   

Part of a research plan is logistics. Those logistics can be straightforward for an office visit or remote interview. But for a field visit of this nature, I realized we would have to expand that part a bit more.  

As I mentioned, it was not a typical in-office setting. We needed to be able to carry out our research safely and successfully while meeting our research objectives, so we created a field study logistics checklist.  

Field Study Logistics Checklist  

Safety is always the first consideration in industrial settings, and with good reason. Building and maintaining a strong, safety culture is important to help prevent illness, injury, and death.    

• Safety plan: Develop and communicate a safety plan for the study, including emergency contact information and procedures for severe weather or other unexpected events, along with any job-specific plans and requirements for the task at hand including personal protective equipment (PPE) and first-aid equipment.  
• Weather: The weather can significantly impact the study, so it’s essential to have a plan to deal with the weather conditions. This may include appropriate clothing, portable shelter, backup locations, or even rescheduling the study if necessary.  
• Permits and Training Certifications: Many industrial locations require specific safety training from anyone that sets foot on-site. It’s critical to discuss any potential requirements with the sponsoring organization to understand if any training may be needed ahead of time.   

2. Food and Water

Consider planning to provide snacks or water if the study takes more than 2 hours or if the conditions make it necessary.

3. Communication

Ensure there is a reliable communication method during the study, not among participants, but to “the outside world” if needed. Cell service is not ubiquitous when going off the beaten path. 

4. Permissions and Site Access

Depending on the location of the study, you may need permission from property owners or local authorities. It’s also important to research the necessary permissions well in advance of the study and ensure that you have all the necessary documentation.

5. Transportation

Arrange appropriate transportation to and from the study location, including rental cars, public transit, or chartering a vehicle if necessary. Review the location on Google Maps aerial and street view ahead of time to see if any special arrangements should be made. 

6. Accommodations/Housing

Identify any need for appropriate lodging for participants and researchers, including options for extreme weather conditions.   

And don’t forget to include the impact on the budget of these considerations.  

These considerations may all seem daunting and potentially discouraging to doing a field study but remember the goal of your study – to improve the lives of the people using the software you’re going to build. And spending a day in their shoes (muddy boots!) out in the freezing rain or baking heat will help you gain an appreciation for their world when designing the perfect software.  

Our User Research Field Study Logistics Plan  

So, for this study, here’s what we came up with as our logistics plan:  

1. Make a safety plan and communicate it to the team.

• In addition to other items, it included the need for specific personal protective equipment , such as fire-resistant clothing.

2. Get any safety training certification ahead of time.

• Our client helped to identify the specific training that the site required , and got us in touch with a training provider. Our entire team had to complete the training and produce certificates of completion.

3. Plan for cold and rainy weather. 

• Dress in layers and have a water-resistant outer shell. Ask for tips from team members with camping, hiking, and field experience.  
• Think about how we’re going to collect data in poor conditions. Writing in a notebook in windy, freezing, or rainy conditions can be challenging. We took tons of photos and made voice and video recordings to augment our notetaking. 

4. Be prepared to work at a remote location all day.

• Have snacks and drinks to last the whole day if needed. Concentrating on the task at hand is hard if we have low energy.   
• Plan appropriate transportation for the weather and condition so the field study site. In the end, we swapped vehicles around so we could take a 4WD vehicle into the site – it was a good thing we did, as the final stretch had a steep berm and deep mud.   

With this plan in place, the team went on our field study well-equipped to gather data and learn from our clients. We came away that day having learned a ton of valuable information about the specifics of our client’s work. We gained a great appreciation for one aspect of the environment they work in daily. That information made us well-equipped as we designed a software product to make their work in the field safer, more efficient, more accurate, and hopefully even a bit more enjoyable.

• Share on Facebook
• Share on Twitter
• Share on LinkedIn
• Share on Email

Be a Fly On the Wall Subscribe to our newsletter, Metamorphosis, and get a leap ahead of your competitors through guest contributed articles, white papers, and company news.

Related ponderings.

Why Design Isn’t Just an Up-Front Activity in Software Development

A Good User Experience for the Legal Industry UX Matters

How Our Expectations for Modern User Experience Evolved in 2020 UX Matters

Let our team client logomark/filled/color be your team, learn how we put these lessons into practice, inspired curious enraged let’s start a conversation..

We don't support Internet Explorer

Please use Chrome, Safari, Firefox, or Edge to view this site.

Research on curriculum resources in mathematics education: a survey of the field

• Original Paper
• Open access
• Published: 15 March 2024

Cite this article

You have full access to this open access article

• Sebastian Rezat   ORCID: orcid.org/0000-0002-8946-894X 1  

154 Accesses

Explore all metrics

This survey describes the structure of the field of research on curriculum resources in mathematics education in the period from 2018 till 2023. Based on the procedures of a systematic review relevant literature was identified using Web of Science as a database. The included literature was analyzed and categorized according to the type of curriculum resource and the area of study. Seven areas of studies were identified: studies on the role of curriculum resources, content analysis, user studies, studies on the effects of curriculum resources, studies on curriculum resource design, curriculum resources as data, and reviews. The areas were further subdivided into different subcategories based on the research questions of the included papers. The findings show that research on mathematics textbooks is still predominant in the field. The most popular areas of research are content analysis, user studies, studies on design, and studies on effects. Emerging areas are research on students’ use of curriculum resources and the employment of user data from digital curriculum resources as data basis in mathematics education research.

Similar content being viewed by others

Ethical principles for artificial intelligence in education

Andy Nguyen, Ha Ngan Ngo, … Bich-Phuong Thi Nguyen

Systematic review of research on artificial intelligence applications in higher education – where are the educators?

Olaf Zawacki-Richter, Victoria I. Marín, … Franziska Gouverneur

Assistive technology for the inclusion of students with disabilities: a systematic review

José María Fernández-Batanero, Marta Montenegro-Rueda, … Inmaculada García-Martínez

Avoid common mistakes on your manuscript.

1 Introduction

Ten years ago, Fan ( 2013 ) and Fan et al. ( 2013 ) provided two comprehensive overviews of the status and future directions of textbook research as a field in mathematics education research. At that time, textbooks were still the generic and dominant curriculum resource. In the past ten years, this status of textbooks has been challenged. Due to digitalization, many new resources have been developed that can be regarded as curriculum resources: for example, paper textbooks have been transformed into e-textbooks. This has not only changed the modality but also yielded new features and design modes (Pepin et al., 2016 ). Platforms are providing opportunities to learn mathematics on a curricular basis and are becoming more and more influential. The  growing variety of curriculum resources is evident in comprehensive volumes such as the one edited by Fan et al. ( 2018 ) and in special issues of ZDM – Mathematics Education (53(5), Rezat et al., 2021 ; 50(6), Schubring & Fan, 2018 ), which comprise an increasing number of papers related to digital curriculum resources. This development is also apparent in volumes and special issues not specifically dedicated to textbooks, but to a wider range of resources (Clark-Wilson et al., 2020 ; Engelbrecht et al., 2020 ; Pepin et al., 2013 , 2017 ; Trouche et al., 2018 ). The Handbook of Digital Resources in Mathematics Education (Pepin et al., 2024 ) provides an up-to-date and comprehensive overview of the recent developments. The COVID-19 pandemic with its requirements for distance learning due to school closures in many countries has even accelerated this development. However, a systematic overview of the field of research on CR is still missing.

In 2013, Fan characterized the field of textbook research in comparison to other fields of research in mathematics education as “still at an early stage of development” (Fan, 2013 , p. 766). Ten years have passed since then, and therefore, it is appropriate to survey the status of the field again. However, due to the above-mentioned developments, it is necessary to broaden the perspective and include a wide array of curriculum resources and not only textbooks. Accordingly, this paper aims to give an account of how the field of research on CRs has evolved in the past five years. This time covers the period since two important overviews of the field of curriculum resources were published: The ZDM – Mathematics Education special issue “Recent advances in mathematics textbook research and development” (Schubring & Fan, 2018 ) and the ICME-13 Monograph “Research on mathematics textbooks and teachers’ resources: Advances and issues” (Fan et al., 2018 ).

2 Curriculum resources (CRs): terms and definitions

The field under study is characterized by a diverse and ambiguous terminology. Therefore, it is sometimes difficult to identify the actual objects of study that lie behind the used terminology. Consequently, it is important to clarify the terminology used in this article.

In its broad meaning, the notion of resources for school mathematics “extends beyond basic material and human resources to include a range of other human and material resources, as well as mathematical, cultural, and social resources.” (Adler, 2000 , p. 210). Ruthven ( 2019 ) refers to a narrower meaning of the notion of resources that developed in the 1960s to denote “curriculum-related materials intended to support learning or teaching activity” (p. 44). In this paper, I refer to the latter meaning. Nevertheless, this set of resources comprises “a wide array of programs and tools, print and digital” (Remillard et al., 2020 , p. 3). Curriculum-relatedness is a defining characteristic though.

The term curriculum has different meanings. It is widely associated with the ideas of structure and sequencing of opportunities to learn (OTL). According to Remillard and Kim ( 2020 , p. 3) the adjunct ‘curriculum’ in ‘curriculum material’ expresses that the materials contain an “intended learning progression for particular mathematical domains” and thus refers to “a course or pathway for learning.” This definition shares commonalities with the notion of Learning Trajectory (LT). While some authors define LTs as “research-based frameworks developed to document in detail the likely progressions, over long periods of time, of students’ reasoning about big ideas in mathematics” (Confrey et al., 2014 , p. 720) and thus do not connect them closely to OTL that engender the relevant mental processes, others regard OTL as a constituent part of LTs (e.g., Clements & Sarama, 2004 ; Simon & Tzur, 2004 ). Advocates of the latter understanding underline that the difference between LTs and learning progressions or developmental sequences is that they are “inextricable interconnected with instruction” in the form of “instructional tasks and pedagogical strategies” (Clements et al., 2019 , pp. 2512–2514). Thus, LTs in the latter understanding also provide sequences of OTL designed to reach an instructional goal. The distinguishing feature is the structure of these OTL as they are organized based on a hypothesis of learners’ cognitive development in the content area.

Focusing on digital curriculum resources, Pepin et al., ( 2017 , p. 647) emphasize that “it is the attention to sequencing—of grade-, or age-level learning topics, or of content associated with a particular course of study (e.g., algebra)—so as to cover (all or part of) a curriculum specification” that distinguish digital curriculum resources form other digital resources. This definition comprises a third idea besides the ideas of structure and sequencing. By mentioning the requirement of covering a “curriculum specification” Pepin et al. ( 2017 ) relate the idea of sequencing and structure to aims and intentions provided by an external specification. This is also apparent in the definition of curriculum provided by Schmidt et al. ( 1997 ) who attribute this specification of aims and goals to an educational authority: “curriculum provides a basic outline of planned and sequenced educational opportunities that express the “aims and intentions of educational authorities” (p. 4). Thus, it is the “idea of structure imposed by authority for the purpose of bringing order to the conduct of schooling” (p. 4).

Curriculum in this understanding can be mediated by different means. Traditionally, these were official curriculum documents, textbooks, and teacher guides. However, CR can be also included in digital technologies, learning environments, or platforms. This review refers to research related to all kinds of resources mediating curriculum.

Thus, in this paper, the term CRs is used to denote all kinds of analogous or digital materials mediating curriculum understood as structured and sequenced progressions of OTL over time, i.e. for a particular mathematical domain, age, or grade level related to the aims and intentions of educational authorities.

3 Analytical framework

As there is no prior synthesis that provides an overview of the broad field of CRs, this review cannot build on existing systematizations of the field. However, Fan et al. ( 2013 ) provided a synthesis of the field of textbook research that can be used as a starting point. This synthesis identifies four categories of research on mathematics textbooks:

Studies focusing on the role of textbooks ; This category relates to studies that focus on the relationship between textbooks and the official curriculum, as well as on textbooks as a means to guide or govern classroom instruction.

Studies analyzing the content of textbooks ; This category comprises studies that analyze the content of a single book, a book series, or different books or book series from either one country or several countries. This category also includes comparative studies comparing similarities and differences between the analyzed books or book series. Fan et al. ( 2013 ) distinguish three further subareas of content that are analyzed: “mathematics content and topics”, “cognition and pedagogy”, and “gender, ethnicity, equity, culture and value”.

Studies on how textbooks are used by teachers and/or students and how this use influences mathematics teaching and learning.

Other areas , comprising the effects of textbooks on students’ achievement and other student variables, such as identity in mathematics learning as well as studies on e-textbooks.

The scheme by Fan et al. ( 2013 ) was developed to systematize research on mathematics textbooks. As textbooks are a particular kind of curriculum resource, there is supposedly a considerable overlap between the two fields. However, the scheme likely needs to be amended to achieve a better fit for the broader field.

4 Aims and scope

The main aim of this review paper is to survey the field of research on CRs and to develop a differentiated framework for systematizing the research in this field. This step is necessary before providing a research synthesis of the results of research on CRs. The latter can be done subsequently by building on this survey and focusing on specific subthemes.

The framework provided by Fan et al. ( 2013 ) is used as an initial access to the field of research on CRs and will be extended in the course of the analysis—if necessary—to apply to the broader field. On the one hand, the amendments will show in what way research on curriculum resources differs from research on mathematics textbooks, and on the other hand, it will help to identify new trends in research on mathematics textbooks and other curriculum resources that were not apparent ten years earlier.

Furthermore, the intention is to characterize each area of research in more detail and develop subcategories for each area. Accordingly, the literature from 2018–2023 is analyzed according to three questions:

What types of CRs can be identified in the literature since 2018 and how is research on CRs distributed over different types?

What different research areas in the field of CRs can be identified and how is research on CRs since 2018 distributed over these areas?

Which subcategories of the major areas of research on CRs can be identified?

5 Methodology

A literature search was carried out using Web of Science as a database. Based on the theoretical considerations about the terminology, the following search term was used:

mathematics (Topic) and resource* OR textbook* OR curriculum OR "curriculum material*" OR "curriculum program*" OR "learning material*" OR "learning trajector*" OR platform* OR "learning environment*" (Title)

As pointed out in the section on terms and definitions, resources referred to as “platform”, “learning trajectory”, or “learning environment” may also mediate curriculum specifications by providing sequenced sets of OTL for a particular mathematical topic. Therefore, these terms were included in the search term to achieve a broad overview of contexts in which CRs are studied.

Before the results were screened, the results were restricted to publications from 2018 or later, covering the range of the past five years. This yielded n = 481 results. The number of results was further reduced by only including publications in English published in the 20 most important journals in mathematics education Footnote 1 and relevant conference proceedings. Furthermore, all chapters from books and from other journals with a Journal Citation Indicator ( Web of Science ) equal to or more than 1.0 Footnote 2 in either 2020 or 2021 were included.

The obtained results were screened by titles and abstracts. Exclusion criteria were (a) not being a study related to mathematics or (b) curriculum resources were not the major object of study. Two examples are provided to illustrate this procedure:

Example 1: The paper by Wan and Lee ( 2022 ) refers to science as the major content area in the title: “Coherence of Topics from Middle-School Integrated Science Textbooks from Taiwan and Korea”. Therefore, it was excluded due to criterion 1.

Example 2: The paper by Fonger et al. ( 2020 ) is an example of a study related to the ambiguous case of LTs that was included because it relates to CRs. The title of the paper generally refers to the notion of ‘learning trajectory’ which was part of the search term: “A quadratic growth learning trajectory”. From this title, it is not clear if this study relates to an understanding of LTs which can be regarded as CR. Therefore, the abstract was checked. There, the authors specify that they define a LT as “a series of transitions in students’ ways of thinking (WoT) and ways of understanding (WoU) quadratic growth in response to instructional supports emphasizing change in linked quantities”. In the paper, it is further specified that the authors “define a learning trajectory to be an empirically based model of students’ understandings, along with an account of changes in understanding in relation to students’ interaction with instructional supports including mathematical tasks, tools and representations, and teacher moves” (Fonger et al., 2020 , p. 3). As this definition considers OTL to be part of the LT, the paper was included.

Applying the exclusion criteria reduced the number of relevant papers to n = 310. An overview of the whole procedure is provided in Fig. 1 . The remaining papers were tagged based on screening titles and abstracts applying the four categories provided by Fan et al. ( 2013 ): Role of CRs, Content of CRs, use of CRs, and Other .

Flow diagram summarizing the flow of information through the different phases of the systematic review

To develop subcategories that enable a more differentiated characterization of studies in each of the four main categories, open coding procedures from Grounded Theory (Corbin & Strauss, 2015 ) were applied. Based on the constant comparison method, studies were grouped according to the similarities of their objects of study and their research questions. This was done until saturation of the categories was achieved, i.e., until all studies in one area could be attributed to one or more of the developed subcategories.

As the main aim of this survey is to give an overview of research on CRs over different types of CRs and areas of research, rigor and quality of the included studies were not evaluated. It is argued that this was part of the review process that these studies had to undergo to be published in the included high-ranked journals, proceedings, or books.

6 Types of CRs

Table 1 and Fig.  2 show the distribution of studies according to the type of CR. About 48 percent of the studies in the literature sample investigate issues related to mathematics textbooks. This is followed by studies investigating issues related to curriculum (19%), curriculum resources (9.7%), and resources (8.1%).

Distribution of types of resources over the sample of studies

A brief description of the different types of CRs is provided. This overview is restricted to types comprising more than ten studies.

6.1 Textbooks

The largest number of studies in the sample is devoted to textbooks as the generic type of CR. Altogether 150 papers (48.4%) were categorized as textbook studies; However, only 24 (16%) of these focus on digital textbooks. More than half of the studies in this category (57%) analyze the content of mathematical textbooks. Almost a quarter of them (23%) investigate the use of textbooks by teachers or students, and 14% analyze the effects of textbooks on teacher or student variables. Seventeen (11%) investigate issues related to the design of textbooks. The remainder comprises studies related to governance issues and reviews.

6.2 Curriculum

The 59 studies (19%) in this category investigate issues related to official curriculum documents as prescribed by educational authorities. Depending on the organization of the school system, these may be national, district-, or school-level curricula. 42% of these studies analyze the content of official curriculum documents. For example, Ow-Yeong et al. ( 2023 ) analyze how and to what extent data knowledge and skills are learned and assessed within the existing mathematics curriculum in Singapore in comparison to other mathematics content domains. Seventeen studies (29%) make suggestions for improving the design of curricula including design principles (e.g., Dreyfus et al., 2021 ). Issues of implementation are also investigated by nineteen of the studies (32%). These may be the design of OTL based on curriculum prescriptions or factors that might affect the implementation of curriculum reforms such as teachers’ perceptions of curriculum reform (e.g., Byrne & Prendergast, 2020 ). Understanding the mechanisms of curriculum reform in educational systems is also an issue in several studies (e.g., Yoon et al., 2021 ). Besides the official curriculum documents, these studies may also include other CRs, e.g., textbooks.

6.3 Curriculum resources (CRs)

This category comprises all studies that investigate issues related to a wider range of CRs that are not specified as textbooks or curricula. Studies may also focus on more than one CR, e.g., the set of CRs provided by curriculum programs such as textbooks, teacher guides, and further supplementary materials for teachers. As opposed to the category resources the focus of these studies is explicitly and solely on curriculum resources. This category also includes studies related to CRs that are not specified in the paper. Almost 10% of the studies relate to CRs.

6.4 Resources

The 25 studies (8.1%) in this category mostly investigate CRs as part of a wider set of resources, such as video-recorded lectures (e.g., Kempen & Liebendorfer, 2021 ; Maclaren, 2018 ), materials provided in professional development programs (e.g., Ntow & Adler, 2019 ), manipulatives, and even social and cognitive resources (Pepin & Kock, 2021 ). Major issues are the selection of resources from an array of available resources and their interplay building on notions such as resource system (e.g., Trouche et al., 2018 ).

6.5 Learning trajectories (LTs)

17 studies in the sample (5.5%) focus on LTs. 41 percent of these studies are concerned with the design or validation of LTs or related assessments. As these studies explicitly or implicitly follow a design research methodology they usually include data related to the use or effectiveness of a LT. The same proportion of studies (41%) is concerned with the effectiveness of LTs. Five studies on LTs have an explicit focus on use and implementation including adaptions of LTs.

6.6 Learning environments (LEs)

The 13 studies on LEs (4.2%) usually take a broader perspective. To be included in this review, the LEs comprise a sequenced set of OTLs to achieve a learning goal as prescribed in a curriculum specification. These are usually either supplemented by technology or integrated into online environments. Further features that distinguish them from other CRs are adaptiveness and immediate feedback, a focus on collaboration, or the implementation of particular pedagogical approaches such as gamification or game-based-learning (e.g., de Mooij et al., 2022 ), embodied learning (e.g., Duijzer et al., 2019 ), or problem-solving with realistic problems combined with simulation or Virtual Reality (e.g., Zwart et al., 2022 ). Some studies take instruction based on a specific CR as a starting point and analyze characteristics of the broader LE (e.g., Berlin & Cohen, 2020 ). Other studies in this category use instruction relying on a classical CR such as the textbook as a control condition compared to instruction implementing a particular learning environment (e.g., Birgin & Topuz, 2021 ).

6.7 Technology

Altogether 11 studies (3.5%) belong to the category “technology”. These studies either use technology that comprises sequenced OTL and thus match the definition of CRs used in this paper, or they investigate issues related to the integration of technology into a curriculum or matters of coordination of different resources including technology and CRs (e.g., Clark-Wilson & Hoyles, 2019 ; Fonger, 2018 ). In the latter case, the technology itself is not a CR but an amendment to CRs. This category also comprises studies that use a CR—mostly textbooks—as a control condition to be compared with a technology-rich intervention (e.g., Birgin & Topuz, 2021 ).

7 Areas of research on CRs

All studies in the sample were coded according to the categories by Fan et al. ( 2013 ): Role, Content Analysis, User Studies, and Other . During the coding process, subcategories that further differentiate the studies in the category Other emerged. These were Design , Effects , CRs as Data , and Reviews .

The coding procedure yielded the distribution of studies over the categories as depicted in Table  2 and Fig. 3 . Most studies are assigned to the category Content Analysis , followed by User Studies , studies related to the Design of CRs, and studies on the Effects of CRs. Only very few studies focus on the Role of CRs.

Distribution of studies on CRs over the different areas

Studies in the category Role analyze how CRs are embedded in broader activities such as reform or the governance of the education system. For example, Polikoff et al. ( 2020 ) ask, how California school districts make decisions about which textbooks to adopt in the core subjects, and the factors that influence these decisions; de Carvalho ( 2018 ) describes the governmental textbook assessment system in Brazil. In these studies, the focus is not on the CR itself but on the policy surrounding societal decisions about CRs.

7.2 Content analysis

Studies in the category Content Analysis apply methods of content or document analysis to make assertions about the content of CRs. The studies in this category can be systematized based on their research questions into the three subcategories presented in Table  3 .

Most studies belong to subcategory 1 and thus aim to answer variations of the question How is (the content related to topic/competence X in) CR A characterized (compared to CR B) in terms of feature α? Studies in this subcategory differ in that they either analyze the content related to a particular topic or competence of CRs or aim to characterize the content of CRs as a whole. The characterization of the content may be either quantitative analyzing the distribution of a particular feature within a CR or across several CRs or qualitative.

Comparative studies are a subset of this category. As apparent in the generalized research questions in Table  3 , the direction of the questions in comparative studies is the same as in studies only focusing on one CR. However, comparative studies ask and answer these questions in relation to other CRs. CRs in comparative studies might be of the same kind, e.g., two textbooks, or of a different kind, e.g., comparing textbooks and the official curriculum. Studies investigating the alignment of textbooks to the official curriculum are an example of the latter type (e.g., Polikoff et al., 2021 ). Comparison between CRs of the same kind may be either between CRs co-existing at the same moment in time, e.g. comparing two textbooks from different textbook series, or between CRs that are from different periods taking a historical perspective (e.g., Jia & Yao, 2021 ).

A second subset of studies in the category Content Analysis uses CR as data to infer information about the educational system. For example, Karp ( 2021 ) investigates “how close contemporary Russian education is once again to American education” based on the analyses of textbooks and other CRs.

A third subset of studies in the category Content Analysis aims to make methodological contributions to content analysis. For example, Zhang et al. ( 2020 ) ask “what is a sufficiently effective sampling design to obtain an accurate representation of the OTL data and/or a simple measure of alignment covered/not-covered with the intended curriculum?” to suggest a method for textbook analysis that is less time consuming than coding the whole book.

Some studies focusing on the use, design, or effects of CRs build their argument or investigation on a content analysis of the involved CRs to make assertions about the relationship between specific characteristics of the content and the other areas. Thus, there is an intersection between studies in this category and studies in other categories.

7.3 User studies

Studies categorized as User Studies focus on the interaction between a user or a group of users and CRs. Based on the analysis of the interaction, the characteristics of this interaction are specified. User studies are mainly differentiated by the groups of users that are investigated.

7.3.1 Research on teachers’ interactions with CRs

Altogether 69 studies (75%) in the sample were classified as User Studies with teachers as users. These studies were subdivided into six sub-categories based on their research questions. The different subcategories obtained are shown in Table  4 .

As studies often have more than one research question, one study may belong to more than one of the subcategories in Table  4 . For example, attending and noticing are often coupled with the aim of understanding teachers’ adaptions of CRs when planning and enacting mathematics instruction. Therefore, some of these studies belong to categories 2 and 3.

The largest number of studies classified as User Studies investigate teachers' use of or interaction with CRs to develop a better understanding of this interaction. Several studies in this subcategory build on the notions of “instrumentalization” and “instrumentation” from the instrumental or documentation approach (Gueudet & Trouche, 2009 ; Rabardel, 2002 ) to conceptualize this interaction (e.g., Mesa et al., 2021 ; Misfeldt et al., 2019 ).

As user studies need to build on an understanding of the used CRs, there is an intersection between the categories User Study and Content Analysis (3 studies, e.g., Pansell & Boistrup, 2018 ; Remillard et al., 2019 ). There is also an intersection with the category Design (4 studies).

7.3.2 Research on students’ use of CRs

Investigating students’ selection of and interaction with CRs is a theme of growing prominence. Based on their research questions, studies can be further differentiated in the sub-categories presented in Table  5 . Several studies on students’ use of CRs belong to two of these sub-categories.

Especially at tertiary level, a growing number of studies investigate students' preferences for specific resources (including CRs) and the reasons explaining the findings. In the sample, four studies investigate students’ selection and use of resources at the tertiary level. While the focus of all studies included in this review is mathematics, the studies in the tertiary context vary in the major study subjects of the participants (business, computer science, engineering, mathematics, physics). Also, the types of resources included and the conditions under which the courses were taught vary. For example, Kempen and Liebendorfer ( 2021 ) investigate German students’ use of resources in a fully digital learning environment during the COVID-19 pandemic; Howard et al. ( 2018 ) study Irish students’ preferences in a context where students have the choice between attending live lectures or watching lecturer-prepared videos; Pepin and Kock ( 2021 ) investigate Dutch students’ use of resources in courses based on a particular pedagogical approach, namely challenge-based education.

At secondary level, the use of CRs by students gets increasing attention in particular related to self-regulated learning (e.g., Otieno & Povey, 2022 ; Wang & Fan, 2021 ). Students’ use of CRs is also studied at primary level. There, the focus is not on self-regulated learning, but more on students’ meaning-making of information from CRs (e.g., Norberg, 2022 ).

There is an intersection with studies on the effects of CRs (2 studies). These studies mainly aim to identify influential factors that could explain the observed effects (e.g., Shechtman et al., 2019 ).

Studies in this category focus on issues related to the design of CRs. They can be subdivided according to the different aims of the research. Subcategories are provided in Table  6 .

Some studies in this category merely describe the design of a particular CR or the related design process (subcategories 1 and 4). Several studies elaborate on design principles (subcategory 2). Among these, some argue for new design principles that gain importance due to changes in societies and the learning culture. For example, Barlovits et al. ( 2022 ) analyze the challenges encountered during distance learning in five European countries. Based on the identified challenges, they build a framework with design requirements for online learning environments in mathematics education. O'Halloran et al. ( 2018 ) investigate how the possibilities of digital artifacts afford particular design features such as new ways of representing and connecting knowledge. Gueudet et al. ( 2018 ) even go a step further and argue for “connectivity” as an important design principle for digital textbooks based on established conceptualizations of learning mathematics and “connectivism” as an epistemological position and theory of learning based on societal developments. However, only a few studies investigate the effect of particular design principles on teachers’ or learners’ behavior in intervention studies. For example, Clinton and Walkington ( 2019 ) investigate how different types of illustrations influence students’ immediate problem-solving accuracy and their learning. Some of the studies are also more exploratory investigating which design principles have a positive effect on specific student or teacher variables. For example, Edson and Phillips ( 2021 ) investigate which teacher dashboard features support teacher enactment of a problem-based mathematics curriculum embedded in a digital collaborative platform. As apparent from the two previous examples, design features may be intended to influence student or teacher variables. While these studies specifically start from a particular design principle or aim to identify related design features, studies in subcategory 3 start with analyzing learning behavior in general or interactions with CRs in particular to derive recommendations for the design of CRs from the findings. For example, Olsher and Even ( 2019 ) ask what changes teachers would make in the mathematics textbook they use in class if they were allowed to do so. Identifying influences on the design or the design process is the focus of the studies in subcategory 5. This category also comprises research on collaboration among teachers or in multi-professional teams (including teachers from other subjects, researchers, or students) as one influential factor. Finally, some studies make methodological or theoretical contributions related to the design of CRs. Literature reviews aiming to provide a synthesis of a particular aspect of learning mathematics to derive design principles from this are also included in this category.

7.5 Effects

Studies in this category are characterized by making assertions about the relationship between using or interacting with a CR and some other user variable. The research questions address this relationship either directly:

How is student/teacher variable X related to (a particular feature of) CR A? Or studies analyze the effect by relating variation in variables to different conditions that differ in terms of CRs: How is variation in student/teacher variables related to/explained by different conditions in terms of CRs? The effect of CRs may also be investigated as a moderator between two other variables: How does CR A moderate the relationship between student/teacher variables X and Y? These studies differ in how the analyzed variables and the relationship to CRs are measured.

Some studies are more explorative, investigating which features of CRs yield variation in student/teacher variable X. Most studies in this category are based on quantitative methods relating the used CRs to measures of user variables. Most typically, studies in this category are carried out in an intervention study design. Few studies aim at understanding the effect in a qualitative way characterizing how interaction with CRs influences students’ learning of mathematics (e.g., Moyer et al., 2018 ; Rezat, 2021 ).

Student variables taken into account are:

Mathematical achievement : The predominant number of studies in this category investigates the effects of CRs on students’ mathematical achievement in general (e.g., Shechtman et al., 2019 ) or related to a particular content area or competency (e.g., mathematical thinking: Drijvers et al., 2019a , 2019b ; adaptive expertise: Sievert et al., 2019 ). Some studies have a longitudinal perspective, covering a whole school year or even longer periods (e.g., van den Ham & Heinze, 2018 ), others are carried out as intervention studies over shorter periods.

Affective variables : A second set of student variables that is investigated by several studies are affective variables, especially attitude (towards mathematics) (e.g., Lindorff et al., 2019 ), motivation, or self-efficacy (e.g., Tarnanen et al., 2023 ). Affective variables are mostly investigated in combination with achievement.

Some studies investigate the effects of CRs on teacher variables. Predominantly, the effect of CRs on teaching practices is considered. Effects of CRs on teacher variables are mostly investigated in connection with effects on student variables. Few studies include data on teacher variables to determine if these moderate differences between the measures of student variables (e.g., Sievert et al., 2021 ). To develop a better understanding of the effects, a few studies (3) also analyze the content of the used CR and thus intersect with the category Content Analysis .

7.6 CRs as data

An emerging area particularly related to dCRs is the employment of user data from digital systems to make inferences about some other aspects of user behavior. The sample of literature comprised eight papers that use different types of user data to derive information about students’ interaction with dCRs and their learning behavior or achievement. Due to the small number of studies, no subcategories were identified.

Data used from dCRs may comprise the used tasks or sections of a textbook and related used materials, the received feedback (type and timing), students’ solutions (e.g., Castro-Rodriguez et al., 2022 ; Zhang et al., 2019 ) or the accuracy of the solutions (Spitzer & Moeller, 2022 ), measures such as time on task (Castro-Rodriguez et al., 2022 ; Hoch et al., 2018 ), or post-error slowing (de Mooij et al., 2022 ).

The studies in this area are mostly double-edged: On the one hand, they have a methodological aim in that they contribute to the operationalization of an aspect of students’ behavior based on user data provided by dCRs. On the other hand, they contribute to a better understanding of the interaction with dCRs and related learning of mathematics.

From the studies in this category, it becomes clear that dCRs provide rich sources of data. Most of these studies do not gather the data in experimental settings but in real learning scenarios, which ensures the ecological validity of the data. However, it also becomes clear that the enormous amounts of data from dCRs require methods that stem from big data analysis. At the same time, dCRs and big data analysis provide new possibilities to describe and understand students' learning of mathematics.

7.7 Reviews

This category contains review papers presenting an overview, summary, or reanalysis of other studies related to some aspect of research on CRs. This category also comprises introductory chapters to edited books or commentary papers/chapters.

8 Discussion and conclusion

The main aim of this review was to characterize and structure the field of research on curriculum resources (CRs). Therefore, a systematic review using Web of Science as a database was conducted. The papers included in this review were classified according to.

the type of CR studied, and

the area of research using the categories provided by Fan et al. ( 2013 ) as a starting point.

Subareas of interest based on a synthesis of research questions addressed by the studies in each area

The distribution of studies over the types of CRs showed that research on mathematics textbooks is still predominant in the field followed by research on curriculum, curriculum resources, and resources. This shows that research still predominantly focuses on the generic CR—the mathematics textbook—and only slowly takes into account that teaching and learning comprise the interaction with a broader range of different resources.

The distribution over the areas of research on CRs showed a more balanced picture than ten years ago. However, content analysis is still the predominant area followed by user studies, studies on design, and studies of effects. Nevertheless, the field has moved in directions identified as important for future research by Fan et al. ( 2013 ). The only area that is still not sufficiently developed is research on the role of CRs. Consequently, the call by Fan et al. to establish.

a more solid fundamental conceptualization and theoretical underpinning of the role of textbooks and the relationship between textbooks and other variables not only in curriculum, teaching and learning but also in a wider educational and social context (Fan et al., 2013 , p. 643)

can be repeated and extended to the broader field of research on CRs.The application of the categories by Fan et al. ( 2013 ) proved to be useful as a first approach to the broader field of CRs. Based on a qualitative analysis of the research questions, it was possible to differentiate these categories further, adapt them to this broader field, and provide an overview of the different areas of research on CRs and related research questions. Seven main categories of studies evolved: studies on the Role of CRs , Content Analysis of CRs, User Studies , studies on the Effects of CRs, studies on the Design of CRs, studies using CRs as Data , and Reviews . Sub-categories were developed that further differentiate these overarching areas of research related to CRs.

The categories show that the field has changed with the further development of CRs. With the introduction of other CRs besides mathematics textbooks issues of coordination, orchestration (Drijvers, Gitirana, et al., 2019), and connectivity (Gueudet et al., 2018 ; Pepin, 2021 ) in resource systems (Wang, 2018 ) became increasingly relevant. The diversification of different types of CRs can be mainly attributed to the fast-evolving development of dCRs. Nevertheless, only 15 percent of studies in this review focus on dCRs. This ratio is mirrored in the subcategory textbook, where almost the same proportion of studies (16%) focus on digital mathematics textbooks.

From the small number of studies that belong to the intersections of two areas, it becomes apparent that many issues related to CRs are addressed in isolation. For example, most studies in the category content analysis solely focus on a particular aspect of the content rarely taking into account the effect of the presentation of the content on teachers or students; studies in the category user study either address the use by teachers or by students. Rarely is the interrelations between the two user groups investigated. From the perspective of research methodology, this is only natural. However, as CRs are but one important agent in the didactical situation (Rezat & Sträßer, 2012 ), a more systemic view considering the interrelations between the mathematics, the CRs, the teachers, and the students is needed. Only a few examples were identified in this review that already take this route. This also becomes apparent in the integration of CRs in the wider context of teaching and learning (e.g., in learning environments or platforms) and how they connect more closely to students’ thinking (e.g., LTs). These efforts could be identified due to the wide perspective taken in this review, including LEs, platforms, and LTs in the search term. However, investigating the complex interrelations of CRs, the mathematics and the other agents in the didactical situation remains a methodological challenge in the field, especially in terms of feasibility. The close interrelationship of CRs with aspects of their users and mathematics shows that research on CRs can almost be considered a micro-cosmos of research in mathematics education. This is reinforced by the detailed account of research questions in the different areas of research on CRs provided in this survey as they cover a wide range of relevant research questions in mathematics education.

Especially related to dCRs, there is a trend that the demarcations between different kinds of resources vanish. Technology-rich learning environments increasingly comprise curricular OTL and thus become inseparately linked with CRs. Similarly, tools such as Dynamic Geometry or Computer Algebra Systems that were not considered curriculum resources also seem to develop in this direction. This is achieved by either combining them with other CRs in more complex learning environments or by enhancing the technologies through sequences of OTL adherent to a curriculum.

Furthermore, the introduction of dCRs has not only led to new issues relating to the design of CRs, but also to new research methodologies as subsumed in the category CR as data . While CRs have been already used as data to make inferences about the educational system (category Content Analysis , subcategory 2), the data that is collected by dCRs allows for making inferences about user variables. This emerging field will even attract more attention in combination with resources comprising artificial intelligence (AI). AI will likely become relevant in all other areas of research on CRs as well.

Due to the wide scope of this survey and the differentiated structure of the field, the findings related to each of the research questions in the different areas could not be synthesized within this article. It will possibly require several subsequent reviews to provide a differentiated synthesis of findings related to each of the sub-areas. In summary, the findings present a broad, rich, and very multifaceted knowledge base with many important contributions to a better understanding of issues related to the role, design, use, and effects of CRs. However, these findings are mostly singular and closely tied to their particular research design as well as their specific socio-cultural contexts. Within the field of research on CRs, it is not clear if and how these findings can be transferred to other socio-cultural contexts. In other words, due to their socio-cultural situatedness, the generalization of the findings is questionable. Consequently, there is a strong need for a more systematic cumulative development of scientific knowledge in the field of research on CRs. Replication studies could be one way to respond to this need. However, none were found in this survey. Additionally, a more solid and more widely shared theoretical foundation of important concepts is equally important to ensure that the findings of different studies are comparable.

This survey has taken a very general perspective. It provides an overview of issues investigated in the field of research on CRs and structures the field into different sub-areas with their related research questions. Thus, it is a good starting point for subsequent research syntheses within each of the fields.

9 Limitations

This study has some limitations that are important to consider when interpreting the results:

Only Web of Science was used as a database. Including other databases may provide a different picture.

As a consequence of using Web of Science as a database, most conference proceedings that are relevant to research in mathematics education were not included in this survey. The time that it takes to publish a paper in a peer-reviewed journal is usually much longer than publishing research in conference proceedings. Especially, since the field of dCRs is evolving very fast this might result in an overview that has not included the most recent developments in the field.

The results are further biased by the search term that was used to identify relevant literature in the database. The decision on the final search term was made after several rounds of exploratory searches and related analyses of the results. The final search term yielded the most relevant selection of literature according to the judgment of the author. However, other search terms may yield a different picture.

These were identified based on a review by Williams & Leatham ( 2017 ).

A value of 1.0 represents world average, with values higher than 1.0 denoting higher-than-average citation impact (2.0 being twice the average) and lower than 1.0 indicating less than average. ( https://clarivate.com/blog/introducing-the-journal-citation-indicator-a-new-field-normalized-measurement-of-journal-citation-impact/ ).

*Barlovits, S., Caldeira, A., Fesakis, G., Jablonski, S., Filippaki, D. K., Lazaro, C., Ludwig, M., Mammana, M. F., Moura, A., Oehler, D. X. K., Recio, T., Taranto, E., & Volika, S. (2022). Adaptive, synchronous, and mobile online education: Developing the ASYMPTOTE learning environment. Mathematics . https://doi.org/10.3390/math10101628

Article   Google Scholar  

*Berlin, R., & Cohen, J. (2020). The convergence of emotionally supportive learning environments and college and career ready mathematical engagement in upper elementary classrooms. Aera Open . https://doi.org/10.1177/2332858420957612

*Birgin, O., & Topuz, F. (2021). Effect of the GeoGebra software-supported collaborative learning environment on seventh grade students’ geometry achievement, retention and attitudes. Journal of Educational Research, 114 (5), 474–494. https://doi.org/10.1080/00220671.2021.1983505

*Byrne, C., & Prendergast, M. (2020). Investigating the concerns of secondary school teachers towards curriculum reform. Journal of Curriculum Studies, 52 (2), 286–306. https://doi.org/10.1080/00220272.2019.1643924

*Castro-Rodriguez, E., Mali, A., & Mesa, V. (2022). University students’ engagement with digital mathematics textbooks: A case of linear algebra. International Journal of Mathematical Education in Science and Technology . https://doi.org/10.1080/0020739X.2022.2147104

*Clark-Wilson, A., & Hoyles, C. (2019). From curriculum design to enactment in technology enhanced mathematics instruction: Mind the gap! International Journal of Educational Research, 94 , 66–76. https://doi.org/10.1016/j.ijer.2018.11.015

*Clark-Wilson, A., Robutti, O., & Thomas, M. (2020). Teaching with digital technology. ZDM Mathematics Education, 52 (7), 1223–1242. https://doi.org/10.1007/s11858-020-01196-0

Article   PubMed   Google Scholar  

Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical Thinking and Learning, 6 (2), 81–89. https://doi.org/10.1207/s15327833mtl0602_1

*Clements, D. H., Sarama, J., Baroody, A. J., Joswick, C., & Wolfe, C. B. (2019). Evaluating the efficacy of a learning trajectory for early shape composition. American Educational Research Journal, 56 (6), 2509–2530. https://doi.org/10.3102/0002831219842788

*Clinton, V., & Walkington, C. (2019). Interest-enhancing approaches to mathematics curriculum design: Illustrations and personalization. Journal of Educational Research, 112 (4), 495–511. https://doi.org/10.1080/00220671.2019.1568958

Confrey, J., Maloney, A. P., & Corley, A. K. (2014). Learning trajectories: A framework for connecting standards with curriculum. ZDM – the International Journal on Mathematics Education, 46 , 719–733. https://doi.org/10.1007/s11858-014-0598-7

Corbin, J., & Strauss, A. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory (4th ed.). Sage.

Google Scholar  

*de Carvalho, J. B. P. (2018). The Brazilian mathematics textbook assessments. ZDM Mathematics Education, 50 (5), 773–785. https://doi.org/10.1007/s11858-018-0949-x

*de Mooij, S. M. M., Dumontheil, I., Kirkham, N. Z., Raijmakers, M. E. J., & van der Maas, H. L. J. (2022). Post-error slowing: Large scale study in an online learning environment for practising mathematics and language. Developmental Science . https://doi.org/10.1111/desc.13174

*Dreyfus, T., Kouropatov, A., & Ron, G. (2021). Research as a resource in a high-school calculus curriculum. ZDM Mathematics Education, 53 (3), 679–693. https://doi.org/10.1007/s11858-021-01236-3

*Drijvers, P., Gitirana, V., Monaghan, J., Okumus, S., Besnier, S., Pfeiffer, C., Mercat, C., Thomas, A., Christo, D., Bellemain, F., Faggiano, E., Orozco-Santiago, J., Ndlovu, M., Van Dijke-Droogers, M., Ignacio, R. D., Swidan, O., Lealdino, P., de Albuquerque, R. M., Hadjerrouit, S., & Rodrigues, A. (2019a). Transitions toward digital resources: Change, invariance, and orchestration. In L. Trouche, G. Gueudet, & B. Pepin (Eds.), Resource approach to mathematics education (pp. 389–444). Springer. https://doi.org/10.1007/978-3-030-20393-1_12

Chapter   Google Scholar  

*Drijvers, P., Kodde-Buitenhuis, H., & Doorman, M. (2019b). Assessing mathematical thinking as part of curriculum reform in the Netherlands. Educational Studies in Mathematics, 102 (3), 435–456. https://doi.org/10.1007/s10649-019-09905-7

*Duijzer, C., Van den Heuvel-Panhuizen, M., Veldhuis, M., Doorman, M., & Leseman, P. (2019). Embodied learning environments for graphing motion: A systematic literature review. Educational Psychology Review, 31 (3), 597–629. https://doi.org/10.1007/s10648-019-09471-7

*Edson, A. J., & Phillips, E. D. (2021). Connecting a teacher dashboard to a student digital collaborative environment: Supporting teacher enactment of problem-based mathematics curriculum. ZDM Mathematics Education, 53 (6), 1285–1298. https://doi.org/10.1007/s11858-021-01310-w

*Engelbrecht, J., Llinares, S., & Borba, M. C. (2020). Transformation of the mathematics classroom with the internet. ZDM Mathematics Education, 52 (5), 825–841. https://doi.org/10.1007/s11858-020-01176-4

Fan, L. (2013). Textbook research as scientific research: Towards a common ground on issues and methods of research on mathematics textbooks. ZDM Mathematics Education, 45 (5), 765–777. https://doi.org/10.1007/s11858-013-0530-6

Fan, L., Trouche, L., Qi, C., Rezat, S., & Visnovska, J. (Eds.). (2018). Research on mathematics textbooks and teachers' resources: Advances and issues . Springer. https://doi.org/10.1007/978-3-319-73253-4 .

Fan, L., Zhu, Y., & Miao, Z. (2013). Textbook research in mathematics education: Development status and directions. ZDM Mathematics Education, 45 (5), 633–646. https://doi.org/10.1007/s11858-013-0539-x

*Fonger, N. L. (2018). An activity structure for supporting students’ coordination of computer algebra systems and paper-and-pencil across phases of curriculum. International Journal for Technology in Mathematics Education, 25 (1), 3–18. https://doi.org/10.1564/tme_v25.1.01

*Fonger, N. L., Ellis, A. B., & Dogan, M. F. (2020). A quadratic growth learning trajectory. Journal of Mathematical Behavior . https://doi.org/10.1016/j.jmathb.2020.100795

*Gueudet, G., Pepin, B., Restrepo, A., Sabra, H., & Trouche, L. (2018). E-textbooks and connectivity: Proposing an analytical framework. International Journal of Science and Mathematics Education, 16 (3), 539–558. https://doi.org/10.1007/s10763-016-9782-2

Article   ADS   Google Scholar  

Gueudet, G., & Trouche, L. (2009). Towards new documentation systems for mathematics teachers? Educational Studies in Mathematics, 71 (3), 199–218.

*Hoch, S., Reinhold, F., Werner, B., Richter-Gebert, J., & Reiss, K. (2018). Design and research potential of interactive textbooks: The case of fractions. ZDM Mathematics Education, 50 (5), 839–848. https://doi.org/10.1007/s11858-018-0971-z

*Howard, E., Meehan, M., & Parnell, A. (2018). Live lectures or online videos: Students’ resource choices in a first-year university mathematics module. International Journal of Mathematical Education in Science and Technology, 49 (4), 530–553. https://doi.org/10.1080/0020739X.2017.1387943

*Jia, S. J., & Yao, Y. L. (2021). 70 Years of problem posing in Chinese primary mathematics textbooks. ZDM Mathematics Education, 53 (4), 951–960. https://doi.org/10.1007/s11858-021-01284-9

*Karp, A. (2021). The Russian experience: National curricula, national standards, textbooks. ZDM Mathematics Education, 53 (7), 1471–1483. https://doi.org/10.1007/s11858-021-01268-9

*Kempen, L., & Liebendorfer, M. (2021). University students’ fully digital study of mathematics: An identification of student-groups via their resources usage and a characterization by personal and affective characteristics. Teaching Mathematics and Its Applications, 40 (4), 436–454. https://doi.org/10.1093/teamat/hrab020

*Lindorff, A. M., Hall, J., & Sammons, P. (2019). Investigating a Singapore-based mathematics textbook and teaching approach in classrooms in England. Frontiers in Education . https://doi.org/10.3389/feduc.2019.00037

*Maclaren, P. (2018). How is that done? Student views on resources used outside the engineering classroom. European Journal of Engineering Education, 43 (4), 620–637. https://doi.org/10.1080/03043797.2017.1396445

*Mesa, V., Ma, Y., Quiroz, C., Gerami, S., Liakos, Y., Judson, T., & Chamberlain, L. (2021). University instructors’ use of questioning devices in mathematics textbooks: An instrumental approach. ZDM Mathematics Education, 53 (6), 1299–1311. https://doi.org/10.1007/s11858-021-01296-5

*Misfeldt, M., Tamborg, A. L., Dreyøe, J., & Allsopp, B. B. (2019). Tools, rules and teachers: The relationship between curriculum standards and resource systems when teaching mathematics. International Journal of Educational Research, 94 , 122–133. https://doi.org/10.1016/j.ijer.2018.12.001

*Moyer, J. C., Robison, V., & Cai, J. F. (2018). Attitudes of high-school students taught using traditional and reform mathematics curricula in middle school: A retrospective analysis. Educational Studies in Mathematics, 98 (2), 115–134. https://doi.org/10.1007/s10649-018-9809-4

*Norberg, M. (2022). Young students’ meaning-making when working with mathematics textbooks: A multimodal study focusing on the designed and the discovered. Research in Mathematics Education, 25 (2), 194–218. https://doi.org/10.1080/14794802.2022.2045624

*Ntow, F. D., & Adler, J. (2019). Identity resources and mathematics teaching identity: An exploratory study. ZDM Mathematics Education, 51 (3), 419–432. https://doi.org/10.1007/s11858-019-01025-z

O’Halloran, K. L., Beezer, R. A., & Farmer, D. W. (2018). A new generation of mathematics textbook research and development. ZDM Mathematics Education, 50 (5), 863–879. https://doi.org/10.1007/s11858-018-0959-8

*Olsher, S., & Even, R. (2019). Organizing tools suggested by teachers in the mathematics textbook they use in class. International Journal of Science and Mathematics Education, 17 (7), 1381–1399. https://doi.org/10.1007/s10763-018-9902-2

*Otieno, H., & Povey, H. (2022). Mathematics textbooks and self-regulated learning: Responses from students in three Kenyan secondary schools. Research in Mathematics Education . https://doi.org/10.1080/14794802.2022.2089907

*Ow-Yeong, Y. K., Yeter, I. H., & Ali, F. (2023). Learning data science in elementary school mathematics: A comparative curriculum analysis. International Journal of Stem Education . https://doi.org/10.1186/s40594-023-00397-9

*Pansell, A., & Boistrup, L. B. (2018). Mathematics teachers’ teaching practices in relation to textbooks: Exploring praxeologies. Mathematics Enthusiast, 15 (3), 541–562.

*Pepin, B. (2021). Connectivity in support of student co-design of innovative mathematics curriculum trajectories. ZDM Mathematics Education, 53 (6), 1221–1232. https://doi.org/10.1007/s11858-021-01297-4

Pepin, B., Choppin, J., Ruthven, K., & Sinclair, N. (2017). Digital curriculum resources in mathematics education: Foundations for change. ZDM Mathematics Education, 49 (5), 645–661. https://doi.org/10.1007/s11858-017-0879-z

Pepin, B., Gueudet, G., & Choppin, J. (Eds.) (2024). Handbook of digital resources in mathematics education. Springer https://doi.org/10.1007/978-3-030-95060-6 .

Pepin, B., Gueudet, G., & Trouche, L. (2013). Re-sourcing teachers’ work and interactions: A collective perspective on resources, their use and transformation. ZDM – the International Journal on Mathematics Education, 45 (7), 929–943. https://doi.org/10.1007/s11858-013-0534-2

Pepin, B., Gueudet, G., Yerushalmy, M., Trouche, L., & Chazan, D. I. (2016). E-textbooks in/for teaching and learning mathematics. In L. D. English & D. Kirshner (Eds.), Handbook of international research in mathematics education (3rd ed., pp. 636–661). Routledge.

*Pepin, B., & Kock, Z. J. (2021). Students’ use of resources in a challenge-based learning context involving mathematics. International Journal of Research in Undergraduate Mathematics Education, 7 (2), 306–327. https://doi.org/10.1007/s40753-021-00136-x

*Polikoff, M. S., Campbell, S. E., Rabovsky, S., Koedel, C., Le, Q. T., Hardaway, T., & Gasparian, H. (2020). The formalized processes districts use to evaluate mathematics textbooks. Journal of Curriculum Studies, 52 (4), 451–477. https://doi.org/10.1080/00220272.2020.1747116

*Polikoff, M. S., Rabovsky, S. J., Silver, D., & Lazar-Wolfe, R. (2021). The equitable distribution of opportunity to learn in mathematics textbooks. Aera Open . https://doi.org/10.1177/23328584211065712

Rabardel, P. (2002). People and technology: A cognitive approach to contemporary instruments . https://hal.archives-ouvertes.fr/hal-01020705/document . Accessed 12 Mar 2024.

*Remillard, J. T., & Kim, O.-K. (Eds.). (2020). Elementary mathematics curriculum materials: Designs for student learning and teacher enactment . Springer. https://doi.org/10.1007/978-3-030-38588-0 .

*Remillard, J. T., Reinke, L. T., & Kapoor, R. (2019). What is the point? Examining how curriculum materials articulate mathematical goals and how teachers steer instruction. International Journal of Educational Research, 93 , 101–117. https://doi.org/10.1016/j.ijer.2018.09.010

*Rezat, S. (2021). How automated feedback from a digital mathematics textbook affects primary students’ conceptual development: Two case studies. ZDM Mathematics Education, 53 (6), 1433–1445. https://doi.org/10.1007/s11858-021-01263-0

*Rezat, S., Fan, L., & Pepin, B. (2021). Mathematics textbooks and curriculum resources as instruments for change. ZDM Mathematics Education, 53 (6), 1189–1206. https://doi.org/10.1007/s11858-021-01309-3 . ZDM - Mathematics Education.

Rezat, S., & Sträßer, R. (2012). From the didactical triangle to the socio-didactical tetrahedron: Artifacts as fundamental constituents of the didactical situation. ZDM – the International Journal on Mathematics Education, 44 (5), 641–651. https://doi.org/10.1007/s11858-012-0448-4

*Ruthven, K. (2019). The construct of ‘resource system’ as an analytic tool in understanding the work of teaching. In L. Trouche, G. Gueudet, & B. Pepin (Eds.), Resource approach to mathematics education (pp. 43–59). Springer. https://doi.org/10.1007/978-3-030-20393-1_3

Schmidt, W. H., McKnight, C. C., Valverde, G. A., Houang, R. T., & Wiley, D. E. (1997). Many visions, many aims. A cross-national investigation of curricular intentions in school mathematics (Vol. 1). Kluwer.

Book   Google Scholar  

*Schubring, G., & Fan, L. (2018). Recent advances in mathematics textbook research and development: An overview. ZDM Mathematics Education, 50 (5), 765–771. https://doi.org/10.1007/s11858-018-0979-4

*Shechtman, N., Roschelle, J., Feng, M. Y., & Singleton, C. (2019). An efficacy study of a digital core curriculum for grade 5 mathematics. Aera Open . https://doi.org/10.1177/2332858419850482

*Sievert, H., van den Ham, A. K., & Heinze, A. (2021). Are first graders’ arithmetic skills related to the quality of mathematics textbooks? A study on students’ use of arithmetic principles. Learning and Instruction . https://doi.org/10.1016/j.learninstruc.2020.101401

*Sievert, H., van den Ham, A. K., Niedermeyer, I., & Heinze, A. (2019). Effects of mathematics textbooks on the development of primary school children’s adaptive expertise in arithmetic. Learning and Individual Differences . https://doi.org/10.1016/j.lindif.2019.02.006

Simon, M. A., & Tzur, R. (2004). Explicating the role of mathematical tasks in conceptual learning: An elaboration of the hypothetical learning trajectory. Mathematical Thinking and Learning, 6 (2), 91–104. https://doi.org/10.1207/s15327833mtl0602_2

*Spitzer, M. W. H., & Moeller, K. (2022). Predicting fraction and algebra achievements online: A large-scale longitudinal study using data from an online learning environment. Journal of Computer Assisted Learning, 38 (6), 1797–1806. https://doi.org/10.1111/jcal.12721

*Tarnanen, M., Raikkonen, E., Martin, A., Kaukonen, V., Kostiainen, E., Toikka, T., & Vauhkonen, V. (2023). Pupils’ academic self-efficacy in subject-specific and integrated curriculum instruction. Scandinavian Journal of Educational Research, 67 (2), 252–267. https://doi.org/10.1080/00313831.2021.2006303

*Trouche, L., Gueudet, G., & Pepin, B. (2018). Open educational resources: A chance for opening mathematics teachers’ resource systems? In L. Fan, L. Trouche, C. Qi, S. Rezat, & J. Visnovska (Eds.), Research on mathematics textbooks and teachers’ resources: Advances and issues (pp. 3–27). Springer. https://doi.org/10.1007/978-3-319-73253-4_1

*Trouche, L., Gueudet, G., & Pepin, B. (Eds.). (2019). The ‘resource’ approach to mathematics education . Springer.

*van den Ham, A. K., & Heinze, A. (2018). Does the the textbook matter? Longitudinal effects of textbook choice on primary school students’ achievement in mathematics. Studies in Educational Evaluation, 59 , 133–140. https://doi.org/10.1016/j.stueduc.2018.07.005

*Wan, D. S., & Lee, Y. J. (2022). Coherence of topics from middle-school integrated science textbooks from Taiwan and Korea. International Journal of Science and Mathematics Education, 20 (5), 881–899. https://doi.org/10.1007/s10763-021-10187-w

*Wang, C. Y. (2018). Mathematics teachers’ expertise in resources work and its development in collectives: A French and a Chinese cases. In L. Fan, L. Trouche, C. Qi, S. Rezat, & J. Visnovska (Eds.), Research on mathematics textbooks and teachers’ resources: Advances and issues (pp. 193–213). Springer. https://doi.org/10.1007/978-3-319-73253-4_9

*Wang, Y., & Fan, L. H. (2021). Investigating students’ perceptions concerning textbook use in mathematics: A comparative study of secondary schools between Shanghai and England. Journal of Curriculum Studies, 53 (5), 675–691. https://doi.org/10.1080/00220272.2021.1941265

Williams, S. R., & Leatham, K. R. (2017). Journal quality in mathematics education. Journal for Research in Mathematics Education, 48 (4), 369–396.

*Yoon, H., Bae, Y., Lim, W., & Kwon, O. N. (2021). A story of the national calculus curriculum: How culture, research, and policy compete and compromise in shaping the calculus curriculum in South Korea. ZDM Mathematics Education, 53 (3), 663–677. https://doi.org/10.1007/s11858-020-01219-w

*Zhang, J. H., Cogan, L. S., & Schmidt, W. H. (2020). Measuring textbook content coverage: Efficient content analysis with lesson sampling. Educational Measurement-Issues and Practice, 39 (2), 74–84. https://doi.org/10.1111/emip.12300

*Zhang, J. J., Gao, M., Holmes, W., Mavrikis, M., & Ma, N. (2019). Interaction patterns in exploratory learning environments for mathematics: A sequential analysis of feedback and external representations in Chinese schools. Interactive Learning Environments, 29 (7), 1211–1228. https://doi.org/10.1080/10494820.2019.1620290

*Zwart, D. P., Goei, S. L., Van Luit, J. E. H., & Noroozi, O. (2022). Nursing students’ satisfaction with the instructional design of a computer-based virtual learning environment for mathematical medication learning. Interactive Learning Environments . https://doi.org/10.1080/10494820.2022.2071946

Note: References with an asterix were retrieved from the literature search

Adler, J. (2000). Conceptualizing resources as a theme for teacher education. Jorunal of Mathematics Teacher Education, 3 (3), 205–224.

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Paderborn University, Paderborn, Germany

Sebastian Rezat

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sebastian Rezat .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Rezat, S. Research on curriculum resources in mathematics education: a survey of the field. ZDM Mathematics Education (2024). https://doi.org/10.1007/s11858-024-01559-x

Download citation

Accepted : 16 February 2024

Published : 15 March 2024

DOI : https://doi.org/10.1007/s11858-024-01559-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mathematics
• Curriculum resources
• Curriculum materials
• Learning trajectories
• Learning environments
• Systematic review
• Find a journal
• Publish with us
• Track your research

ChatGPT is an effective tool for planning field work, school trips and even holidays

Researchers exploring ways to utilise ChatGPT for work, say it could save organisations and individuals a lot of time and money when it comes to planning trips.

A new study, published in Innovations in Education and Teaching International (IETI), has tested whether ChatGPT can be used to design University field studies. It found that the free-to-use AI model is an effective tool for not only planning educational trips around the world, but also could be used by other industries.

The research, led by scientists from the University of Portsmouth and University of Plymouth, specifically focused on marine biology courses. It involved the creation of a brand new field course using ChatGPT, and the integration of the AI-planned activities into an existing university module.

The team developed a comprehensive guide for using the chatbot, and successfully organised a single-day trip in the UK using the AI's suggestion of a beach clean-up activity to raise awareness about marine pollution and its impact on marine ecosystems.

They say the established workflow could also be easily adapted to support other projects and professions outside of education, including environmental impact studies, travel itineraries, and business trips.

Dr Mark Tupper, from the University of Portsmouth's School of Biological Sciences, said: "It's well known that universities and schools across the UK are stretched thin when it comes to resources. We set out to find a way to utilise ChatGPT for planning field work, because of the considerable amount of effort that goes into organising these trips. There's a lot to consider, including safety procedures, risks, and design logistics. This process can take several days, but we found ChatGPT effectively does most of the leg work in just a few hours. The simple framework we've created can be used across the whole education sector, not just by universities. With many facing budget constraints and staffing limitations, this could save a lot of time and money."

Chatbots like ChatGPT are powered by large amounts of data and computing techniques to make predictions to string words together in a meaningful way. They not only tap into a vast amount of vocabulary and information, but also understand words in context.

Since OpenAI launched the 3.0 model in November 2022, millions of users have used the technology to improve their personal lives and boost productivity. Some workers have used it to write papers, make music, develop code, and create lesson plans.

"If you're a school teacher and want to plan a class with 40 kids, our ChatGPT roadmap will be a game changer," said Dr Reuben Shipway, Lecturer in Marine Biology at the University of Plymouth. "All a person needs to do is input some basic data, and the AI model will be able to design a course or trip based on their needs and requirements. It can competently handle various tasks, from setting learning objectives to outlining assessment criteria. For businesses, ChatGPT is like having a personal planning assistant at your fingertips. Imagine trips with itineraries that unfold effortlessly, or fieldwork logistics handled with the ease of conversation."

The paper says while the AI model is adaptable and user-friendly, there are limitations when it comes to field course planning, including risk assessments.

Dr Ian Hendy, from the University of Portsmouth, explained: "We asked ChatGPT to identify the potential hazards of this course and assess the overall risk of this activity from low to high, and the results were mixed. In some instances, ChatGPT was able to identify hazards specific to the activity -- like the increased risk of slipping on seaweed-covered rocks exposed at low tide -- but in other instances, ChatGPT exaggerated threats. For example, we find the risk of students suffering from physical strain and fatigue from carrying bags of collected litter to be low. That's why there still needs to be a human element in the planning stages, to iron out any issues. It's also important that the individual sifting through the results understands the nuances of successful field courses so they can recognise these discrepancies."

The paper concludes with a series of recommendations for best practices in using ChatGPT for field course design, underscoring the need for thoughtful human input, logical prompt sequencing, critical evaluation, and adaptive management to refine course designs.

Top tips to help potential users get the most out of ChatGPT:

• Get the ball rolling with ChatGPT: Ask what details it thrives on for crafting the perfect assignment plan. By understanding the key information it needs, you'll be well-equipped to structure your prompts effectively and ensure ChatGPT provides tailored and insightful assistance;
• Time Management Made Easy: Share your preferred schedule, and let ChatGPT handle the logistics. Whether you're a back-to-back meetings person or prefer a more relaxed pace, ChatGPT creates an itinerary that suits your working style;
• Flexible Contingency Plans: Anticipate the unexpected. ChatGPT can help you create contingency plans in case of unforeseen events, ensuring that the trip remains adaptable to changing circumstances without compromising the educational goals;
• Cultural Etiquette Guidance: Familiarise yourself with local cultural norms and business etiquette. ChatGPT can provide tips on appropriate greetings, gift-giving customs, and other cultural considerations, ensuring smooth interactions with local business partners;
• Become a proficient Prompt Engineer: There are many quality, low-cost courses in the field of ChatGPT prompt engineering. These are available from online learning platforms such as Udemy, Coursera, and LinkedIn Learning. Poor input leads to poor ChatGPT output, so improving your prompt engineering will always lead to better results;
• Use your unique experiences to improve ChatGPT output: Remember that AI knowledge cannot replace personal experience, but AI can learn from your experiences and use them to improve its recommendations;
• Remember, planning is a two-way street! Engage in feedback with ChatGPT. Don't hesitate to tweak and refine the itinerary until it feels just right. It's your trip, after all.
• Civil Engineering
• Engineering and Construction
• Engineering
• Environmental Awareness
• Sustainability
• Educational Technology
• Computer Modeling
• Urban planning
• Shift of equinox
• Microeconomics
• Reindeer (Caribou)
• 1991 Halloween Nor'easter
• Antarctic Circle
• Seasonal affective disorder

Story Source:

Materials provided by University of Plymouth . Original written by Alan Williams. Note: Content may be edited for style and length.

Journal Reference :

• Mark Tupper, Ian W. Hendy, J. Reuben Shipway. Field courses for dummies: To what extent can ChatGPT design a higher education field course? Innovations in Education and Teaching International , 2024; 1 DOI: 10.1080/14703297.2024.2316716

Cite This Page :

Explore More

• The Milky Way's Earliest Building Blocks
• Say Hello to Biodegradable Microplastics
• Bacteria and Human Colorectal Cancers
• Harnessing Hydrogen at Life's Origin
• How Thick Is the Ice On Europa?
• 'Odor Sensor' Male/ Female Blood Pressure
• Ancient Star Formed in Another Galaxy
• New Insights Into Early Human Migration
• Metamaterial: An Endless Domino Effect
• Quantum Tornado and Black Holes

Trending Topics

Strange & offbeat.

An official website of the United States government

Here's how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

•   Facebook
•   Twitter
•   Linkedin
•   Digg
•   Reddit
•   Pinterest
•   Email

Latest Earthquakes |    Chat Share Social Media  

New Mexico Landscapes Field Station: Internship Program Active

• Publications

Each year, the New Mexico Landscapes Field Station hires interns through Stewards Individual Placement Program in partnership with the National Park Service. Interns participate in a variety of ecological research and long-term monitoring in Bandelier National Monument and surrounding landscapes in northern New Mexico. 

Primary intern responsibilities include fieldwork related to dendrochronology, vegetation change, forest restoration, and other landscape change research. Tasks include the collection, processing, and analysis of data, and the drafting of reports related to research and monitoring projects. Around half the time is spent in the field, with subsequent time spent on data entry, wood processing, quality control, data analysis and report writing in the office at Bandelier National Monument.  

Past Intern Projects 

Kara Fox: Sagebrush Ecosystem Dynamics  

Ella Kasten: Fire in the Jemez Mountains StoryMap  

Hope Nowak: Effects of long-term forest treatment on understory vegetation dynamics in Northern New Mexico (poster) 

New Mexico Tree-Ring Lab

The New Mexico Landscapes Field Station

New Mexico Landscapes Field Station: Wildlife Research

New Mexico Landscapes Field Station: Forest Ecosystem Research

New Mexico Landscapes Field Station: Fire Research

New Mexico Landscapes Field Station: People

Vegetation change over 140 years in a sagebrush landscape of the rio grande del norte national monument, new mexico, usa.

• Latest news
• UCL in the media
• Services for media
• Student news
• Tell us your story

Similar DNA changes found in cells of both smokers and e-cigarette users

19 March 2024

E-cigarette users with a limited smoking history experience similar DNA changes to specific cheek cells as smokers, finds a new study led by researchers at UCL and University of Innsbruck.

This study is an incremental step in helping researchers to build a deeper understanding of the long-term effects of e-cigarettes on health. Although it does not show that e-cigarettes cause cancer, studies with long-term follow up are important to assess whether e-cigarettes have harmful effects and, if so, what they are.

The study, published in Cancer Research , analysed the epigenetic effects of tobacco and e-cigarettes on DNA methylation in over 3,500 samples, to investigate the impact on cells that are directly exposed to tobacco (e.g. in the mouth) and those that are not directly exposed (e.g. in blood or cervical cells).

The epigenome refers to an extra layer of information that is superimposed on our genetic material – the DNA. While DNA can be compared to the ‘hardware’ of a computer, epigenetics are comparable to the computer’s ‘software’ and define how, where and when the programs used by the computer are run.

Epigenomes change throughout our lives and can be affected by a variety of genetic or nongenetic factors – including ageing, our lifestyles, exposure to hormones, chemicals and environmental factors, and even stress and psychological trauma. One commonly studied type of epigenetic modification is called DNA methylation.

The researchers found that epithelial cells (cells that typically line organs and are often the cells of origin for cancer) in the mouth showed substantial epigenomic changes in smokers. Importantly, these changes are further elevated in lung cancers or pre-cancers (abnormal cells or tissue that have the potential to develop into cancer), when compared to the normal lung tissue, supporting the idea that the epigenetic changes associated with smoking allow cells to grow more quickly.

The publication also includes new data showing the similar epigenomic changes were likewise observed in the cells of e-cigarette users who had only ever smoked less than 100 tobacco cigarettes in their lives.

First author, Dr Chiara Herzog (UCL EGA Institute for Women’s Health and University of Innsbruck), said: “This is the first study to investigate the impact of smoking and vaping on different kinds of cells – rather than just blood – and we’ve also strived to consider the longer-term health implications of using e-cigarettes.

“We cannot say that e-cigarettes cause cancer based on our study, but we do observe e-cigarette users exhibit some similar epigenetic changes in buccal cells as smokers, and these changes are associated with future lung cancer development in smokers. Further studies will be required to investigate whether these features could be used to individually predict cancer in smokers and e-cigarette users.

“While the scientific consensus is that e-cigarettes are safer than smoking tobacco, we cannot assume they are completely safe to use and it is important to explore their potential long-term risks and links to cancer.

“We hope this study may help form part of a wider discussion into e-cigarette usage – especially in people who have never previously smoked tobacco.”

Through their computational analysis of the samples, the researchers also found that some smoking-related epigenetic changes remain more stable than others after giving up smoking, including smoking-related epigenetic changes in cervical samples – something that has not previously been studied.

Senior author, Professor Martin Widschwendter (UCL EGA Institute for Women’s Health and University of Innsbruck), said: “The epigenome allows us, on one side, to look back. It tells us about how our body responded to a previous environmental exposure like smoking. Likewise exploring the epigenome may also enable us to predict future health and disease. Changes that are observed in lung cancer tissue can also be measured in cheek cells from smokers who have not (yet) developed a cancer.

"Importantly, our research points to the fact that e-cigarette users exhibit the same changes, and these devices might not be as harmless as originally thought. Long-term studies of e-cigarettes are needed. We are grateful for the support the European Commission has provided to obtain these data.”

Tobacco is well known as a modifiable contributor to adverse health outcomes, and it has been estimated to have caused 7.69 million deaths globally in 2019, with numbers expected to increase in the future. The NHS says e-cigarettes are substantially safer than smoking tobacco and smokers are recommended to switch to vaping to improve their health.

The researchers involved in the latest study now hope to further investigate how epigenetic changes related to smoking in cheek swabs could be used for identifying individuals at highest risk of developing cancer and assess the long-term health risks of e-cigarettes. 

This work was supported by funding from the European Union’s Horizon 2020 Research and Innovation programme, The Eve Appeal, and Cancer Research UK.

Dr Ian Walker, Cancer Research UK’s executive director of policy, said: “This study contributes to our understanding of e-cigarettes, but it does not show that e-cigarettes cause cancer. Decades of research has proven the link between smoking and cancer, and studies have so far shown that e-cigarettes are far less harmful than smoking and can help people quit. This paper does however highlight that e-cigarettes are not risk-free, and so we need additional studies to uncover their potential longer-term impacts on human health.

“Smoking tobacco causes 150 cases of cancer every single day in the UK, which is why we look forward to seeing the Government’s age of sale legislation being presented in parliament. Nothing would have a bigger impact on reducing the number of preventable deaths in the UK than ending smoking, and this policy will take us one step closer to a smokefree future.”

• The paper in Cancer Research
• Professor Martin Widschwendter’s academic profile
• Dr Chiara Herzog’s academic profile
• UCL EGA Institute for Women’s Health
• UCL Population Health Sciences
• Cancer Research UK
• The Eve Appeal
• University of Innsbruck
• Credit: iStock / licsiren

Media contact

Mark greaves.

m.greaves [at] ucl.ac.uk

+44 (0)20 3108 9485

An official website of the United States government

Here's how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Housing Needs of Survivors of Human Trafficking Study

Report Acceptance Date: February 2024 (91 pages)

Posted Date: March 21, 2024

In Section 606 of the Violence Against Women Act of 2022, Congress directed HUD to conduct a study assessing the availability and accessibility of housing and services for individuals experiencing homelessness or housing instability and who are survivors of trafficking or at risk of being trafficked. Congress set out a specific set of themes for the research team to examine:

• Approaches to outreach and engagement with survivors, and methods of assessing their needs;
• Availability of homelessness and housing services;
• Policies and procedures that impact access to mainstream housing and services;
• Barriers to fair housing; and
• Best practices in housing and service delivery.

This report reflects research conducted by staff from HUD's Office of Policy Development and Research with support from HUD's Director for Gender-based Violence Prevention and Equity in the Office of the Secretary. The study consisted of a literature review and listening sessions with a wide range of stakeholders including: partners in state and local government; Public Housing Agencies; Continuums of Care; direct service providers working in the housing, homelessness, victims services, and trafficking fields; and people with lived experience of sex and labor trafficking. In addition to extensive consultation with these external stakeholders, the study team actively coordinated with program offices across HUD and partners from the U.S. Advisory Council on Human Trafficking, Interagency Human Trafficking Housing Working Group, the Interagency Task Force to Monitor and Combat Trafficking, White House Gender Policy Council, and a range of other stakeholder offices at the Department of Justice and Department of Health and Human Services.

Facility for Rare Isotope Beams

At michigan state university, new long range plan for nuclear science recommends frib enhancements to forward the field.

The Facility for Rare Isotope Beams (FRIB) figures largely in the Nuclear Science Advisory Committee ’s (NSAC) newly released “ A New Era of Discovery: The 2023 Long Range Plan for Nuclear Science .”

The new plan, released on 4 October , provides a roadmap for advancing the nation's nuclear science research programs over the next decade. It is the 8 th long range plan published by NSAC since 1979.

NSAC is chartered under the Federal Advisory Committee Act and composed of 20 members who represent a range of subject matter expertise in nuclear physics. The major programs, accelerators, instruments, and experiments that enable nuclear physics research in the U.S. are primarily funded by the U.S. Department of Energy Office of Science (DOE-SC) and the National Science Foundation (NSF). To ensure that these federal investments reflect the national interest, the two agencies regularly solicit input from practicing nuclear physicists through NSAC.

The plan highlights the scientific opportunities of nuclear physics today to maintain world-leadership in the context of four different budget scenarios and details progress since the last long range plan. The document also features the impact of nuclear science on other fields and applications of the research that benefit society.

The plan’s first recommendation affirms that the nuclear science community’s highest priority is to capitalize on the extraordinary opportunities for scientific discovery made possible by the substantial and sustained investments of the United States government. FRIB and its user community  and the FRIB Theory Alliance membership are directly impacted by the associated requests for an increase in research funding, the continued effective operations of FRIB, a compensation for graduate researchers commensurate with the cost of living, and the provision of resources to ensure a respectful and safe environment for all.

FRIB400 —an energy upgrade to expand the already broad scientific reach of FRIB—is explicitly mentioned in the executive summary following Recommendation IV, which calls for investments in additional projects and new strategic opportunities that advance discovery science. Instruments aspired by the community for FRIB, such as the High Rigidity Spectrometer , the Gamma-Ray Energy Tracking Array , the FRIB Decay Station , and the Isochronous Spectrometer with Large Acceptance , feature in the science section of the long range plan.

FRIB hosted a long range plan update event on 6 October—one of 21 sites simultaneously hosting events that collectively represented FRIB’s 1,800 scientific users. FRIB enables scientific research with fast, stopped and reaccelerated rare isotope beams, supporting a user community of 1,800 scientists from around the world. 

“The 2023 Long Range Plan for Nuclear Science details a strategic path forward for nuclear science, and highlights FRIB’s important contributions toward advancing the field and ensuring the nation’s continued leadership,” said Thomas Glasmacher, FRIB Laboratory Director. “We thank the committee for their diligent work in preparing this forward-thinking plan that will drive our field and guide our focus for the next decade.”

Nuclear physicists lead the nation’s journey of discovery into the quarks and gluons that make up the protons and neutrons and atomic nuclei that build our visible universe. They investigate the nucleus of the atom at the heart of all matter, as well as the birth, life, and death of stars, and the mysterious and fleeting neutrino. Vital cutting-edge research and applications essential to national security, medicine, and the environment are the product of nuclear physicists working to expand our knowledge of the past, present, and future of our fascinating universe.

Developing the plan

On July 11, 2022, DOE and NSF charged NSAC with conducting the study of the opportunities and priorities for U.S. nuclear physics research and with crafting a new long range plan.

In the fall of 2022, town hall events were held to foster input for the new plan and to facilitate discussion among nuclear physicists. The town halls solicited and received numerous white papers for consideration, covering the subfields (1) quantum chromodynamics, (2) nuclear structure, reactions, and astrophysics, and (3) fundamental symmetries, neutrons, and neutrinos. Other groups also provided input related to crosscutting research and applications, such as quantum information science and technology for nuclear physics.

Dozens of nuclear physicists from across the nation developed the plan document with community-wide input. The development process included open forums, ongoing dialogue and grassroots input, which were embraced by the many working physicists in nuclear science. “I am grateful to the many physicists who engaged with the process to provide input,” said Dodge. “The committee that developed the plan worked incredibly hard in a collegial and respectful manner, culminating in today’s approval.”

In addition to providing a framework for the coordinated advancement of the nation's nuclear science research programs, the plan features detailed information about the field’s national and international research programs and partnerships, describes the initiatives to advance science through cross-discipline collaboration, and details how efforts to promote and sustain a diverse, equitable and inclusive nuclear science workforce are fully integrated into every aspect of the vision for the future of U.S. nuclear science.

“Every five to eight years, the federal agencies charge the Nuclear Science Advisory Committee to develop a plan to ensure the nation’s leadership in nuclear science, based on community input. Today NSAC approved the 2023 Long Range Plan for Nuclear Science, after over a year’s work and difficult choices,” said NSAC Chairperson Gail Dodge, a nuclear physicist and Dean of the College of Sciences at Old Dominion University, on 4 October upon the plan’s release. “The 2023 Long Range Plan lays out a compelling vision for nuclear science in the United States under multiple budget scenarios and informed by the international context. Implementation of the Long Range Plan’s recommendations will maintain the nation’s leadership and workforce in nuclear science,” she added. “What comes out of these discussions is a plan that will serve as input to DOE and NSF as they consider their research funding plans and priorities.”

For more information, visit  NuclearScienceFuture.org  for timely updates on the 2023 planning process.

• NSAC release: NSAC recommends future directions for nuclear science
• MSUToday:  New 'Long Range Plan for Nuclear Science' recommends FRIB enhancements

Michigan State University operates the Facility for Rare Isotope Beams (FRIB) as a user facility for the U.S. Department of Energy Office of Science (DOE-SC), supporting the mission of the DOE-SC Office of Nuclear Physics. Hosting what is designed to be the most powerful heavy-ion accelerator, FRIB enables scientists to make discoveries about the properties of rare isotopes in order to better understand the physics of nuclei, nuclear astrophysics, fundamental interactions, and applications for society, including in medicine, homeland security, and industry.

The U.S. Department of Energy Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of today’s most pressing challenges. For more information, visit  energy.gov/science .

Michigan State University has been advancing the common good with uncommon will for more than 165 years. One of the world's leading research universities, MSU pushes the boundaries of discovery to make a better, safer, healthier world for all while providing life-changing opportunities to a diverse and inclusive academic community through more than 400 programs of study in 17 degree-granting colleges.

Silicon Valley is pricing academics out of AI research

With eye-popping salaries and access to costly computing power, ai companies are draining academia of talent.

Fei-Fei Li, the “godmother of artificial intelligence,” delivered an urgent plea to President Biden in the glittering ballroom of San Francisco’s Fairmont Hotel in June.

The Stanford professor asked Biden to fund a national warehouse of computing power and data sets — part of a “moonshot investment” allowing the country’s top AI researchers to keep up with tech giants.

She elevated the ask Thursday at Biden’s State of the Union address, which Li attended as a guest of Rep. Anna G. Eshoo (D-Calif.) to promote a bill to fund a national AI repository.

Li is at the forefront of a growing chorus of academics, policymakers and former employees who argue that the sky-high cost of working with AI models is boxing researchers out of the field, compromising independent study of the burgeoning technology.

As such tech behemoths as Meta, Google and Microsoft funnel billions of dollars into AI, a massive resources gap is building with even the country’s richest universities. Meta aims to procure 350,000 of the specialized computer chips — called GPUs — that are essential to run the gargantuan calculations needed for AI models. In contrast, Stanford’s Natural Language Processing Group has 68 GPUs for all of its work.

After attending State of the Union speech #SOTU tonight, I had a brief exchange w/ President Biden @POTUS . Me: “Mr. President, you gave a historical speech by mentioning AI in the SOTU speech for the first time in history.” @POTUS (smiling): “Yes! And keep it safe”. 1/ pic.twitter.com/cJ7vs440fx — Fei-Fei Li (@drfeifei) March 8, 2024

To obtain the expensive computing power and data required to research AI systems, scholars frequently partner with tech employees. Meanwhile, tech firms’ eye-popping salaries are draining academia of star talent.

Big tech companies now dominate breakthroughs in the field. In 2022, the tech industry created 32 significant machine learning models, while academics produced three, a significant reversal from 2014, when the majority of AI breakthroughs originated in universities, according to a Stanford report .

Researchers say this lopsided power dynamic is shaping the field in subtle ways, pushing AI scholars to tailor their research for commercial use. Last month, Meta CEO Mark Zuckerberg announced that the company’s independent AI research lab would move closer to its product team, ensuring “some level of alignment” between the groups, he said.

“The public sector is now significantly lagging in resources and talent compared to that of industry,” said Li, a former Google employee and the co-director of the Stanford Institute for Human-Centered AI. “This will have profound consequences because industry is focused on developing technology that is profit-driven, whereas public-sector AI goals are focused on creating public goods.”

This agency is tasked with keeping AI safe. Its offices are crumbling.

Some are pushing for new sources of funding. Li has been making the rounds in Washington, huddling with White House Office of Science and Technology Policy Director Arati Prabhakar, dining with the political press at a swanky seafood and steak restaurant and visiting Capitol Hill for meetings with lawmakers working on AI, including Sens. Martin Heinrich (D-N.M.), Mike Rounds (R-S.D.) and Todd Young (R-Ind.).

Large tech companies have contributed computing resources to the National AI Research Resource, the national warehouse project, including a $20 million donation in computing credits from Microsoft.

“We have long embraced the importance of sharing knowledge and compute resources with our colleagues within academia,” Microsoft Chief Scientific Officer Eric Horvitz said in a statement.

Policymakers are taking some steps to address the funding gaps. Last year, the National Science Foundation announced a $140 million investment to launch seven university-led National AI Research Institutes to examine how AI could mitigate the effects of climate change and improve education, among other topics.

Eshoo said she hopes to pass the Create AI Act, which has bipartisan backing in the House and the Senate, by the end of the year, when she is scheduled to retire. The legislation “essentially democratizes AI,” Eshoo said.

But scholars say this infusion may not come quickly enough.

As Silicon Valley races to build chatbots and image generators, it is drawing would-be computer science professors with high salaries and the chance to work on interesting AI problems. Nearly 70 percent of people with PhDs in AI end up in private industry compared with 21 percent of graduates two decades ago, according to a 2023 report .

Amid explosive demand, America is running out of power

Big Tech’s AI boom has pushed the salaries for the best researchers to new heights. Median compensation packages for AI research scientists at Meta climbed from $256,000 in 2020 to $335,250 in 2023, according to Levels.fyi , a salary-tracking website. True stars can attract even more cash: AI engineers with a PhD and several years of experience building AI models can command compensation as high as $20 million over four years, said Ali Ghodsi, who as CEO of AI start-up Databricks is regularly competing to hire AI talent.

“The compensation is through the roof. It’s ridiculous,” he said. “It’s not an uncommon number to hear, roughly.”

University academics often have little choice but to work with industry researchers , with the companies footing the bill for computing power and offering data. Nearly 40 percent of papers presented at leading AI conferences in 2020 had at least one tech employee author, according to the 2023 report . And industry grants often fund PhD students to perform research, said Mohamed Abdalla, a scientist at the Canada-based Institute for Better Health at Trillium Health Partners and incoming assistant professor at the University of Alberta, who has conducted research on the effect of industry on academics’ AI research.

“It was like a running joke that, like, everyone is getting hired by them,” Abdalla said. “And the people that were remaining, they were funded by them — so, in a way, hired by them.”

Google believes private companies and universities should work together to develop the science behind AI, said Jane Park, a spokesperson for the company. Google still routinely publishes its research publicly to benefit the broader AI community, Park said.

David Harris, a former research manager for Meta’s responsible AI team, said corporate labs may not censor the outcome of research but may influence which projects get tackled.

“Anytime you see a mix of authors who are employed by a company and authors who work at a university, you should really scrutinize the motives of the company for contributing to that work,” said Harris, who is now a chancellor’s public scholar at the University of California at Berkeley. “We used to look at people employed in academia to be neutral scholars, motivated only by the pursuit of truth and the interest of society.”

These fake images reveal how AI amplifies our worst stereotypes

Tech giants procure huge amounts of computing power through data centers and have access to GPUs. These resources are expensive: A recent report from Stanford University researchers estimated that Google DeepMind’s large language model, Chinchilla, cost $2.1 million to develop. More than 100 top artificial intelligence researchers on Tuesday urged generative AI companies to offer a legal and technical safe harbor to researchers so they can scrutinize their products without the fear that internet platforms will suspend their accounts or threaten legal action.

The necessity for advanced computing power is likely to only grow as AI scientists crunch more data to improve the performance of their models, said Neil Thompson, director of the FutureTech research project at MIT’s Computer Science and Artificial Intelligence Laboratory, which studies progress in computing.

“To keep getting better, [what] you expect to need is more and more money, more and more computers, more and more data,” Thompson said. “What that’s going to mean is that people who do not have as much compute [and] who do not have as many resources are going to stop being able to participate.”

Tech companies, including Meta and Google, have historically run their AI research labs to resemble universities where scientists decide what projects to pursue to advance the state of research, according to people familiar with the subject who spoke on the condition of anonymity to discuss private company matters.

Those workers were largely isolated from teams focused on building products or generating revenue, the people said. They were judged on influential papers they published or notable breakthroughs — similar to metrics used for their university peers, the people said. Top AI Meta scientists Yann LeCun and Joelle Pineau hold dual appointments at New York University and McGill University, blurring the lines between industry and academia.

Top AI researchers say OpenAI, Meta and more hinder independent evaluations

In an increasingly competitive market for generative AI products , research freedom inside companies could wane. In April, Google announced it was merging two of its AI research groups — DeepMind, which it acquired in 2014, and the Brain team from Google Research — into one department called Google DeepMind. Last year, Google started to take more advantage of its own AI discoveries, sharing research papers only after the lab work had been turned into products, The Washington Post has reported .

Meta has also reshuffled its research teams. The company placed its Fundamental AI Research team, known as FAIR, under the helm of its virtual-reality division, Reality Labs, in 2022 and last year reassigned some of the group’s researchers to a new generative AI product team. Last month, Zuckerberg told investors that FAIR would work “closer together” with the generative AI product team, arguing that while the two groups would still conduct research on “different time horizons,” it was helpful to the company “to have some level of alignment” between them.

“In a lot of tech companies right now, they hired research scientists that knew something about AI and maybe set certain expectations about how much freedom they would have to set their own schedule and set their own research agenda,” Harris said. “That’s changing, especially for the companies that are moving frantically right now to ship these products.”

A previous version of this article incorrectly said that Google acquired DeepMind in 2010. Google acquired the AI start-up in 2014. The article has been corrected.