research papers on learning methods

• Open supplemental data
• Reference Manager
• Simple TEXT file

People also looked at

Original research article, insights into students’ experiences and perceptions of remote learning methods: from the covid-19 pandemic to best practice for the future.

• 1 Minerva Schools at Keck Graduate Institute, San Francisco, CA, United States
• 2 Ronin Institute for Independent Scholarship, Montclair, NJ, United States
• 3 Department of Physics, University of Toronto, Toronto, ON, Canada

This spring, students across the globe transitioned from in-person classes to remote learning as a result of the COVID-19 pandemic. This unprecedented change to undergraduate education saw institutions adopting multiple online teaching modalities and instructional platforms. We sought to understand students’ experiences with and perspectives on those methods of remote instruction in order to inform pedagogical decisions during the current pandemic and in future development of online courses and virtual learning experiences. Our survey gathered quantitative and qualitative data regarding students’ experiences with synchronous and asynchronous methods of remote learning and specific pedagogical techniques associated with each. A total of 4,789 undergraduate participants representing institutions across 95 countries were recruited via Instagram. We find that most students prefer synchronous online classes, and students whose primary mode of remote instruction has been synchronous report being more engaged and motivated. Our qualitative data show that students miss the social aspects of learning on campus, and it is possible that synchronous learning helps to mitigate some feelings of isolation. Students whose synchronous classes include active-learning techniques (which are inherently more social) report significantly higher levels of engagement, motivation, enjoyment, and satisfaction with instruction. Respondents’ recommendations for changes emphasize increased engagement, interaction, and student participation. We conclude that active-learning methods, which are known to increase motivation, engagement, and learning in traditional classrooms, also have a positive impact in the remote-learning environment. Integrating these elements into online courses will improve the student experience.

Introduction

The COVID-19 pandemic has dramatically changed the demographics of online students. Previously, almost all students engaged in online learning elected the online format, starting with individual online courses in the mid-1990s through today’s robust online degree and certificate programs. These students prioritize convenience, flexibility and ability to work while studying and are older than traditional college age students ( Harris and Martin, 2012 ; Levitz, 2016 ). These students also find asynchronous elements of a course are more useful than synchronous elements ( Gillingham and Molinari, 2012 ). In contrast, students who chose to take courses in-person prioritize face-to-face instruction and connection with others and skew considerably younger ( Harris and Martin, 2012 ). This leaves open the question of whether students who prefer to learn in-person but are forced to learn remotely will prefer synchronous or asynchronous methods. One study of student preferences following a switch to remote learning during the COVID-19 pandemic indicates that students enjoy synchronous over asynchronous course elements and find them more effective ( Gillis and Krull, 2020 ). Now that millions of traditional in-person courses have transitioned online, our survey expands the data on student preferences and explores if those preferences align with pedagogical best practices.

An extensive body of research has explored what instructional methods improve student learning outcomes (Fink. 2013). Considerable evidence indicates that active-learning or student-centered approaches result in better learning outcomes than passive-learning or instructor-centered approaches, both in-person and online ( Freeman et al., 2014 ; Chen et al., 2018 ; Davis et al., 2018 ). Active-learning approaches include student activities or discussion in class, whereas passive-learning approaches emphasize extensive exposition by the instructor ( Freeman et al., 2014 ). Constructivist learning theories argue that students must be active participants in creating their own learning, and that listening to expert explanations is seldom sufficient to trigger the neurological changes necessary for learning ( Bostock, 1998 ; Zull, 2002 ). Some studies conclude that, while students learn more via active learning, they may report greater perceptions of their learning and greater enjoyment when passive approaches are used ( Deslauriers et al., 2019 ). We examine student perceptions of remote learning experiences in light of these previous findings.

In this study, we administered a survey focused on student perceptions of remote learning in late May 2020 through the social media account of @unjadedjade to a global population of English speaking undergraduate students representing institutions across 95 countries. We aim to explore how students were being taught, the relationship between pedagogical methods and student perceptions of their experience, and the reasons behind those perceptions. Here we present an initial analysis of the results and share our data set for further inquiry. We find that positive student perceptions correlate with synchronous courses that employ a variety of interactive pedagogical techniques, and that students overwhelmingly suggest behavioral and pedagogical changes that increase social engagement and interaction. We argue that these results support the importance of active learning in an online environment.

Materials and Methods

Participant pool.

Students were recruited through the Instagram account @unjadedjade. This social media platform, run by influencer Jade Bowler, focuses on education, effective study tips, ethical lifestyle, and promotes a positive mindset. For this reason, the audience is presumably academically inclined, and interested in self-improvement. The survey was posted to her account and received 10,563 responses within the first 36 h. Here we analyze the 4,789 of those responses that came from undergraduates. While we did not collect demographic or identifying information, we suspect that women are overrepresented in these data as followers of @unjadedjade are 80% women. A large minority of respondents were from the United Kingdom as Jade Bowler is a British influencer. Specifically, 43.3% of participants attend United Kingdom institutions, followed by 6.7% attending university in the Netherlands, 6.1% in Germany, 5.8% in the United States and 4.2% in Australia. Ninety additional countries are represented in these data (see Supplementary Figure 1 ).

Survey Design

The purpose of this survey is to learn about students’ instructional experiences following the transition to remote learning in the spring of 2020.

This survey was initially created for a student assignment for the undergraduate course Empirical Analysis at Minerva Schools at KGI. That version served as a robust pre-test and allowed for identification of the primary online platforms used, and the four primary modes of learning: synchronous (live) classes, recorded lectures and videos, uploaded or emailed materials, and chat-based communication. We did not adapt any open-ended questions based on the pre-test survey to avoid biasing the results and only corrected language in questions for clarity. We used these data along with an analysis of common practices in online learning to revise the survey. Our revised survey asked students to identify the synchronous and asynchronous pedagogical methods and platforms that they were using for remote learning. Pedagogical methods were drawn from literature assessing active and passive teaching strategies in North American institutions ( Fink, 2013 ; Chen et al., 2018 ; Davis et al., 2018 ). Open-ended questions asked students to describe why they preferred certain modes of learning and how they could improve their learning experience. Students also reported on their affective response to learning and participation using a Likert scale.

The revised survey also asked whether students had responded to the earlier survey. No significant differences were found between responses of those answering for the first and second times (data not shown). See Supplementary Appendix 1 for survey questions. Survey data was collected from 5/21/20 to 5/23/20.

Qualitative Coding

We applied a qualitative coding framework adapted from Gale et al. (2013) to analyze student responses to open-ended questions. Four researchers read several hundred responses and noted themes that surfaced. We then developed a list of themes inductively from the survey data and deductively from the literature on pedagogical practice ( Garrison et al., 1999 ; Zull, 2002 ; Fink, 2013 ; Freeman et al., 2014 ). The initial codebook was revised collaboratively based on feedback from researchers after coding 20–80 qualitative comments each. Before coding their assigned questions, alignment was examined through coding of 20 additional responses. Researchers aligned in identifying the same major themes. Discrepancies in terms identified were resolved through discussion. Researchers continued to meet weekly to discuss progress and alignment. The majority of responses were coded by a single researcher using the final codebook ( Supplementary Table 1 ). All responses to questions 3 (4,318 responses) and 8 (4,704 responses), and 2,512 of 4,776 responses to question 12 were analyzed. Valence was also indicated where necessary (i.e., positive or negative discussion of terms). This paper focuses on the most prevalent themes from our initial analysis of the qualitative responses. The corresponding author reviewed codes to ensure consistency and accuracy of reported data.

Statistical Analysis

The survey included two sets of Likert-scale questions, one consisting of a set of six statements about students’ perceptions of their experiences following the transition to remote learning ( Table 1 ). For each statement, students indicated their level of agreement with the statement on a five-point scale ranging from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”). The second set asked the students to respond to the same set of statements, but about their retroactive perceptions of their experiences with in-person instruction before the transition to remote learning. This set was not the subject of our analysis but is present in the published survey results. To explore correlations among student responses, we used CrossCat analysis to calculate the probability of dependence between Likert-scale responses ( Mansinghka et al., 2016 ).

Table 1. Likert-scale questions.

Mean values are calculated based on the numerical scores associated with each response. Measures of statistical significance for comparisons between different subgroups of respondents were calculated using a two-sided Mann-Whitney U -test, and p -values reported here are based on this test statistic. We report effect sizes in pairwise comparisons using the common-language effect size, f , which is the probability that the response from a random sample from subgroup 1 is greater than the response from a random sample from subgroup 2. We also examined the effects of different modes of remote learning and technological platforms using ordinal logistic regression. With the exception of the mean values, all of these analyses treat Likert-scale responses as ordinal-scale, rather than interval-scale data.

Students Prefer Synchronous Class Sessions

Students were asked to identify their primary mode of learning given four categories of remote course design that emerged from the pilot survey and across literature on online teaching: live (synchronous) classes, recorded lectures and videos, emailed or uploaded materials, and chats and discussion forums. While 42.7% ( n = 2,045) students identified live classes as their primary mode of learning, 54.6% ( n = 2613) students preferred this mode ( Figure 1 ). Both recorded lectures and live classes were preferred over uploaded materials (6.22%, n = 298) and chat (3.36%, n = 161).

Figure 1. Actual (A) and preferred (B) primary modes of learning.

In addition to a preference for live classes, students whose primary mode was synchronous were more likely to enjoy the class, feel motivated and engaged, be satisfied with instruction and report higher levels of participation ( Table 2 and Supplementary Figure 2 ). Regardless of primary mode, over two-thirds of students reported they are often distracted during remote courses.

Table 2. The effect of synchronous vs. asynchronous primary modes of learning on student perceptions.

Variation in Pedagogical Techniques for Synchronous Classes Results in More Positive Perceptions of the Student Learning Experience

To survey the use of passive vs. active instructional methods, students reported the pedagogical techniques used in their live classes. Among the synchronous methods, we identify three different categories ( National Research Council, 2000 ; Freeman et al., 2014 ). Passive methods (P) include lectures, presentations, and explanation using diagrams, white boards and/or other media. These methods all rely on instructor delivery rather than student participation. Our next category represents active learning through primarily one-on-one interactions (A). The methods in this group are in-class assessment, question-and-answer (Q&A), and classroom chat. Group interactions (F) included classroom discussions and small-group activities. Given these categories, Mann-Whitney U pairwise comparisons between the 7 possible combinations and Likert scale responses about student experience showed that the use of a variety of methods resulted in higher ratings of experience vs. the use of a single method whether or not that single method was active or passive ( Table 3 ). Indeed, students whose classes used methods from each category (PAF) had higher ratings of enjoyment, motivation, and satisfaction with instruction than those who only chose any single method ( p < 0.0001) and also rated higher rates of participation and engagement compared to students whose only method was passive (P) or active through one-on-one interactions (A) ( p < 0.00001). Student ratings of distraction were not significantly different for any comparison. Given that sets of Likert responses often appeared significant together in these comparisons, we ran a CrossCat analysis to look at the probability of dependence across Likert responses. Responses have a high probability of dependence on each other, limiting what we can claim about any discrete response ( Supplementary Figure 3 ).

Table 3. Comparison of combinations of synchronous methods on student perceptions. Effect size (f).

Mann-Whitney U pairwise comparisons were also used to check if improvement in student experience was associated with the number of methods used vs. the variety of types of methods. For every comparison, we found that more methods resulted in higher scores on all Likert measures except distraction ( Table 4 ). Even comparison between four or fewer methods and greater than four methods resulted in a 59% chance that the latter enjoyed the courses more ( p < 0.00001) and 60% chance that they felt more motivated to learn ( p < 0.00001). Students who selected more than four methods ( n = 417) were also 65.1% ( p < 0.00001), 62.9% ( p < 0.00001) and 64.3% ( p < 0.00001) more satisfied with instruction, engaged, and actively participating, respectfully. Therefore, there was an overlap between how the number and variety of methods influenced students’ experiences. Since the number of techniques per category is 2–3, we cannot fully disentangle the effect of number vs. variety. Pairwise comparisons to look at subsets of data with 2–3 methods from a single group vs. 2–3 methods across groups controlled for this but had low sample numbers in most groups and resulted in no significant findings (data not shown). Therefore, from the data we have in our survey, there seems to be an interdependence between number and variety of methods on students’ learning experiences.

Table 4. Comparison of the number of synchronous methods on student perceptions. Effect size (f).

Variation in Asynchronous Pedagogical Techniques Results in More Positive Perceptions of the Student Learning Experience

Along with synchronous pedagogical methods, students reported the asynchronous methods that were used for their classes. We divided these methods into three main categories and conducted pairwise comparisons. Learning methods include video lectures, video content, and posted study materials. Interacting methods include discussion/chat forums, live office hours, and email Q&A with professors. Testing methods include assignments and exams. Our results again show the importance of variety in students’ perceptions ( Table 5 ). For example, compared to providing learning materials only, providing learning materials, interaction, and testing improved enjoyment ( f = 0.546, p < 0.001), motivation ( f = 0.553, p < 0.0001), satisfaction with instruction ( f = 0.596, p < 0.00001), engagement ( f = 0.572, p < 0.00001) and active participation ( f = 0.563, p < 0.00001) (row 6). Similarly, compared to just being interactive with conversations, the combination of all three methods improved five out of six indicators, except for distraction in class (row 11).

Table 5. Comparison of combinations of asynchronous methods on student perceptions. Effect size (f).

Ordinal logistic regression was used to assess the likelihood that the platforms students used predicted student perceptions ( Supplementary Table 2 ). Platform choices were based on the answers to open-ended questions in the pre-test survey. The synchronous and asynchronous methods used were consistently more predictive of Likert responses than the specific platforms. Likewise, distraction continued to be our outlier with no differences across methods or platforms.

Students Prefer In-Person and Synchronous Online Learning Largely Due to Social-Emotional Reasoning

As expected, 86.1% (4,123) of survey participants report a preference for in-person courses, while 13.9% (666) prefer online courses. When asked to explain the reasons for their preference, students who prefer in-person courses most often mention the importance of social interaction (693 mentions), engagement (639 mentions), and motivation (440 mentions). These students are also more likely to mention a preference for a fixed schedule (185 mentions) vs. a flexible schedule (2 mentions).

In addition to identifying social reasons for their preference for in-person learning, students’ suggestions for improvements in online learning focus primarily on increasing interaction and engagement, with 845 mentions of live classes, 685 mentions of interaction, 126 calls for increased participation and calls for changes related to these topics such as, “Smaller teaching groups for live sessions so that everyone is encouraged to talk as some people don’t say anything and don’t participate in group work,” and “Make it less of the professor reading the pdf that was given to us and more interaction.”

Students who prefer online learning primarily identify independence and flexibility (214 mentions) and reasons related to anxiety and discomfort in in-person settings (41 mentions). Anxiety was only mentioned 12 times in the much larger group that prefers in-person learning.

The preference for synchronous vs. asynchronous modes of learning follows similar trends ( Table 6 ). Students who prefer live classes mention engagement and interaction most often while those who prefer recorded lectures mention flexibility.

Table 6. Most prevalent themes for students based on their preferred mode of remote learning.

Student Perceptions Align With Research on Active Learning

The first, and most robust, conclusion is that incorporation of active-learning methods correlates with more positive student perceptions of affect and engagement. We can see this clearly in the substantial differences on a number of measures, where students whose classes used only passive-learning techniques reported lower levels of engagement, satisfaction, participation, and motivation when compared with students whose classes incorporated at least some active-learning elements. This result is consistent with prior research on the value of active learning ( Freeman et al., 2014 ).

Though research shows that student learning improves in active learning classes, on campus, student perceptions of their learning, enjoyment, and satisfaction with instruction are often lower in active-learning courses ( Deslauriers et al., 2019 ). Our finding that students rate enjoyment and satisfaction with instruction higher for active learning online suggests that the preference for passive lectures on campus relies on elements outside of the lecture itself. That might include the lecture hall environment, the social physical presence of peers, or normalization of passive lectures as the expected mode for on-campus classes. This implies that there may be more buy-in for active learning online vs. in-person.

A second result from our survey is that student perceptions of affect and engagement are associated with students experiencing a greater diversity of learning modalities. We see this in two different results. First, in addition to the fact that classes that include active learning outperform classes that rely solely on passive methods, we find that on all measures besides distraction, the highest student ratings are associated with a combination of active and passive methods. Second, we find that these higher scores are associated with classes that make use of a larger number of different methods.

This second result suggests that students benefit from classes that make use of multiple different techniques, possibly invoking a combination of passive and active methods. However, it is unclear from our data whether this effect is associated specifically with combining active and passive methods, or if it is associated simply with the use of multiple different methods, irrespective of whether those methods are active, passive, or some combination. The problem is that the number of methods used is confounded with the diversity of methods (e.g., it is impossible for a classroom using only one method to use both active and passive methods). In an attempt to address this question, we looked separately at the effect of number and diversity of methods while holding the other constant. Across a large number of such comparisons, we found few statistically significant differences, which may be a consequence of the fact that each comparison focused on a small subset of the data.

Thus, our data suggests that using a greater diversity of learning methods in the classroom may lead to better student outcomes. This is supported by research on student attention span which suggests varying delivery after 10–15 min to retain student’s attention ( Bradbury, 2016 ). It is likely that this is more relevant for online learning where students report high levels of distraction across methods, modalities, and platforms. Given that number and variety are key, and there are few passive learning methods, we can assume that some combination of methods that includes active learning improves student experience. However, it is not clear whether we should predict that this benefit would come simply from increasing the number of different methods used, or if there are benefits specific to combining particular methods. Disentangling these effects would be an interesting avenue for future research.

Students Value Social Presence in Remote Learning

Student responses across our open-ended survey questions show a striking difference in reasons for their preferences compared with traditional online learners who prefer flexibility ( Harris and Martin, 2012 ; Levitz, 2016 ). Students reasons for preferring in-person classes and synchronous remote classes emphasize the desire for social interaction and echo the research on the importance of social presence for learning in online courses.

Short et al. (1976) outlined Social Presence Theory in depicting students’ perceptions of each other as real in different means of telecommunications. These ideas translate directly to questions surrounding online education and pedagogy in regards to educational design in networked learning where connection across learners and instructors improves learning outcomes especially with “Human-Human interaction” ( Goodyear, 2002 , 2005 ; Tu, 2002 ). These ideas play heavily into asynchronous vs. synchronous learning, where Tu reports students having positive responses to both synchronous “real-time discussion in pleasantness, responsiveness and comfort with familiar topics” and real-time discussions edging out asynchronous computer-mediated communications in immediate replies and responsiveness. Tu’s research indicates that students perceive more interaction with synchronous mediums such as discussions because of immediacy which enhances social presence and support the use of active learning techniques ( Gunawardena, 1995 ; Tu, 2002 ). Thus, verbal immediacy and communities with face-to-face interactions, such as those in synchronous learning classrooms, lessen the psychological distance of communicators online and can simultaneously improve instructional satisfaction and reported learning ( Gunawardena and Zittle, 1997 ; Richardson and Swan, 2019 ; Shea et al., 2019 ). While synchronous learning may not be ideal for traditional online students and a subset of our participants, this research suggests that non-traditional online learners are more likely to appreciate the value of social presence.

Social presence also connects to the importance of social connections in learning. Too often, current systems of education emphasize course content in narrow ways that fail to embrace the full humanity of students and instructors ( Gay, 2000 ). With the COVID-19 pandemic leading to further social isolation for many students, the importance of social presence in courses, including live interactions that build social connections with classmates and with instructors, may be increased.

Limitations of These Data

Our undergraduate data consisted of 4,789 responses from 95 different countries, an unprecedented global scale for research on online learning. However, since respondents were followers of @unjadedjade who focuses on learning and wellness, these respondents may not represent the average student. Biases in survey responses are often limited by their recruitment techniques and our bias likely resulted in more robust and thoughtful responses to free-response questions and may have influenced the preference for synchronous classes. It is unlikely that it changed students reporting on remote learning pedagogical methods since those are out of student control.

Though we surveyed a global population, our design was rooted in literature assessing pedagogy in North American institutions. Therefore, our survey may not represent a global array of teaching practices.

This survey was sent out during the initial phase of emergency remote learning for most countries. This has two important implications. First, perceptions of remote learning may be clouded by complications of the pandemic which has increased social, mental, and financial stresses globally. Future research could disaggregate the impact of the pandemic from students’ learning experiences with a more detailed and holistic analysis of the impact of the pandemic on students.

Second, instructors, students and institutions were not able to fully prepare for effective remote education in terms of infrastructure, mentality, curriculum building, and pedagogy. Therefore, student experiences reflect this emergency transition. Single-modality courses may correlate with instructors who lacked the resources or time to learn or integrate more than one modality. Regardless, the main insights of this research align well with the science of teaching and learning and can be used to inform both education during future emergencies and course development for online programs that wish to attract traditional college students.

Global Student Voices Improve Our Understanding of the Experience of Emergency Remote Learning

Our survey shows that global student perspectives on remote learning agree with pedagogical best practices, breaking with the often-found negative reactions of students to these practices in traditional classrooms ( Shekhar et al., 2020 ). Our analysis of open-ended questions and preferences show that a majority of students prefer pedagogical approaches that promote both active learning and social interaction. These results can serve as a guide to instructors as they design online classes, especially for students whose first choice may be in-person learning. Indeed, with the near ubiquitous adoption of remote learning during the COVID-19 pandemic, remote learning may be the default for colleges during temporary emergencies. This has already been used at the K-12 level as snow days become virtual learning days ( Aspergren, 2020 ).

In addition to informing pedagogical decisions, the results of this survey can be used to inform future research. Although we survey a global population, our recruitment method selected for students who are English speakers, likely majority female, and have an interest in self-improvement. Repeating this study with a more diverse and representative sample of university students could improve the generalizability of our findings. While the use of a variety of pedagogical methods is better than a single method, more research is needed to determine what the optimal combinations and implementations are for courses in different disciplines. Though we identified social presence as the major trend in student responses, the over 12,000 open-ended responses from students could be analyzed in greater detail to gain a more nuanced understanding of student preferences and suggestions for improvement. Likewise, outliers could shed light on the diversity of student perspectives that we may encounter in our own classrooms. Beyond this, our findings can inform research that collects demographic data and/or measures learning outcomes to understand the impact of remote learning on different populations.

Importantly, this paper focuses on a subset of responses from the full data set which includes 10,563 students from secondary school, undergraduate, graduate, or professional school and additional questions about in-person learning. Our full data set is available here for anyone to download for continued exploration: https://dataverse.harvard.edu/dataset.xhtml?persistentId= doi: 10.7910/DVN/2TGOPH .

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

GS: project lead, survey design, qualitative coding, writing, review, and editing. TN: data analysis, writing, review, and editing. CN and PB: qualitative coding. JW: data analysis, writing, and editing. CS: writing, review, and editing. EV and KL: original survey design and qualitative coding. PP: data analysis. JB: original survey design and survey distribution. HH: data analysis. MP: writing. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We want to thank Minerva Schools at KGI for providing funding for summer undergraduate research internships. We also want to thank Josh Fost and Christopher V. H.-H. Chen for discussion that helped shape this project.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2021.647986/full#supplementary-material

Aspergren, E. (2020). Snow Days Canceled Because of COVID-19 Online School? Not in These School Districts.sec. Education. USA Today. Available online at: https://www.usatoday.com/story/news/education/2020/12/15/covid-school-canceled-snow-day-online-learning/3905780001/ (accessed December 15, 2020).

Google Scholar

Bostock, S. J. (1998). Constructivism in mass higher education: a case study. Br. J. Educ. Technol. 29, 225–240. doi: 10.1111/1467-8535.00066

CrossRef Full Text | Google Scholar

Bradbury, N. A. (2016). Attention span during lectures: 8 seconds, 10 minutes, or more? Adv. Physiol. Educ. 40, 509–513. doi: 10.1152/advan.00109.2016

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, B., Bastedo, K., and Howard, W. (2018). Exploring best practices for online STEM courses: active learning, interaction & assessment design. Online Learn. 22, 59–75. doi: 10.24059/olj.v22i2.1369

Davis, D., Chen, G., Hauff, C., and Houben, G.-J. (2018). Activating learning at scale: a review of innovations in online learning strategies. Comput. Educ. 125, 327–344. doi: 10.1016/j.compedu.2018.05.019

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., and Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc. Natl. Acad. Sci. 116, 19251–19257. doi: 10.1073/pnas.1821936116

Fink, L. D. (2013). Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. Somerset, NJ: John Wiley & Sons, Incorporated.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., et al. (2014). Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. 111, 8410–8415. doi: 10.1073/pnas.1319030111

Gale, N. K., Heath, G., Cameron, E., Rashid, S., and Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13:117. doi: 10.1186/1471-2288-13-117

Garrison, D. R., Anderson, T., and Archer, W. (1999). Critical inquiry in a text-based environment: computer conferencing in higher education. Internet High. Educ. 2, 87–105. doi: 10.1016/S1096-7516(00)00016-6

Gay, G. (2000). Culturally Responsive Teaching: Theory, Research, and Practice. Multicultural Education Series. New York, NY: Teachers College Press.

Gillingham, and Molinari, C. (2012). Online courses: student preferences survey. Internet Learn. 1, 36–45. doi: 10.18278/il.1.1.4

Gillis, A., and Krull, L. M. (2020). COVID-19 remote learning transition in spring 2020: class structures, student perceptions, and inequality in college courses. Teach. Sociol. 48, 283–299. doi: 10.1177/0092055X20954263

Goodyear, P. (2002). “Psychological foundations for networked learning,” in Networked Learning: Perspectives and Issues. Computer Supported Cooperative Work , eds C. Steeples and C. Jones (London: Springer), 49–75. doi: 10.1007/978-1-4471-0181-9_4

Goodyear, P. (2005). Educational design and networked learning: patterns, pattern languages and design practice. Australas. J. Educ. Technol. 21, 82–101. doi: 10.14742/ajet.1344

Gunawardena, C. N. (1995). Social presence theory and implications for interaction and collaborative learning in computer conferences. Int. J. Educ. Telecommun. 1, 147–166.

Gunawardena, C. N., and Zittle, F. J. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. Am. J. Distance Educ. 11, 8–26. doi: 10.1080/08923649709526970

Harris, H. S., and Martin, E. (2012). Student motivations for choosing online classes. Int. J. Scholarsh. Teach. Learn. 6, 1–8. doi: 10.20429/ijsotl.2012.060211

Levitz, R. N. (2016). 2015-16 National Online Learners Satisfaction and Priorities Report. Cedar Rapids: Ruffalo Noel Levitz, 12.

Mansinghka, V., Shafto, P., Jonas, E., Petschulat, C., Gasner, M., and Tenenbaum, J. B. (2016). CrossCat: a fully Bayesian nonparametric method for analyzing heterogeneous, high dimensional data. J. Mach. Learn. Res. 17, 1–49. doi: 10.1007/978-0-387-69765-9_7

National Research Council (2000). How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Washington, DC: National Academies Press, doi: 10.17226/9853

Richardson, J. C., and Swan, K. (2019). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Online Learn. 7, 68–88. doi: 10.24059/olj.v7i1.1864

Shea, P., Pickett, A. M., and Pelz, W. E. (2019). A Follow-up investigation of ‘teaching presence’ in the suny learning network. Online Learn. 7, 73–75. doi: 10.24059/olj.v7i2.1856

Shekhar, P., Borrego, M., DeMonbrun, M., Finelli, C., Crockett, C., and Nguyen, K. (2020). Negative student response to active learning in STEM classrooms: a systematic review of underlying reasons. J. Coll. Sci. Teach. 49, 45–54.

Short, J., Williams, E., and Christie, B. (1976). The Social Psychology of Telecommunications. London: John Wiley & Sons.

Tu, C.-H. (2002). The measurement of social presence in an online learning environment. Int. J. E Learn. 1, 34–45. doi: 10.17471/2499-4324/421

Zull, J. E. (2002). The Art of Changing the Brain: Enriching Teaching by Exploring the Biology of Learning , 1st Edn. Sterling, VA: Stylus Publishing.

Keywords : online learning, COVID-19, active learning, higher education, pedagogy, survey, international

Citation: Nguyen T, Netto CLM, Wilkins JF, Bröker P, Vargas EE, Sealfon CD, Puthipiroj P, Li KS, Bowler JE, Hinson HR, Pujar M and Stein GM (2021) Insights Into Students’ Experiences and Perceptions of Remote Learning Methods: From the COVID-19 Pandemic to Best Practice for the Future. Front. Educ. 6:647986. doi: 10.3389/feduc.2021.647986

Received: 30 December 2020; Accepted: 09 March 2021; Published: 09 April 2021.

Reviewed by:

Copyright © 2021 Nguyen, Netto, Wilkins, Bröker, Vargas, Sealfon, Puthipiroj, Li, Bowler, Hinson, Pujar and Stein. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Geneva M. Stein, [email protected]

This article is part of the Research Topic

Covid-19 and Beyond: From (Forced) Remote Teaching and Learning to ‘The New Normal’ in Higher Education

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.73(1); 2009 Feb 19

Learning Styles: A Review of Theory, Application, and Best Practices

Much pedagogical research has focused on the concept of “learning styles.” Several authors have proposed that the ability to typify student learning styles can augment the educational experience. As such, instructors might tailor their teaching style so that it is more congruent with a given student's or class of students' learning style. Others have argued that a learning/teaching style mismatch encourages and challenges students to expand their academic capabilities. Best practice might involve offering courses that employ a variety of teaching styles. Several scales are available for the standardization of learning styles. These scales employ a variety of learning style descriptors and are sometimes criticized as being measures of personality rather than learning style. Learning styles may become an increasingly relevant pedagogic concept as classes increase in size and diversity. This review will describe various learning style instruments as well as their potential use and limitations. Also discussed is the use of learning style theory in various concentrations including pharmacy.

INTRODUCTION

The diversity of students engaged in higher education continues to expand. Students come to colleges with varied ethnic and cultural backgrounds, from a multitude of training programs and institutions, and with differing learning styles. 1 Coupled with this increase in diversification has been a growth in distance education programs and expansions in the types of instructional media used to deliver information. 2 , 3 These changes and advances in technology have led many educators to reconsider traditional, uniform instruction methods and stress the importance of considering student learning styles in the design and delivery of course content. 4 - 5 Mismatches between an instructor's style of teaching and a student's method of learning have been cited as potential learning obstacles within the classroom and as a reason for using a variety of teaching modalities to deliver instruction. 6 - 8 The concept of using a menu of teaching modalities is based on the premise that at least some content will be presented in a manner suited to every type of learner within a given classroom or course. Some research has focused on profiling learning types so that instructors have a better understanding of the cohort of students they are educating. 7 - 8 This information can be used to guide the selection of instruction modalities employed in the classroom. Limited research has also focused on describing and characterizing composite learning styles and patterns for students in various concentrations of study (eg, medicine, engineering). 5 , 6 , 9 This review will describe the potential utility and limitations in assessing learning styles.

LEARNING STYLES

A benchmark definition of “learning styles” is “characteristic cognitive, effective, and psychosocial behaviors that serve as relatively stable indicators of how learners perceive, interact with, and respond to the learning environment. 10 Learning styles are considered by many to be one factor of success in higher education. Confounding research and, in many instances, application of learning style theory has begat the myriad of methods used to categorize learning styles. No single commonly accepted method currently exists, but alternatively several potential scales and classifications are in use. Most of these scales and classifications are more similar than dissimilar and focus on environmental preferences, sensory modalities, personality types, and/or cognitive styles. 11 Lack of a conceptual framework for both learning style theory and measurement is a common and central criticism in this area. In 2004 the United Kingdom Learning and Skills Research Center commissioned a report intended to systematically examine existing learning style models and instruments. In the commission report, Coffield et al identified several inconsistencies in learning style models and instruments and cautioned educators with regards to their use. 12 The authors also outlined a suggested research agenda for this area.

Alternatively, many researchers have argued that knowledge of learning styles can be of use to both educators and students. Faculty members with knowledge of learning styles can tailor pedagogy so that it best coincides with learning styles exhibited by the majority of students. 4 Alternatively, students with knowledge of their own preferences are empowered to use various techniques to enhance learning, which in turn may impact overall educational satisfaction. This ability is particularly critical and useful when an instructor's teaching style does not match a student's learning style. Compounding the issue of learning styles in the classroom has been the movement in many collegiate environments to distance and/or asynchronous education. 2 , 3 This shift in educational modality is inconsistent with the learning models with which most older students and adult learners are accustomed from their primary and high school education. 3 , 13 , 14 Alternatively, environmental influences and more widespread availability of technological advances (eg, personal digital assistants, digital video, the World Wide Web, wireless Internet) may make younger generations of students more comfortable with distance learning. 15 - 17

LEARNING STYLES INSTRUMENTS

As previously stated, several models and measures of learning styles have been described in the literature. Kolb proposed a model involving a 4-stage cyclic structure that begins with a concrete experience, which lends to a reflective observation and subsequently an abstract conceptualization that allows for active experimentation. 18 Kolb's model is associated with the Learning Style Inventory instrument (LSI). The LSI focuses on learner's preferences in terms of concrete versus abstract, and action versus reflection. Learners are subsequently described as divergers, convergers, assimilators, or accommodators.

Honey and Mumford developed an alternative instrument known as the Learning Style Questionnaire (LSQ). 6 Presumably, the LSQ has improved validity and predictive accuracy compared to the LSI. The LSQ describes 4 distinct types of learners: activists (learn primarily by experience), reflectors (learn from reflective observation), theorists (learn from exploring associations and interrelationships), and pragmatics (learn from doing or trying things with practical outcomes). The LSQ has been more widely used and studied in management and business settings and its applicability to academia has been questioned. 6 An alternative to the LSQ, the Canfield Learning Style Inventory (CLSI) describes learning styles along 4 dimensions. 19 These dimensions include conditions for learning, area of interest, mode of learning, and conditions for performance. Analogous to the LSQ, applicability of the CLSI to academic settings has been questioned. Additionally, some confusion surrounding scoring and interpretation of certain result values also exists.

Felder and Silverman introduced a learning style assessment instrument that was specifically designed for classroom use and was first applied in the context of engineering education. 20 The instrument consists of 44 short items with a choice between 2 responses to each sentence. Learners are categorized in 4 dichotomous areas: preference in terms of type and mode of information perception (sensory or intuitive; visual or verbal), approaches to organizing and processing information (active or reflective), and the rate at which students progress towards understanding (sequential or global). The instrument associated with the model is known as the Index of Learning Survey (ILS). 21 The ILS is based on a 44-item questionnaire and outputs a preference profile for a student or an entire class. The preference profile is based on the 4 previously defined learning dimensions. The ILS has several advantages over other instruments including conciseness and ease of administration (in both a written and computerized format). 20 , 21 No published data exist with regards to the use of the ILS in populations of pharmacy students or pharmacists. Cook described a study designed to examine the reliability of the ILS for determining learning styles among a population of internal medicine residents. 20 The researchers administered the ILS twice and the Learning Style Type Indicator (LSTI) once to 138 residents (86 men, 52 women). The LSTI has been previously compared to the ILS by several investigators. 8 , 19 Cook found that the Cronbach's alpha scores for the ILS and LSTI ranged from 0.19 to 0.69. They preliminarily concluded that the ILS scores were reliable and valid among this cohort of residents, particularly within the active-reflective and sensing-intuitive domains. In a separate study, Cook et al attempted to evaluate convergence and discrimination among the ILS, LSI, and another computer-based instrument known as the Cognitive Styles Analysis (CSA). 11 The cohort studied consisted of family medicine and internal medicine residents as well as first- and third-year medical students. Eighty-nine participants completed all 3 instruments, and responses were analyzed using calculated Pearson's r and Cronbach's α. The authors found that the ILS active-reflective and sensing-intuitive scores as well as the LSI active-reflective scores were valid in determining learning styles. However, the ILS sequential-global domain failed to correlate well with other instruments and may be flawed, at least in this given population. The authors advised the use of caution when interpreting scores without a strong knowledge of construct definitions and empirical evidence.

Several other instruments designed to measure personality indexes or psychological types may overlap and describe learning styles in nonspecific fashions. One example of such an indicator is the Myers-Briggs Index. 6 While some relation between personality indexes and learning styles may exist, the use of instruments intended to describe personality to characterize learning style has been criticized by several authors. Therefore, the use of these markers to measure learning styles is not recommended. 6 The concept of emotional intelligence is another popular way to characterize intellect and learning capacity but similarly should not be misconstrued as an effective means of describing learning styles. 23

Several authors have proposed correlations between culture and learning styles. 6 , 24 This is predicated on the concept that culture influences environmental perceptions which, in turn, to some degree determine the way in which information is processed and organized. The storage, processing, and assimilation methods for information contribute to how new knowledge is learned. Culture also plays a role in conditioning and reinforcing learning styles and partially explains why teaching methods used in certain parts of the world may be ineffective or less effective when blindly transplanted to another locale. 6 , 24 Teachers should be aware of this phenomenon and the influence it has on the variety of learning styles that are present in classrooms. This is especially true in classrooms that have a large contingency of international students. Such classrooms are becoming increasingly common as more and more schools expand their internationalization efforts. 25

The technological age may also be influencing the learning styles of younger students and emerging generations of learners. The Millennial Generation has been described as more technologically advanced than their Generation X counterparts, with higher expectations for the use of computer-aided media in the classroom. 15 , 16 , 26 Younger students are accustomed to enhanced visual images associated with various computer- and television-based games and game systems. 16 , 26 Additionally, video technology is increasingly becoming “transportable” in the way of mobile computing, MP3 devices, personal digital video players, and other technologies. 26 All of these advances have made visual images more pervasive and common within industrialized nations.

APPLYING LEARNING STYLES TO THE CLASSROOM

As class sizes increase, so do the types and numbers of student learning styles. Also, as previously mentioned, internationalization and changes in the media culture may affect the spectrum of classroom learning styles as well. 24 , 25 Given the variability in learning styles that may exist in a classroom, some authors suggested that students should adapt their learning styles to coincide with a given instruction style. 6 , 27 This allows instructors to dictate the methods used to instruct in the classroom. This approach also allows instructors to “teach from their strengths,” with little consideration to other external factors such as learning style of students. While convenient, this unilateral approach has been criticized for placing all of the responsibility for aligning teaching and learning on the student. When the majority of information is presented in formats that are misaligned with learning styles, students may spend more time manipulating material than they do in comprehending and applying the information. Additionally, a unilaterally designed classroom may reinforce a “do nothing” approach among faculty members. 6 Alternatively, a teaching style-learning style mismatch might challenge students to adjust, grow intellectually, and learn in more integrated ways. However, it may be difficult to predict which students have the baseline capacity to adjust, particularly when significant gaps in knowledge of a given subject already exist or when the learner is a novice to the topic being instructed. 6 , 27 This might be especially challenging within professional curricula where course load expectations are significant.

Best practice most likely involves a teaching paradigm which addresses and accommodates multiple dimensions of learning styles that build self-efficacy. 27 Instructing in a way that encompasses multiple learning styles gives the teacher an opportunity to reach a greater extent of a given class, while also challenging students to expand their range of learning styles and aptitudes at a slower pace. This may avoid lost learning opportunities and circumvent unnecessary frustration from both the teacher and student. For many instructors, multi-style teaching is their inherent approach to learning, while other instructors more commonly employ unilateral styles. Learning might be better facilitated if instructors were cognizant of both their teaching styles and the learning styles of their students. An understanding and appreciation of a given individual's teaching style requires self-reflection and introspection and should be a component of a well-maintained teaching portfolio. Major changes or modifications to teaching styles might not be necessary in order to effectively create a classroom atmosphere that addresses multiple learning styles or targets individual ones. However, faculty members should be cautious to not over ambitiously, arbitrarily, or frivolously design courses and activities with an array of teaching modalities that are not carefully connected, orchestrated, and delivered.

Novice learners will likely be more successful when classrooms, either by design or by chance, are tailored to their learning style. However, the ultimate goal is to instill within students the skills to recognize and react to various styles so that learning is maximized no matter what the environment. 28 This is an essential skill for an independent learner and for students in any career path.

Particular consideration of learning styles might be given to asynchronous courses and other courses where a significant portion of time is spent online. 29 As technology advances and classroom sizes in many institutions become increasingly large, asynchronous instruction is becoming more pervasive. In many instances, students who have grown accustomed to technological advances may prefer asynchronous courses. Online platforms may inherently affect learning on a single dimension (visual or auditory). Most researchers who have compared the learning styles of students enrolled in online versus traditional courses have found no correlations between the learning styles and learning outcomes of cohorts enrolled in either course type. Johnson et al compared learning style profiles to student satisfaction with either online or face-to-face study groups. 30 Forty-eight college students participated in the analysis. Learning styles were measured using the ILS. Students were surveyed with regard to their satisfaction with various study group formats. These results were then correlated to actual performance on course examinations. Active and visual learners demonstrated a significant preference for face-to-face study groups. Alternatively, students who were reflective learners demonstrated a preference for online groups. Likely due to the small sample size, none of these differences achieved statistical significance. The authors suggest that these results are evidence for courses employing hybrid teaching styles that reach as many different students as possible. Cook et al studied 121 internal medicine residents and also found no association (p > 0.05) between ILS-measured learning styles and preferences for learning formats (eg, Web-based versus paper-based learning modules). 31 Scores on assessment questions related to learning modules administered to the residents were also not statistically correlated with learning styles.

Cook et al examined the effectiveness of adapting Web-based learning modules to a given learner's style. 32 The investigators created 2 versions of a Web-based instructional module on complementary and alternative medications. One version of the modules directed the learner to “active” questions that provided learners immediate and comprehensive feedback, while the other version involved “reflective” questions that directed learners back to the case content for answers. Eighty-nine residents were randomly matched or mismatched based on their active-reflective learning styles (as determined by ILS) to either the “active” or “reflective” test version. Posttest scores for either question type among mismatched subjects did not differ significantly ( p = 0.97), suggesting no interaction between learning styles and question types. The authors concluded from this small study that learning styles had no influence on learning outcomes. The study was limited in its lack of assessment of baseline knowledge, motivation, or other characteristics. Also, the difficulty of the assessment may not have been sufficient enough to distinguish a difference and/or “mismatched” learners may have automatically adapted to the information they received regardless of type.

STUDIES OF PHARMACY STUDENTS

There are no published studies that have systematically examined the learning styles of pharmacy students. Pungente et al collected some learning styles data as part of a study designed to evaluate how first-year pharmacy students' learning styles influenced preferences toward different activities associated with problem-based learning (PBL). 33 One hundred sixteen first-year students completed Kolb's LSI. Learning styles were then matched to responses from a survey designed to assess student preferences towards various aspects of PBL. The majority of students were classified by the LSI as being accommodators (36.2%), with a fairly even distribution of styles among remaining students (19.8% assimilators, 22.4% convergers, 21.6% divergers). There was a proportional distribution of learning styles among a convenience sample of pharmacy students. Divergers were the least satisfied with the PBL method of instruction, while convergers demonstrated the strongest preference for this method of learning. The investigators proposed that the next step might be to correlate learning styles and PBL preferences with actual academic success.

Limited research correlating learning styles to learning outcomes has hampered the application of learning style theory to actual classroom settings. Complicating research is the plethora of different learning style measurement instruments available. Despite these obstacles, efforts to better define and utilize learning style theory is an area of growing research. A better knowledge and understanding of learning styles may become increasingly critical as classroom sizes increase and as technological advances continue to mold the types of students entering higher education. While research in this area continues to grow, faculty members should make concentrated efforts to teach in a multi-style fashion that both reaches the greatest extent of students in a given class and challenges all students to grow as learners.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Learning Methods

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• Last »
• Un-Schooling and Un-Traditional Education Follow Following
• Spam Filtering Follow Following
• Support vector machine Follow Following
• Learning to Rank Follow Following
• Tagucghi Loss Function Follow Following
• Alternative Education Follow Following
• Comparative Analysis Follow Following
• Social Geography Follow Following
• Support Vector Machines Follow Following
• Collaborative Filtering Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Schools are using research to try to improve children’s learning – but it’s not working

Senior Research Fellow in the Centre for Teachers and Teaching Research, UCL

Disclosure statement

Sally Riordan is currently working on two projects that receive funding from the Education Endowment Foundation.

University College London provides funding as a founding partner of The Conversation UK.

View all partners

Evidence is obviously a good thing. We take it for granted that evidence from research can help solve the post-lockdown crises in education – from how to keep teachers in the profession to how to improve behaviour in schools, get children back into school and protect the mental health of a generation.

But my research and that of others shows that incorporating strategies that have evidence backing them into teaching doesn’t always yield the results we want.

The Department for Education encourages school leadership teams to cite evidence from research studies when deciding how to spend school funding. Teachers are more frequently required to conduct their own research as part of their professional training than they were a decade ago. Independent consultancies have sprung up to support schools to bring evidence-based methods into their teaching.

This push for evidence to back up teaching methods has become particularly strong in the past ten years. The movement has been driven by the Education Endowment Foundation (EEF), a charity set up in 2011 with funding from the Conservative-Liberal Democrat coalition government to provide schools with information about which teaching methods and other approaches to education actually work.

The EEF funds randomised controlled trials – large-scale studies in which students are randomly assigned to an educational initiative or not and then comparisons are then made to see which students perform better. For instance, several of these studies have been carried out in which some children received one-on-one reading sessions with a trained classroom assistant, and their reading progress was compared to children who had not. The cost of one of these trials was around £500,000 over the course of a year.

Trials such as this in education were lobbied for by Ben Goldacre , a doctor and data scientist who wrote a report in 2013 on behalf of the Department for Education. Goldacre suggested that education should follow the lead of medicine in the use of evidence.

Using evidence

In 2023, however, researchers at the University of Warwick pointed out something that should have been obvious for some time but has been very much overlooked – that following the evidence is not resulting in the progress we might expect.

Reading is the most heavily supported area of the EEF’s research, accounting for more than 40% of projects . Most schools have implemented reading programmes with significant amounts of evidence behind them. But, despite this, reading abilities have not changed much in the UK for decades.

This flatlining of test scores is a global phenomenon . If reading programmes worked as the evidence says they do, reading abilities should be better.

And the evidence is coming back with unexpected results. A series of randomised controlled trials, including one looking at how to improve literacy through evidence , have suggested that schools that use methods based on research are not performing better than schools that do not.

In fact, research by a team at Sheffield Hallam University have demonstrated that on average, these kinds of education initiatives have very little to no impact .

My work has shown that when the findings of different research studies are brought together and synthesised, teachers may end up implementing these findings in contradictory ways. Research messages are frequently too vague to be effective because the skills and expertise of teaching are difficult to transfer.

It is also becoming apparent that the gains in education are usually very small, perhaps because learning is the sum total of trillions of interactions. It is possible that the research trials we really need in education would be so vast that they are currently too impractical to do.

It seems that evidence is much harder to tame and to apply sensibly in education than elsewhere. In my view, it was inevitable and necessary that educators had to follow medicine in our search for answers. But we now need to think harder about the peculiarities of how evidence works in education.

Right now, we don’t have enough evidence to be confident that evidence should always be our first port of call.

• Young people
• Department for Education

Senior Research Development Coordinator

Audience Development Coordinator (fixed-term maternity cover)

Lecturer (Hindi-Urdu)

Director, Defence and Security

Opportunities with the new CIEHF

Advertisement

Machine learning in landscape ecological analysis: a review of recent approaches

• Review Article
• Published: 01 December 2021
• Volume 37 , pages 1227–1250, ( 2022 )

Cite this article

• Mihai-Sorin Stupariu   ORCID: orcid.org/0000-0002-9531-4576 1 , 2 ,
• Samuel A. Cushman 3 ,
• Alin-Ionuţ Pleşoianu 2 ,
• Ileana Pătru-Stupariu 2 , 4 &
• Christine Fürst 5  

3659 Accesses

25 Citations

3 Altmetric

Explore all metrics

Artificial Intelligence (AI) has rapidly developed over the past several decades. Several related AI approaches, such as Machine Learning (ML), have been applied to research on landscape patterns and ecological processes.

Our goal was to review the methods of AI, particularly ML, used in studies related to landscape ecology and the main topics addressed. We aimed to assess the trend in the number of ML papers and the methods used therein, and provide a synopsis and prospectus of current use and future applications of ML in landscape ecology.

We conducted a systematic literature search and selected 125 papers for review. These were examined and scored according to multiple criteria regarding methods and topic. We applied quantitative statistical methods, including cluster analysis based on titles, abstracts, and keywords and a non-metric multidimensional scaling based on attributes assigned during the review. We used Random Forests machine learning to describe the differences between identified clusters in terms of the topics and methods they included.

The most frequent method found was Random Forests, but it is noteworthy to mention the increasing popularity of tools related to Deep Learning. The topics cover both ecologically oriented issues and the landscape-human interface. There has been a rapid increase in ML and AI methods in landscape ecology research, with Deep Learning and complex multi-step pipeline AI methods emerging in the last several years.

Conclusions

The rapid increase in the number of ML papers in landscape ecology research, and the range of methods employed in them, suggest explosive growth in application of these methods in landscape ecology. The increase of Deep Learning approaches in the most recent years suggest a major change in analytical paradigms and methodologies that we feel may transform the field and enable analyses of more complex pattern process relationships across vaster data sets than has been possible previously.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

adapted from Willcock et al. 2018 , Fayyad et al. 1996 . The machine learning approaches are adapted from Géron 2019 , Hapke and Nelson 2020 , and Zhou 2018

Similar content being viewed by others

Don’t throw the baby out with the bathwater: reappreciating the dynamic relationship between humans, machines, and landscape images

Raechel A. Portelli

Remote Sensing’s Recent and Future Contributions to Landscape Ecology

Morgan A. Crowley & Jeffrey A. Cardille

Landscape Applications of Machine Learning: Comparing Random Forests and Logistic Regression in Multi-Scale Optimized Predictive Modeling of American Marten Occurrence in Northern Idaho, USA

Data availability.

Data included as supplementary material.

Code availability

Not applicable.

Abbreviations

• Artificial intelligence

Boosted regression trees

Convolutional neural networks

Decision trees

Expert systems

Generalized additive models

Logistic regression

• Machine learning

Maximum entropy

Model improvement ratio

Non-metric multidimensional scaling

Neural networks

Random forests

Recurrent neural networks

Supervised learning

Support vector machines

Unsupervised learning

Web of Science

XGBoost—gradient boosting machine

Aiba M, Kurokawa H, Onoda Y, Oguro M, Nakashizuka T, Masaki T (2016) Context-dependent changes in the functional composition of tree communities along successional gradients after land-use change. J Ecol 104(5):1347–1356

Article   Google Scholar  

Albert A, Kaur J, Gonzalez MC (2017) Using convolutional networks and satellite imagery to identify patterns in Urban environments at a large scale. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1357–1366

Alshehhi R, Marpu PR, Woon WL, Dalla Mura M (2017) Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS J Photogramm Remote Sens 130:139–149

Altartouri A, Nurminen L, Jolma A (2015) Spatial neighborhood effect and scale issues in the calibration and validation of a dynamic model of Phragmites australis distribution—a cellular automata and machine learning approach. Environ Model Softw 71:15–29

Anderson EF, McLoughlin L, Liarokapis F, Peters C, Petridis P, de Freitas S (2010) Developing serious games for cultural heritage: a state-of-the-art review. Virtual Real 14(4):255–275

Androutsopoulou A, Karacapilidis N, Loukis E, Charalabidis Y (2019) Transforming the communication between citizens and government through AI-guided chatbots. Gov Inf Q 36(2):358–367

Arndt J, Lunga D, Weaver J, LeDoux T, Tennille S (2019) Multiscale based characterization and classification of urban land-use. 2019 In: IEEE International geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 9470–9473

Aschwanden G (2016) Neighbourhood detection with analytical tools. In: Proceedings of the 21st international conference on computer—aided architectural design research in Asia, pp. 13–22

Asner GP, Sousan S, Knapp DE et al (2016) Rapid forest carbon assessments of oceanic islands: a case study of the Hawaiian archipelago. Carbon Balance and Manag. https://doi.org/10.1186/s13021-015-0043-4

Athukorala D, Murayama Y (2020) Spatial variation of land use/cover composition and impact on surface Urban Heat Island in a Tropical Sub-Saharan City of Accra, Ghana. Sustainability. https://doi.org/10.3390/su12197953

Bahrammirzaee A (2010) A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems. Neural Comput Appl 19(8):1165–1195

Baltensperger AP, Joly K (2019) Using seasonal landscape models to predict space use and migratory patterns of an arctic ungulate. Mov Ecol. https://doi.org/10.1186/s40462-019-0162-8

Article   PubMed   PubMed Central   Google Scholar  

Balzotti CS, Asner GP (2018) Biotic and abiotic controls over canopy function and structure in humid Hawaiian forests. Ecosystems 21(2):331–348

Article   CAS   Google Scholar  

Balzotti CS, Asner GP, Adkins ED, Parsons EW (2020) Spatial drivers of composition and connectivity across endangered tropical dry forests. J Appl Ecol 57(8):1593–1604

Barbosa JM, Asner GP (2017) Prioritizing landscapes for restoration based on spatial patterns of ecosystem controls and plant-plant interactions. J Appl Ecol 54(5):1459–1468

Bastille-Rousseau G, Wittemyer G (2021) Characterizing the landscape of movement to identify critical wildlife habitat and corridors. Conserv Biol 35(1):346–359

Article   PubMed   Google Scholar  

Bayr U, Puschmann O (2019) Automatic detection of woody vegetation in repeat landscape photographs using a convolutional neural network. Eco Inform 50:220–233

Belgiu M, Dragut L (2016) Random forest in remote sensing: a review of applications and future directions. ISPRS J Photogramm Remote Sens 114:24–31

Bone C, Dragicevic S (2009) Defining transition rules with reinforcement learning for modeling land cover change. Simul-Trans Soc Modeling Simula Int 85(5):291–305

Bounas A, Keroglidou M, Toli EA et al (2020) Constrained by aliens, shifting landscape, or poor water quality? Factors affecting the persistence of amphibians in an urban pond network. Aquat Conserv-Mar Freshw Ecosyst 30(5):1037–1049

Breiman L (2001) Random forests. Mach Learn 45(1):5–32

Brieuc MSO, Waters CD, Drinan DP, Naish KA (2018) A practical introduction to random forest for genetic association studies in ecology and evolution. Mol Ecol Resour 18(4):755–766

Brock PM, Fornace KM, Grigg MJ et al (2019) Predictive analysis across spatial scales links zoonotic malaria to deforestation. In: Proceedings of the Royal Society B-Biological Sciences 286(1894)

Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning going beyond Euclidean data. IEEE Signal Process Mag 34(4):18–42

Brovelli MA, Crespi M, Fratarcangeli F, Giannone F, Realini E (2008) Accuracy assessment of high resolution satellite imagery orientation by leave-one-out method. ISPRS J Photogramm Remote Sens 63(4):427–440

Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167

Chang NB, Xuan ZM, Wimberly B (2011) MODIS-based spatiotemporal patterns of soil moisture and evapotranspiration interactions in Tampa Bay urban watershed. In: Gao W, Jackson TJ, Wang J, Chang NB (eds), Remote sensing and modeling of ecosystems for sustainability VIII, Proceedings of SPIE.

Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

Chapman DS, Bonn A, Kunin WE, Cornell SJ (2010) random forest characterization of upland vegetation and management burning from aerial imagery. J Biogeogr 37(1):37–46

Chen S, Whiteman A, Li A et al (2019) An operational machine learning approach to predict mosquito abundance based on socioeconomic and landscape patterns. Landsc Ecol 34(6):1295–1311

Chen X, Zhao J, Chen YH, Zhou W, Hughes AC (2020) Automatic standardized processing and identification of tropical bat calls using deep learning approaches. Biol Conserv. https://doi.org/10.1016/j.biocon.2019.108269

Christensen M, Arsanjani JJ (2020) Stimulating implementation of sustainable development goals and conservation action: predicting future land use/cover change in Virunga National Park, Congo. Sustainability. https://doi.org/10.3390/su12041570

Christin S, Hervet E, Lecomte N (2019) Applications for deep learning in ecology. Methods Ecol Evol 10(10):1632–1644

COE (2020) Council of Europe and Artificial Intelligence. https://www.coe.int/en/web/artificial-intelligence/home . Accessed 16 March 2021

Cracknell MJ, Reading AM (2014) Geological mapping using remote sensing data: a comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information. Comput Geosci 63:22–33

Curry CM, Ross JD, Contina AJ, Bridge ES (2018) Varying dataset resolution alters predictive accuracy of spatially explicit ensemble models for avian species distribution. Ecol Evol 8(24):12867–12878

Cushman SA, Huettmann F (2010) Spatial Complexity, Informatics, and Wildlife Conservation. Springer

Cushman SA, Wasserman TN (2018) Landscape applications of machine learning: Comparing random forests and logistic regression in multi-scale optimized predictive modeling of American marten occurrence in northern Idaho, USA. In: Humphries G, Magness D, Huettmann F (eds) Machine learning for ecology and sustainable natural resource management. Springer, Cham, pp 185–203

Chapter   Google Scholar  

Cushman SA, Macdonald E, Landguth E, Malhi Y, Macdonald D (2017) Multiple-scale prediction of forest loss risk across Borneo. Landsc Ecol 32(8):1581–1598

Cutler DR, Edwards TC, Beard KH, Cutler A, Hess KT (2007) Random forests for classification in ecology. Ecology 88(11):2783–2792

Davies AB, Brodrick PG, Parr CL, Asner GP (2020) Resistance of mound-building termites to anthropogenic land-use change. Environ Res Lett. https://doi.org/10.1088/1748-9326/aba0ff

Day CC, Zollner PA, Gilbert JH, McCann NP (2020) Individual-based modeling highlights the importance of mortality and landscape structure in measures of functional connectivity. Landsc Ecol 35(10):2191–2208

Debanshi S, Pal S (2020) Effects of water richness and seasonality on atmospheric methane emission from the wetlands of deltaic environment. Ecol Indic. https://doi.org/10.1016/j.ecolind.2020.106767

Debanshi S, Pal S (2020) Modelling water richness and habitat suitability of the wetlands and measuring their spatial linkages in mature Ganges delta of India. J Environ Manag. https://doi.org/10.1016/j.jenvman.2020.110956

Debats SR, Luo D, Estes LD, Fuchs TJ, Caylor KK (2016) A generalized computer vision approach to mapping crop fields in heterogeneous agricultural landscapes. Remote Sens Environ 179:210–221

Deo RC (2015) Machine learning in medicine. Circulation 132(20):1920–1930

Desjardins-Proulx P, Poisot T, Gravel D (2019) Artificial intelligence for ecological and evolutionary synthesis. Front Ecol Evol 7:12

Dimov D, Low F, Ibrakhimov M, Sarah Schonbrodt S, Conrad C (2017) Feature extraction and machine learning for the classification of active cropland in the Aral Sea basin. In: 2017 IEEE international geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 1804–1807

Djuric U, Marjanovic M, Susic V et al (2013) Land-use suitability analysis of Belgrade city suburbs using machine learning algorithm. In: Ivan I, Longley P, Fritsch D et al (eds) GIS Ostrava 2013 - Geoinformatics for City Transformation, pp. 49-61

Donnelly JP, Tack JD, Doherty KE, Naugle DE, Allred BW, Dreitz VJ (2017) Extending conifer removal and landscape protection strategies from sage-grouse to songbirds, a range-wide assessment. Rangel Ecol Manag 70:95–105

Drake JM, Randin C, Guisan A (2006) Modelling ecological niches with support vector machines. J Appl Ecol 43(3):424–443

Dronova I, Gong P, Clinton NE et al (2012) Landscape analysis of wetland plant functional types: the effects of image segmentation scale, vegetation classes and classification methods. Remote Sens Environ 127:357–369

Eikelboom JAJ, de Knegt HJ, Klaver M, van Langevelde F, van der Wal T, Prins HHT (2020) Inferring an animal’s environment through biologging: quantifying the environmental influence on animal movement. Mov Ecol. https://doi.org/10.1186/s40462-020-00228-4

Elith J, Graham CH, Anderson RP et al (2006) Novel methods improve prediction of species’ distributions from occurrence data. Ecography 29(2):129–151

Elith J, Leathwick JR, Hastie T (2008) A working guide to boosted regression trees. J Anim Ecol 77:802–813

Article   CAS   PubMed   Google Scholar  

European Commission (2019) A definition of AI: main capabilities and scientific disciplines. Definition developed for the purpose of the deliverables of the EC high-level expert group on artificial intelligence. https://digital-strategy.ec.europa.eu/en/library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines . Accessed 10 July 2021

Evans JS, Cushman SA (2009) Gradient modeling of conifer species using random forests. Landsc Ecol 24:673–683

Fagerholm N, Kayhko N, Ndumbaro F, Khamis M (2012) Community stakeholders’ knowledge in landscape assessments—mapping indicators for landscape services. Ecol Ind 18:421–433

Farley SS, Dawson A, Goring SJ, Williams JW (2018) Situating ecology as a big-data science: current advances, challenges, and solutions. Bioscience 68(8):563–576

Fayyad U, Piatetsky-Shapiro G, Smyth P (1996) From data mining to knowledge discovery in databases. AI Mag 17(3):37–54

Google Scholar  

Feng YJ, Liu Y, Batty M (2016) Modeling urban growth with GIS based cellular automata and least squares SVM rules: a case study in Qingpu-Songjiang area of Shanghai, China. Stoch Env Res Risk Assess 30(5):1387–1400

Folmer EO, van Beusekom JEE, Dolch T et al (2016) Consensus forecasting of intertidal seagrass habitat in the Wadden Sea. J Appl Ecol 53(6):1800–1813

Fountain-Jones NM, Craft ME, Funk WC et al (2017) Urban landscapes can change virus gene flow and evolution in a fragmentation-sensitive carnivore. Mol Ecol 26(22):6487–6498

Fox JT, Magoulick DD (2019) Predicting hydrologic disturbance of streams using species occurrence data. Sci Total Environ 686:254–263

Franklin J (1995) Predictive vegetation mapping: geographic modelling of biospatial patterns in relation to environmental gradients. Prog Phys Geogr 19(4):474–499

Frommhold M, Heim A, Barabanov M et al (2019) Breeding habitat and nest-site selection by an obligatory “nest-cleptoparasite”, the Amur Falcon Falco amurensis . Ecol Evol 9(24):14430–14441

Géron A (2019) Hands-on machine learning with scikit-learn, keras, and tensorflow. O’Reilly, Sebastopol

Geurts P, Irrthum A, Wehenkel L (2009) Supervised learning with decision tree-based methods in computational and systems biology. Mol BioSyst 5(12):1593–1605

Giles JR, Plowright RK, Eby P, Peel AJ, McCallum H (2016) Models of eucalypt phenology predict bat population flux. Ecol Evol 6(20):7230–7245

Ginau A, Steiniger D, Hartmann R et al (2020) What settlements leave behind—pXRF compositional data analysis of archaeological layers from Tell el-Fara’in using machine learning (Buto, Egypt). Palaeogeogr Palaeoclimatol Palaeoecol. https://doi.org/10.1016/j.palaeo.2020.109666

Giri S, Zhang Z, Krasnuk D, Lathrop RG (2019) Evaluating the impact of land uses on stream integrity using machine learning algorithms. Sci Total Environ. https://doi.org/10.1016/j.scitotenv.2019.133858

Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

Gray ME, Zachmann LJ, Dickson BG (2018) A weekly, continually updated dataset of the probability of large wildfires across western US forests and woodlands. Earth Syst Sci Data 10(3):1715–1727

Greaves HE, Eitel JUH, Vierling LA et al (2019) 20 cm resolution mapping of Tundra vegetation communities provides an ecological baseline for important research areas in a changing Arctic environment. Environ Res Commun. https://doi.org/10.1088/2515-7620/ab4a85

Griffiths P, Hostert P, Gruebner O, van der Linden S (2010) Mapping megacity growth with multi-sensor data. Remote Sens Environ 114(2):426–439

Grys BT, Lo DS, Sahin N et al (2017) Machine learning and computer vision approaches for phenotypic profiling. J Cell Biol 216(1):65–71

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gutzwiller KJ, Chaudhary A (2020) Machine-learning models, cost matrices, and conservation based reduction of selected landscape classification errors. Landsc Ecol 35:249–255

Halilaj E, Rajagopal A, Fiterau M, Hicks JL, Hastie TJ, Delp SL (2018) Machine learning in human movement biomechanics: Best practices, common pitfalls, and new opportunities. J Biomech 81:1–11

Hapke H, Nelson C (2020) Building machine learning pipelines. O’Reilly, Sebastopol

Hassell JM, Bettridge JM, Ward MJ et al (2021) Socio-ecological drivers of vertebrate biodiversity and human-animal interfaces across an urban landscape. Glob Change Biol 27(4):781–792

Helmer EH, Kennaway TA, Pedreros DH et al (2008) Land cover and forest formation distributions for St. Kitts, Nevis, St. Eustatius, Grenada and Barbados from decision tree classification of cloud-cleared satellite imagery. Caribb J Sci 44(2):175–198

Henderson EB, Ohmann JL, Gregory MJ, Roberts HM, Zald H (2014) Species distribution modelling for plant communities: stacked single species or multivariate modelling approaches? Appl Veg Sci 17(3):516–527

Hernandez-Orallo J (2017) Evaluation in artificial intelligence: from task-oriented to ability-oriented measurement. Artif Intell Rev 48(3):397–447

Hether TD, Hoffman EA (2012) Machine learning identifies specific habitats associated with genetic connectivity in Hyla squirella. J Evol Biol 25(6):1039–1052

Hopton ME, Mayer AL (2006) Using self-organizing maps to explore patterns in species richness and protection. Biodivers Conserv 15(14):4477–4494

Horning N, Fleishman E, Ersts PJ, Fogarty FA, Zillig MW (2020) Mapping of land cover with open-source software and ultra-high-resolution imagery acquired with unmanned aerial vehicles. Remote Sens Ecol Conserv 6(4):487–497

Hounkpatin OKL, de Hipt FO, Bossa AY, Welp G, Amelung W (2018) Soil organic carbon stocks and their determining factors in the Dano catchment (Southwest Burkina Faso). CATENA 166:298–309

Huang C, Davis LS, Townshend JRG (2002) An assessment of support vector machines for land cover classification. Int J Remote Sens 23(4):725–749

Huang B, Xie CL, Tay R, Wu B (2009) Land-use-change modeling using unbalanced support-vector machines. Environ Plan B-Plan Des 36(3):398–416

Huang B, Zhao B, Song YM (2018) Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery. Remote Sens Environ 214:73–86

Hudak AT, Strand EK, Vierling LA et al (2012) Quantifying aboveground forest carbon pools and fluxes from repeat LiDAR surveys. Remote Sens Environ 123:25–40

Huete A, Tran NN, Nguyen H, Xie QY, Katelaris C (2019) Forecasting pollen aerobiology with modis evi, land cover, and phenology using machine learning tools. In: 2019 IEEE international geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 5429–5432

Huettmann F, Craig EH, Herrick KA, Baltensperger AP, Humphries GRW, Lieske DJ, Miller K, Mullet TC, Oppel S, Resendiz C, Rutzen I, Schmid MS, Suwal MK, Young BD (2018) Use of machine learning (ML) for predicting and analyzing ecological and ‘presence only’ data: an overview of applications and a good outlook. In: Humphries G, Magness DR, Huettmann F (eds) Machine learning for ecology and sustainable natural resource management. Springer, Cham, pp 27–61

Jahani A, Rayegani B (2020) Forest landscape visual quality evaluation using artificial intelligence techniques as a decision support system. Stoch Env Res Risk Assess 34(10):1473–1486

Jasiewicz J, Netzel P, Stepinski TF (2014) Landscape similarity, retrieval, and machine mapping of physiographic units. Geomorphology 221:104–112

Jeong GY, Oeverdieck H, Park SJ, Huwe B, Liess M (2017) Spatial soil nutrients prediction using three supervised learning methods for assessment of land potentials in complex terrain. CATENA 154:73–84

Jones JS, Tullis JA, Haavik LJ, Guldin JM, Stephen FM (2014) Monitoring oak-hickory forest change during an unprecedented red oak borer outbreak in the Ozark Mountains: 1990 to 2006. J Appl Remote Sens. https://doi.org/10.1117/1.JRS.8.083687

Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260

Joshi A (2020) Machine learning and artificial intelligence. Springer, Cham

Book   Google Scholar  

Kampichler C, Sierdsema H (2018) On the usefulness of prediction intervals for local species distribution model forecasts. Eco Inform 47:67–72

Karasov O, Heremans S, Kulvik M, Domnich A, Chervanyov I (2020) On how crowdsourced data and landscape organisation metrics can facilitate the mapping of cultural ecosystem services: an Estonian case study. Land 9(5):158

Kennedy RE, Yang ZQ, Braaten J et al (2015) Attribution of disturbance change agent from Landsat time-series in support of habitat monitoring in the Puget Sound region, USA. Remote Sens Environ 166:271–285

Kerebel A, Gelinas N, Dery S, Voigt B, Munson A (2019) Landscape aesthetic modelling using Bayesian networks: conceptual framework and participatory indicator weighting. Landsc Urban Plan 185:258–271

Keshtkar H, Voigt W, Alizadeh E (2017) Land-cover classification and analysis of change using machine-learning classifiers and multi-temporal remote sensing imagery. Arabian J Geosci 10(6):1–5

Khan A, Sohail A, Zahoora U, Qureshi AS (2020) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev 53(8):5455–5516

Klippel A, Zhao JY, Jackson KL et al (2019) Transforming earth science education through immersive experiences: delivering on a long held promise. J Educ Comput Res 57(7):1745–1771

Konapala G, Kao SC, Painter SL, Lu D (2020) Machine learning assisted hybrid models can improve streamflow simulation in diverse catchments across the conterminous US. Environ Res Lett 15(10):104022

Kong L, Liu Z, Wu J (2020) A systematic review of big data-based urban sustainability research: state-of-the-science and future directions. J Clean Prod 273:123142

Kruskal JB (1964) Nonmetric multidimensional-scaling—a numerical-method. Psychometrika 29(2):115–129

Kumar SU, Maini PK, Chiaverini L, Hearn AJ, Macdonald DW, Kaszta Z, Cushman SA (2021) Smoothing and the environmental manifold. Ecol Infor 66:101472. https://doi.org/10.1016/j.ecoinf.2021.101472

Lary DJ, Alavi AH, Gandomi AH, Walker AL (2016) Machine learning in geosciences and remote sensing. Geosci Front 7(1):3–10

Levers C, Schneider M, Prishchepov AV, Estel S, Kuemmerle T (2018) Spatial variation in determinants of agricultural land abandonment in Europe. Sci Total Environ 644:95–111

Li CM, Zhang K, Dai ZX, Ma ZT, Liu XL (2020) Investigation of the impact of land-use distribution on PM(2.5)in Weifang: seasonal variations. Int J Environ Res Public Health 17(14):5135

Article   PubMed Central   Google Scholar  

Libbrecht MW, Noble WS (2015) Machine learning applications in genetics and genomics. Nat Rev Genet 16(6):321–332

Lidberg W, Nilsson M, Agren A (2020) Using machine learning to generate high-resolution wet area maps for planning forest management: a study in a boreal forest landscape. Ambio 49(2):475–486

Liu ZL, Peng CH, Work T, Candau JN, DesRochers A, Kneeshaw D (2018) Application of machine-learning methods in forest ecology: recent progress and future challenges. Environ Rev 26(4):339–350

Liu C, Cao YJ, Yang C, Zhou Y, Ai MC (2020) Pattern identification and analysis for the traditional village using low altitude UAV-borne remote sensing: multifeatured geospatial data to support rural landscape investigation, documentation and management. J Cult Herit 44:185–195

Liu C, White M, Newell G, Griffioen P (2011) Species distribution modelling for conservation planning in Victoria of Australia. In: Chan F, Marinova D, Anderssen RS (eds) 19th International Congress on Modelling and Simulation (MODSIM2011), pp 2247–2253

Lorilla RS, Poirazidis K, Detsis V, Kalogirou S, Chalkias C (2020) Socio-ecological determinants of multiple ecosystem services on the Mediterranean landscapes of the Ionian Islands (Greece). Ecol Modelling. https://doi.org/10.1016/j.ecolmodel.2020.108994

Lu WX, Atkinson DE, Newlands NK (2017) ENSO climate risk: predicting crop yield variability and coherence using cluster-based PCA. Model Earth Syst Environ 3(4):1343–1359

Luan J, Zhang CL, Xu BD, Xue Y, Ren YP (2020) The predictive performances of random forest models with limited sample size and different species traits. Fish Res 227:10

Lucas TCD (2020) A translucent box: interpretable machine learning in ecology. Ecol Monogr 90(4):17

Lucas P, van der Gaag L (1991) Principles of expert systems. Addison-Wesley, Wokingham

Lucash MS, Ruckert KL, Nicholas RE, Scheller RM, Smithwick EAH (2019) Complex interactions among successional trajectories and climate govern spatial resilience after severe windstorms in central Wisconsin, USA. Landsc Ecol 34(12):2897–2915

Ma JS, Perkins S (2003) Time-series novelty detection using one-class support vector machines. In: International joint conference on neural networks, Portland, Or 2003. IEEE international joint conference on neural networks (IJCNN). IEEE, New York, p. 1741–1745

Ma C, Zhang HH, Wang XF (2014) Machine learning for big data analytics in plants. Trends Plant Sci 19(12):798–808

Mac Aodha O, Stathopoulos V, Terry M et al (2014) Putting the Scientist in the Loop - Accelerating Scientific Progress with Interactive Machine Learning. In: 22nd international conference on pattern recognition (ICPR), Swedish Soc Automated Image Anal, Stockholm, SWEDEN 2014. International conference on pattern recognition. Ieee Computer Soc, LOS ALAMITOS, p. 9–17

Magness DR, Huettmann F, Morton JM (2008) Using random forests to provide predicted species distribution maps as a metric for ecological inventory & monitoring programs. In: Smolinski TG, Milanova MG, Hassanien AE (eds), Applications of computational intelligence in biology: current trends and open problems, studies in computational intelligence. pp. 209–229

Mahmud M, Kaiser MS, Hussain A, Vassanelli S (2018) Applications of deep learning and reinforcement learning to biological data. IEEE Trans Neural Netw Learn Syst 29(6):2063–2079

Maldonaldo AD, Ramos-Lopez D, Aguilera PA (2018) A comparison of machine-learning methods to select socioeconomic indicators in cultural landscapes. Sustainability 10(11):16

Maxwell AE, Warner TA, Fang F (2018) Implementation of machine-learning classification in remote sensing: an applied review. Int J Remote Sens 39(9):2784–2817

McGarigal K, Stafford S, Cushman S (2000) Multivariate statistics for wildlife and ecology research. Springer, Cham

McGarigal K, Wan HY, Zeller KA, Timm BC, Cushman SA (2016) Multi-scale habitat selection modeling: a review and outlook. Landsc Ecol 31(6):1161–1175

McLaren JD, Buler JJ, Schreckengost T et al (2018) Artificial light at night confounds broad-scale habitat use by migrating birds. Ecol Lett 21(3):356–364

Minchin PR (1987) An evaluation of the relative robustness of techniques for ecological ordination. Theory and models in vegetation science. Springer, Dordrecht, pp 89–107

Miranda A, Carrasco J, Gonzalez M et al (2020) Evidence-based mapping of the wildland-urban interface to better identify human communities threatened by wildfires. Environ Res Lett 15(9):094069

Mishra M, Nayak J, Naik B, Abraham A (2020b) Deep learning in electrical utility industry: a comprehensive review of a decade of research. Eng Appl Artif Intell 96:32

Mishra AP, Rai ID, Pangtey D, Padalia H (2020) Vegetation characterization at community level using sentinel-2 satellite data and random forest classifier in western Himalayan Foothills, Uttarakhand. J Indian Soc Remote Sens. https://doi.org/10.1007/s12524-020-01253-x

Mitchell TM (1999) Machine learning and data mining. Commun ACM 42(11):30–36

Moriondo M, Jones GV, Bois B et al (2013) Projected shifts of wine regions in response to climate change. Clim Change 119(3–4):825–839

Mountrakis G, Im J, Ogole C (2011) Support vector machines in remote sensing: a review. ISPRS J Photogramm Remote Sens 66(3):247–259

Mullet TC, Gage SH, Morton JM, Huettmann F (2016) Temporal and spatial variation of a winter soundscape in south-central Alaska. Landsc Ecol 31(5):1117–1137

Murphy MA, Evans JS, Storfer A (2010) Quantifying Bufo boreas connectivity in Yellowstone National Park with landscape genetics. Ecology 91:252–261

Naderi JR, Raman B (2005) Capturing impressions of pedestrian landscapes used for healing purposes with decision tree learning. Landsc Urban Plan 73(2–3):155–166

Naveh Z (1994) From biodiversity to ecodiversity: a landscape-ecology approach to conservation and restoration. Restor Ecol 2(3):180–189

Ng ICL, Wakenshaw SYL (2017) The internet-of-things: review and research directions. Int J Res Mark 34(1):3–21

Oksanen, J., Blanchet, F. G., Kindt, R., Legendre, P., Minchin, P. R., O’Hara, R. B., Simpson, G. L., Solymos, P., Henry, M., Stevens, M. H. H., et al. (2016). Vegan: community ecology package. Ordination methods, diversity analysis and other functions for community and vegetation ecologists. R Package Ver, 2.3–3

Olden JD, Lawler JJ, Poff NL (2008) Machine learning methods without tears: a primer for ecologists. Q R Biol 83:171–193

Olsoy PJ, Forbey JS, Shipley LA et al (2020) Mapping foodscapes and sagebrush morphotypes with unmanned aerial systems for multiple herbivores. Landsc Ecol 35(4):921–936

Osborne PE, Alvares-Sanches T (2019) Quantifying how landscape composition and configuration affect urban land surface temperatures using machine learning and neutral landscapes. Comput Environ Urban Syst 76:80–90

Papadimitriou F (2012) Artificial Intelligence in modelling the complexity of Mediterranean landscape transformations. Comput Electron Agric 81:87–96

Pavri F, Farrell D (2020) Arctic landscape transitions: ice cap and terrestrial margins across Hofsjokull, Iceland. Phys Geogr. https://doi.org/10.1080/02723646.2020.1839212

Payntar ND, Hsiao WL, Covey RA, Grauman K (2021) Learning patterns of tourist movement and photography from geotagged photos at archaeological heritage sites in Cuzco Peru. Tour Manag 82:104165

Pazur R, Lieskovsky J, Burgi M et al (2020) Abandonment and recultivation of agricultural lands in Slovakia-patterns and determinants from the past to the future. Land 9(9):316

Pechanec V, Brus J, Sgem (2012) Expert system for landscape assesment based on GIS. In: 12th international multidisciplinary scientific geoconference, SGEM 2012, Vol. III, international multidisciplinary scientific geoconference-SGEM. pp. 369–376

Pechanec V (2013) Evaluation of landscape segments using decision-support systems. In: Fialova J, Kubickova H (eds) Conference on Public Recreation and Landscape Protection - with Man Hand in Hand, pp. 51

Perignon M, Adams J, Overeem I, Passalacqua P (2020) Dominant process zones in a mixed fluvial-tidal delta are morphologically distinct. Earth Surf Dyn 8(3):809–824

Portelli RA (2020) Don’t throw the baby out with the bathwater: reappreciating the dynamic relationship between humans, machines, and landscape images. Landsc Ecol 35(4):815–822

Pourmohammadi P, Adjeroh D, Strager MP (2019) Predicting impervious land expansion using deep deconvolutional neural networks. In: 2019 IEEE international geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 9855–9858

Qian YH, Xing WR, Guan XF, Yang TT, Wu HY (2020) Coupling cellular automata with area partitioning and spatiotemporal convolution for dynamic land use change simulation. Sci Total Environ 722:137738

R Core Team (2017) R: a language and environment for statistical computing. https://www.r-project.org/ Accessed 15 March 2021

Rachmawan IEW, Tadono T, Hayashi M, Kiyoki Y (2018) Temporal difference and density-based learning method applied for deforestation detection using alos-2/Palsar-2. In: IGARSS 2018—2018 IEEE international geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 4905–4908

Rajaraman V (2014) JohnMcCarthy—father of artificial intelligence. Reson-J Sci Educ 19(3):198–207

Rodriguez-Galiano VF, Ghimire B, Rogan J, Chica-Olmo M, Rigol-Sanchez JP (2012) An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS J Photogramm Remote Sens 67:93–104

Rodriguez-Galiano VF, Sanchez-Castillo M, Chica-Olmo M, Chica-Rivas M (2015) Machine learning predictive models for mineral prospectivity: an evaluation of neural networks, random forest, regression trees and support vector machines. Ore Geol Rev 71:804–818

Ross S, Friedman NR, Dudley KL, Yoshimura M, Yoshida T, Economo EP (2018) Listening to ecosystems: data-rich acoustic monitoring through landscape-scale sensor networks. Ecol Res 33(1):135–147

Rotolo D, Hicks D, Martin BR (2015) What is an emerging technology? Res Policy 44(10):1827–1843

Russell S, Norvig P (2020) Artificial intelligence: a modern approach, 4th edn. Pearson, New York

Saeidi S, Mohammadzadeh M, Salmanmahiny A, Mirkarimi SH (2017) Performance evaluation of multiple methods for landscape aesthetic suitability mapping: a comparative study between multi-criteria evaluation, logistic regression and multi-layer perceptron neural network. Land Use Policy 67:1–12

Samarkhanov K, Abuduwaili J, Samat A, Issanova G (2019) The spatial and temporal land cover patterns of the Qazaly irrigation zone in 2003–2018: the case of Syrdarya River’s lower reache, KazakhstanKazakhstan. Sustainability 11(15):4035

Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3(3):210–229

Samuel AL (2000) Some studies in machine learning using the game of checkers (Reprinted from journal of research and development, vol 3, 1959). IBM J Res Dev 44(1–2):207–226

Sejnowski TJ (2018) Deep learning revolution deep learning revolution. MIT Press, Cambridge, pp 1–342

Senior AW, Evans R, Jumper J et al (2020) Improved protein structure prediction using potentials from deep learning. Nature 577(7792):706

Shade C, Kremer P (2019) Predicting land use changes in Philadelphia following green infrastructure policies. Land 8(2):28

Shafizadeh-Moghadam H, Tayyebi A, Ahmadlou M, Delavar MR, Hasanlou M (2017) Integration of genetic algorithm and multiple kernel support vector regression for modeling urban growth. Comput Environ Urban Syst 65:28–40

Shukla G, Garg RD, Srivastava HS, Garg PK (2018) Performance analysis of different predictive models for crop classification across an aridic to ustic area of Indian states. Geocarto Int 33(3):240–259

Shumack S, Hesse P, Farebrother W (2020) Deep learning for dune pattern mapping with the AW3D30 global surface model. Earth Surf Proc Land 45(11):2417–2431

Silva JCF, Teixeira RM, Silva FF, Brommonschenkel SH, Fontes EPB (2019) Machine learning approaches and their current application in plant molecular biology: a systematic review. Plant Sci 284:37–47

Simensen T, Halvorsen R, Erikstad L (2018) Methods for landscape characterisation and mapping: a systematic review. Land Use Policy 75:557–569

Singh M, Tokola T, Hou ZY, Notarnicola C (2017) Remote sensing-based landscape indicators for the evaluation of threatened-bird habitats in a tropical forest. Ecol Evol 7(13):4552–4567

Sommerfeld A, Rammer W, Heurich M, Hilmers T, Muller J, Seidl R (2021) Do bark beetle outbreaks amplify or dampen future bark beetle disturbances in Central Europe? J Ecol 109(2):737–749

Stockwell DRB, Peterson AT (2002) Effects of sample size on accuracy of species distribution models. Ecol Model 148(1):1–13

Storie CD, Henry CJ (2018) Deep learning neural networks for land use land cover mapping. In: IGARSS 2018—2018 IEEE international geoscience and remote sensing symposium, IEEE international symposium on geoscience and remote sensing IGARSS. pp. 3445–3448

Sun FY, Liu M, Wang YC, Wang H, Che Y (2020) The effects of 3D architectural patterns on the urban surface temperature at a neighborhood scale: relative contributions and marginal effects. J Clean Prod 258:120706

Talukdar S, Pal S (2020) Modeling flood plain wetland transformation in consequences of flow alteration in Punarbhaba river in India and Bangladesh. J Clean Prod 261:120767

Tao F, Qi QL, Liu A, Kusiak A (2018) Data-driven smart manufacturing. J Manuf Syst 48:157–169

Taverna K, Urban DL, McDonald RI (2004) Modeling landscape vegetation pattern in response to historic land-use: a hypothesis-driven approach for the North Carolina Piedmont, USA. Landsc Ecol 20:689–702

Thessen A (2016) Adoption of machine learning techniques in ecology and earth science. One Ecosyst 1:e8621

Thomson J, Regan TJ, Hollings T et al (2020) Spatial conservation action planning in heterogeneous landscapes. Biol Conserv 250:108735

Tian FH, Li MY, Han XL, Liu H, Mo BX (2020) A production-living-ecological space model for land-use optimisation: a case study of the core Tumen River region in China. Ecol Model 437:109310

Valletta JJ, Torney C, Kings M, Thornton A, Madden J (2017) Applications of machine learning in animal behaviour studies. Anim Behav 124:203–220

Van Beusekom AE, Gould WA, Monmany AC et al (2018) Fire weather and likelihood: characterizing climate space for fire occurrence and extent in Puerto Rico. Clim Change 146(1–2):117–131

Vanderhaegen S, Canters F (2017) Mapping urban form and function at city block level using spatial metrics. Landsc Urban Plan 167:399–409

Vidal-Macua JJ, Nicolau JM, Vicente E, Moreno-de Heras M (2020) Assessing vegetation recovery in reclaimed opencast mines of the Teruel coalfield (Spain) using Landsat time series and boosted regression trees. Sci Total Environ 717:137250

Wagner B, Baker PJ, Stewart SB et al (2020) Climate change drives habitat contraction of a nocturnal arboreal marsupial at its physiological limits. Ecosphere. https://doi.org/10.1002/ecs2.3262

Wang Q, Ren QF, Liu JF (2016) Identification and apportionment of the drivers of land use change on a regional scale: unbiased recursive partitioning-based stochastic model application. Agric Ecosyst Environ 217:99–110

Werneck FP, Nogueira C, Colli GR, Sites JW, Costa GC (2012) Climatic stability in the Brazilian cerrado: implications for biogeographical connections of South American savannas, species richness and conservation in a biodiversity hotspot. J Biogeogr 39(9):1695–1706

Willcock S, Martinez-Lopez J, Hooftman DAP, Bagstad KJ, Balbi S, Marzo A, Prato C, Sciandrello S, Signorello G, Voigt B, Villa F, Bullock JM, Athanasiadis IN (2018) Machine learning for ecosystem services. Ecosyst Serv 33:165–174

Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1:67–82

Wooldridge M (2020) A brief history of artificial intelligence: what it is, where we are, and where we are going. Macmillan Flatiron Books, New York

Xu SP, Zhao QJ, Yin K, Zhang FF, Liu DB, Yang G (2019) Combining random forest and support vector machines for object-based rural-land-cover classification using high spatial resolution imagery. J Appl Remote Sens 13(1):1

Xu X, Guan MX, Jiang HL, Wang LF (2019) Dynamic simulation of land use change of the upper and middle streams of the Luan River, Northern China. Sustainability 11(18):4909

Yaramasu R, Bandaru V, Pnvr K (2020) Pre-season crop type mapping using deep neural networks. Comput Electron Agric 176:105664

Yin H, Pflugmacher D, Li A, Li ZG, Hostert P (2018) Land use and land cover change in Inner Mongolia—understanding the effects of China’s re-vegetation programs. Remote Sens Environ 204:918–930

Zhang XY, Du SH, Du SJ, Liu B (2020) How do land-use patterns influence residential environment quality? A multiscale geographic survey in Beijing. Remote Sens Environ 249:112014

Zheng MR, Tang WW, Ogundiran A, Yang JX (2020) Spatial simulation modeling of settlement distribution driven by random forest: consideration of landscape visibility. Sustainability 12(11):4748

Zhou L (2018) How to build a better machine learning pipeline. https://www.datanami.com/2018/09/05/how-to-build-a-better-machine-learning-pipeline/ . Accessed 10 October 2021

Zou J, Han Y, So SS (2008) Overview of artificial neural networks. In: Livingstone DJ (ed) Artificial neural networks methods in molecular biology, vol 458. Humana Press, Totowa

Zubair OA, Ji W, Weilert TE (2017) Modeling the impact of urban landscape change on urban wetlands using similarity weighted instance-based machine learning and Markov model. Sustainability 9(12):2223

Download references

Author information

Authors and affiliations.

Faculty of Mathematics and Computer Science, University of Bucharest, Academiei, 14, 010014, Bucharest, Romania

Mihai-Sorin Stupariu

Institute of Research of University of Bucharest, ICUB, Transdisciplinary Research Centre Landscape-Territory-Information Systems, CeLTIS, Splaiul Independentei nr. 91-95, 050095, Bucharest, Romania

Mihai-Sorin Stupariu, Alin-Ionuţ Pleşoianu & Ileana Pătru-Stupariu

USDA, Rocky Mountain Research Station, 2500 S. Pine Knoll Dr, Flagstaff, AZ, 86001, USA

Samuel A. Cushman

Department of Regional Geography and Environment, Faculty of Geography, University of Bucharest, Bd. N. Bălcescu, 1, 010041, Bucharest, Romania

Ileana Pătru-Stupariu

Institute for Geoscience and Geography, Chair for Sustainable Landscape Development, Martin Luther University Halle, Halle, Germany

Christine Fürst

You can also search for this author in PubMed   Google Scholar

Contributions

Idea for the article: M-SS, CF. Conceptualization: M-SS, SAC. Literature search: M-SS, IP-S. Data analysis: SAC, M-SS, A-IP, IP-S. The first draft of the manuscript was written by M-SS and critically revised by SAC. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mihai-Sorin Stupariu .

Ethics declarations

Conflict of interest, ethical approval, consent to participate, consent for publication, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 950 KB)

Rights and permissions.

Reprints and permissions

About this article

Stupariu, MS., Cushman, S.A., Pleşoianu, AI. et al. Machine learning in landscape ecological analysis: a review of recent approaches. Landsc Ecol 37 , 1227–1250 (2022). https://doi.org/10.1007/s10980-021-01366-9

Download citation

Received : 23 March 2021

Accepted : 31 October 2021

Published : 01 December 2021

Issue Date : May 2022

DOI : https://doi.org/10.1007/s10980-021-01366-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Landscape ecology
• Data analysis
• Classification
• Find a journal
• Publish with us
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

Predicting and improving complex beer flavor through machine learning

• Michiel Schreurs   ORCID: orcid.org/0000-0002-9449-5619 1 , 2 , 3   na1 ,
• Supinya Piampongsant 1 , 2 , 3   na1 ,
• Miguel Roncoroni   ORCID: orcid.org/0000-0001-7461-1427 1 , 2 , 3   na1 ,
• Lloyd Cool   ORCID: orcid.org/0000-0001-9936-3124 1 , 2 , 3 , 4 ,
• Beatriz Herrera-Malaver   ORCID: orcid.org/0000-0002-5096-9974 1 , 2 , 3 ,
• Christophe Vanderaa   ORCID: orcid.org/0000-0001-7443-5427 4 ,
• Florian A. Theßeling 1 , 2 , 3 ,
• Łukasz Kreft   ORCID: orcid.org/0000-0001-7620-4657 5 ,
• Alexander Botzki   ORCID: orcid.org/0000-0001-6691-4233 5 ,
• Philippe Malcorps 6 ,
• Luk Daenen 6 ,
• Tom Wenseleers   ORCID: orcid.org/0000-0002-1434-861X 4 &
• Kevin J. Verstrepen   ORCID: orcid.org/0000-0002-3077-6219 1 , 2 , 3  

Nature Communications volume  15 , Article number:  2368 ( 2024 ) Cite this article

52k Accesses

856 Altmetric

Metrics details

• Chemical engineering
• Gas chromatography
• Machine learning
• Metabolomics
• Taste receptors

The perception and appreciation of food flavor depends on many interacting chemical compounds and external factors, and therefore proves challenging to understand and predict. Here, we combine extensive chemical and sensory analyses of 250 different beers to train machine learning models that allow predicting flavor and consumer appreciation. For each beer, we measure over 200 chemical properties, perform quantitative descriptive sensory analysis with a trained tasting panel and map data from over 180,000 consumer reviews to train 10 different machine learning models. The best-performing algorithm, Gradient Boosting, yields models that significantly outperform predictions based on conventional statistics and accurately predict complex food features and consumer appreciation from chemical profiles. Model dissection allows identifying specific and unexpected compounds as drivers of beer flavor and appreciation. Adding these compounds results in variants of commercial alcoholic and non-alcoholic beers with improved consumer appreciation. Together, our study reveals how big data and machine learning uncover complex links between food chemistry, flavor and consumer perception, and lays the foundation to develop novel, tailored foods with superior flavors.

Similar content being viewed by others

BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules

Rudraksh Tuwani, Somin Wadhwa & Ganesh Bagler

Sensory lexicon and aroma volatiles analysis of brewing malt

Xiaoxia Su, Miao Yu, … Tianyi Du

Predicting odor from molecular structure: a multi-label classification approach

Kushagra Saini & Venkatnarayan Ramanathan

Introduction

Predicting and understanding food perception and appreciation is one of the major challenges in food science. Accurate modeling of food flavor and appreciation could yield important opportunities for both producers and consumers, including quality control, product fingerprinting, counterfeit detection, spoilage detection, and the development of new products and product combinations (food pairing) 1 , 2 , 3 , 4 , 5 , 6 . Accurate models for flavor and consumer appreciation would contribute greatly to our scientific understanding of how humans perceive and appreciate flavor. Moreover, accurate predictive models would also facilitate and standardize existing food assessment methods and could supplement or replace assessments by trained and consumer tasting panels, which are variable, expensive and time-consuming 7 , 8 , 9 . Lastly, apart from providing objective, quantitative, accurate and contextual information that can help producers, models can also guide consumers in understanding their personal preferences 10 .

Despite the myriad of applications, predicting food flavor and appreciation from its chemical properties remains a largely elusive goal in sensory science, especially for complex food and beverages 11 , 12 . A key obstacle is the immense number of flavor-active chemicals underlying food flavor. Flavor compounds can vary widely in chemical structure and concentration, making them technically challenging and labor-intensive to quantify, even in the face of innovations in metabolomics, such as non-targeted metabolic fingerprinting 13 , 14 . Moreover, sensory analysis is perhaps even more complicated. Flavor perception is highly complex, resulting from hundreds of different molecules interacting at the physiochemical and sensorial level. Sensory perception is often non-linear, characterized by complex and concentration-dependent synergistic and antagonistic effects 15 , 16 , 17 , 18 , 19 , 20 , 21 that are further convoluted by the genetics, environment, culture and psychology of consumers 22 , 23 , 24 . Perceived flavor is therefore difficult to measure, with problems of sensitivity, accuracy, and reproducibility that can only be resolved by gathering sufficiently large datasets 25 . Trained tasting panels are considered the prime source of quality sensory data, but require meticulous training, are low throughput and high cost. Public databases containing consumer reviews of food products could provide a valuable alternative, especially for studying appreciation scores, which do not require formal training 25 . Public databases offer the advantage of amassing large amounts of data, increasing the statistical power to identify potential drivers of appreciation. However, public datasets suffer from biases, including a bias in the volunteers that contribute to the database, as well as confounding factors such as price, cult status and psychological conformity towards previous ratings of the product.

Classical multivariate statistics and machine learning methods have been used to predict flavor of specific compounds by, for example, linking structural properties of a compound to its potential biological activities or linking concentrations of specific compounds to sensory profiles 1 , 26 . Importantly, most previous studies focused on predicting organoleptic properties of single compounds (often based on their chemical structure) 27 , 28 , 29 , 30 , 31 , 32 , 33 , thus ignoring the fact that these compounds are present in a complex matrix in food or beverages and excluding complex interactions between compounds. Moreover, the classical statistics commonly used in sensory science 34 , 35 , 36 , 37 , 38 , 39 require a large sample size and sufficient variance amongst predictors to create accurate models. They are not fit for studying an extensive set of hundreds of interacting flavor compounds, since they are sensitive to outliers, have a high tendency to overfit and are less suited for non-linear and discontinuous relationships 40 .

In this study, we combine extensive chemical analyses and sensory data of a set of different commercial beers with machine learning approaches to develop models that predict taste, smell, mouthfeel and appreciation from compound concentrations. Beer is particularly suited to model the relationship between chemistry, flavor and appreciation. First, beer is a complex product, consisting of thousands of flavor compounds that partake in complex sensory interactions 41 , 42 , 43 . This chemical diversity arises from the raw materials (malt, yeast, hops, water and spices) and biochemical conversions during the brewing process (kilning, mashing, boiling, fermentation, maturation and aging) 44 , 45 . Second, the advent of the internet saw beer consumers embrace online review platforms, such as RateBeer (ZX Ventures, Anheuser-Busch InBev SA/NV) and BeerAdvocate (Next Glass, inc.). In this way, the beer community provides massive data sets of beer flavor and appreciation scores, creating extraordinarily large sensory databases to complement the analyses of our professional sensory panel. Specifically, we characterize over 200 chemical properties of 250 commercial beers, spread across 22 beer styles, and link these to the descriptive sensory profiling data of a 16-person in-house trained tasting panel and data acquired from over 180,000 public consumer reviews. These unique and extensive datasets enable us to train a suite of machine learning models to predict flavor and appreciation from a beer’s chemical profile. Dissection of the best-performing models allows us to pinpoint specific compounds as potential drivers of beer flavor and appreciation. Follow-up experiments confirm the importance of these compounds and ultimately allow us to significantly improve the flavor and appreciation of selected commercial beers. Together, our study represents a significant step towards understanding complex flavors and reinforces the value of machine learning to develop and refine complex foods. In this way, it represents a stepping stone for further computer-aided food engineering applications 46 .

To generate a comprehensive dataset on beer flavor, we selected 250 commercial Belgian beers across 22 different beer styles (Supplementary Fig.  S1 ). Beers with ≤ 4.2% alcohol by volume (ABV) were classified as non-alcoholic and low-alcoholic. Blonds and Tripels constitute a significant portion of the dataset (12.4% and 11.2%, respectively) reflecting their presence on the Belgian beer market and the heterogeneity of beers within these styles. By contrast, lager beers are less diverse and dominated by a handful of brands. Rare styles such as Brut or Faro make up only a small fraction of the dataset (2% and 1%, respectively) because fewer of these beers are produced and because they are dominated by distinct characteristics in terms of flavor and chemical composition.

Extensive analysis identifies relationships between chemical compounds in beer

For each beer, we measured 226 different chemical properties, including common brewing parameters such as alcohol content, iso-alpha acids, pH, sugar concentration 47 , and over 200 flavor compounds (Methods, Supplementary Table  S1 ). A large portion (37.2%) are terpenoids arising from hopping, responsible for herbal and fruity flavors 16 , 48 . A second major category are yeast metabolites, such as esters and alcohols, that result in fruity and solvent notes 48 , 49 , 50 . Other measured compounds are primarily derived from malt, or other microbes such as non- Saccharomyces yeasts and bacteria (‘wild flora’). Compounds that arise from spices or staling are labeled under ‘Others’. Five attributes (caloric value, total acids and total ester, hop aroma and sulfur compounds) are calculated from multiple individually measured compounds.

As a first step in identifying relationships between chemical properties, we determined correlations between the concentrations of the compounds (Fig.  1 , upper panel, Supplementary Data  1 and 2 , and Supplementary Fig.  S2 . For the sake of clarity, only a subset of the measured compounds is shown in Fig.  1 ). Compounds of the same origin typically show a positive correlation, while absence of correlation hints at parameters varying independently. For example, the hop aroma compounds citronellol, and alpha-terpineol show moderate correlations with each other (Spearman’s rho=0.39 and 0.57), but not with the bittering hop component iso-alpha acids (Spearman’s rho=0.16 and −0.07). This illustrates how brewers can independently modify hop aroma and bitterness by selecting hop varieties and dosage time. If hops are added early in the boiling phase, chemical conversions increase bitterness while aromas evaporate, conversely, late addition of hops preserves aroma but limits bitterness 51 . Similarly, hop-derived iso-alpha acids show a strong anti-correlation with lactic acid and acetic acid, likely reflecting growth inhibition of lactic acid and acetic acid bacteria, or the consequent use of fewer hops in sour beer styles, such as West Flanders ales and Fruit beers, that rely on these bacteria for their distinct flavors 52 . Finally, yeast-derived esters (ethyl acetate, ethyl decanoate, ethyl hexanoate, ethyl octanoate) and alcohols (ethanol, isoamyl alcohol, isobutanol, and glycerol), correlate with Spearman coefficients above 0.5, suggesting that these secondary metabolites are correlated with the yeast genetic background and/or fermentation parameters and may be difficult to influence individually, although the choice of yeast strain may offer some control 53 .

Spearman rank correlations are shown. Descriptors are grouped according to their origin (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)), and sensory aspect (aroma, taste, palate, and overall appreciation). Please note that for the chemical compounds, for the sake of clarity, only a subset of the total number of measured compounds is shown, with an emphasis on the key compounds for each source. For more details, see the main text and Methods section. Chemical data can be found in Supplementary Data  1 , correlations between all chemical compounds are depicted in Supplementary Fig.  S2 and correlation values can be found in Supplementary Data  2 . See Supplementary Data  4 for sensory panel assessments and Supplementary Data  5 for correlation values between all sensory descriptors.

Interestingly, different beer styles show distinct patterns for some flavor compounds (Supplementary Fig.  S3 ). These observations agree with expectations for key beer styles, and serve as a control for our measurements. For instance, Stouts generally show high values for color (darker), while hoppy beers contain elevated levels of iso-alpha acids, compounds associated with bitter hop taste. Acetic and lactic acid are not prevalent in most beers, with notable exceptions such as Kriek, Lambic, Faro, West Flanders ales and Flanders Old Brown, which use acid-producing bacteria ( Lactobacillus and Pediococcus ) or unconventional yeast ( Brettanomyces ) 54 , 55 . Glycerol, ethanol and esters show similar distributions across all beer styles, reflecting their common origin as products of yeast metabolism during fermentation 45 , 53 . Finally, low/no-alcohol beers contain low concentrations of glycerol and esters. This is in line with the production process for most of the low/no-alcohol beers in our dataset, which are produced through limiting fermentation or by stripping away alcohol via evaporation or dialysis, with both methods having the unintended side-effect of reducing the amount of flavor compounds in the final beer 56 , 57 .

Besides expected associations, our data also reveals less trivial associations between beer styles and specific parameters. For example, geraniol and citronellol, two monoterpenoids responsible for citrus, floral and rose flavors and characteristic of Citra hops, are found in relatively high amounts in Christmas, Saison, and Brett/co-fermented beers, where they may originate from terpenoid-rich spices such as coriander seeds instead of hops 58 .

Tasting panel assessments reveal sensorial relationships in beer

To assess the sensory profile of each beer, a trained tasting panel evaluated each of the 250 beers for 50 sensory attributes, including different hop, malt and yeast flavors, off-flavors and spices. Panelists used a tasting sheet (Supplementary Data  3 ) to score the different attributes. Panel consistency was evaluated by repeating 12 samples across different sessions and performing ANOVA. In 95% of cases no significant difference was found across sessions ( p  > 0.05), indicating good panel consistency (Supplementary Table  S2 ).

Aroma and taste perception reported by the trained panel are often linked (Fig.  1 , bottom left panel and Supplementary Data  4 and 5 ), with high correlations between hops aroma and taste (Spearman’s rho=0.83). Bitter taste was found to correlate with hop aroma and taste in general (Spearman’s rho=0.80 and 0.69), and particularly with “grassy” noble hops (Spearman’s rho=0.75). Barnyard flavor, most often associated with sour beers, is identified together with stale hops (Spearman’s rho=0.97) that are used in these beers. Lactic and acetic acid, which often co-occur, are correlated (Spearman’s rho=0.66). Interestingly, sweetness and bitterness are anti-correlated (Spearman’s rho = −0.48), confirming the hypothesis that they mask each other 59 , 60 . Beer body is highly correlated with alcohol (Spearman’s rho = 0.79), and overall appreciation is found to correlate with multiple aspects that describe beer mouthfeel (alcohol, carbonation; Spearman’s rho= 0.32, 0.39), as well as with hop and ester aroma intensity (Spearman’s rho=0.39 and 0.35).

Similar to the chemical analyses, sensorial analyses confirmed typical features of specific beer styles (Supplementary Fig.  S4 ). For example, sour beers (Faro, Flanders Old Brown, Fruit beer, Kriek, Lambic, West Flanders ale) were rated acidic, with flavors of both acetic and lactic acid. Hoppy beers were found to be bitter and showed hop-associated aromas like citrus and tropical fruit. Malt taste is most detected among scotch, stout/porters, and strong ales, while low/no-alcohol beers, which often have a reputation for being ‘worty’ (reminiscent of unfermented, sweet malt extract) appear in the middle. Unsurprisingly, hop aromas are most strongly detected among hoppy beers. Like its chemical counterpart (Supplementary Fig.  S3 ), acidity shows a right-skewed distribution, with the most acidic beers being Krieks, Lambics, and West Flanders ales.

Tasting panel assessments of specific flavors correlate with chemical composition

We find that the concentrations of several chemical compounds strongly correlate with specific aroma or taste, as evaluated by the tasting panel (Fig.  2 , Supplementary Fig.  S5 , Supplementary Data  6 ). In some cases, these correlations confirm expectations and serve as a useful control for data quality. For example, iso-alpha acids, the bittering compounds in hops, strongly correlate with bitterness (Spearman’s rho=0.68), while ethanol and glycerol correlate with tasters’ perceptions of alcohol and body, the mouthfeel sensation of fullness (Spearman’s rho=0.82/0.62 and 0.72/0.57 respectively) and darker color from roasted malts is a good indication of malt perception (Spearman’s rho=0.54).

Heatmap colors indicate Spearman’s Rho. Axes are organized according to sensory categories (aroma, taste, mouthfeel, overall), chemical categories and chemical sources in beer (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)). See Supplementary Data  6 for all correlation values.

Interestingly, for some relationships between chemical compounds and perceived flavor, correlations are weaker than expected. For example, the rose-smelling phenethyl acetate only weakly correlates with floral aroma. This hints at more complex relationships and interactions between compounds and suggests a need for a more complex model than simple correlations. Lastly, we uncovered unexpected correlations. For instance, the esters ethyl decanoate and ethyl octanoate appear to correlate slightly with hop perception and bitterness, possibly due to their fruity flavor. Iron is anti-correlated with hop aromas and bitterness, most likely because it is also anti-correlated with iso-alpha acids. This could be a sign of metal chelation of hop acids 61 , given that our analyses measure unbound hop acids and total iron content, or could result from the higher iron content in dark and Fruit beers, which typically have less hoppy and bitter flavors 62 .

Public consumer reviews complement expert panel data

To complement and expand the sensory data of our trained tasting panel, we collected 180,000 reviews of our 250 beers from the online consumer review platform RateBeer. This provided numerical scores for beer appearance, aroma, taste, palate, overall quality as well as the average overall score.

Public datasets are known to suffer from biases, such as price, cult status and psychological conformity towards previous ratings of a product. For example, prices correlate with appreciation scores for these online consumer reviews (rho=0.49, Supplementary Fig.  S6 ), but not for our trained tasting panel (rho=0.19). This suggests that prices affect consumer appreciation, which has been reported in wine 63 , while blind tastings are unaffected. Moreover, we observe that some beer styles, like lagers and non-alcoholic beers, generally receive lower scores, reflecting that online reviewers are mostly beer aficionados with a preference for specialty beers over lager beers. In general, we find a modest correlation between our trained panel’s overall appreciation score and the online consumer appreciation scores (Fig.  3 , rho=0.29). Apart from the aforementioned biases in the online datasets, serving temperature, sample freshness and surroundings, which are all tightly controlled during the tasting panel sessions, can vary tremendously across online consumers and can further contribute to (among others, appreciation) differences between the two categories of tasters. Importantly, in contrast to the overall appreciation scores, for many sensory aspects the results from the professional panel correlated well with results obtained from RateBeer reviews. Correlations were highest for features that are relatively easy to recognize even for untrained tasters, like bitterness, sweetness, alcohol and malt aroma (Fig.  3 and below).

RateBeer text mining results can be found in Supplementary Data  7 . Rho values shown are Spearman correlation values, with asterisks indicating significant correlations ( p  < 0.05, two-sided). All p values were smaller than 0.001, except for Esters aroma (0.0553), Esters taste (0.3275), Esters aroma—banana (0.0019), Coriander (0.0508) and Diacetyl (0.0134).

Besides collecting consumer appreciation from these online reviews, we developed automated text analysis tools to gather additional data from review texts (Supplementary Data  7 ). Processing review texts on the RateBeer database yielded comparable results to the scores given by the trained panel for many common sensory aspects, including acidity, bitterness, sweetness, alcohol, malt, and hop tastes (Fig.  3 ). This is in line with what would be expected, since these attributes require less training for accurate assessment and are less influenced by environmental factors such as temperature, serving glass and odors in the environment. Consumer reviews also correlate well with our trained panel for 4-vinyl guaiacol, a compound associated with a very characteristic aroma. By contrast, correlations for more specific aromas like ester, coriander or diacetyl are underrepresented in the online reviews, underscoring the importance of using a trained tasting panel and standardized tasting sheets with explicit factors to be scored for evaluating specific aspects of a beer. Taken together, our results suggest that public reviews are trustworthy for some, but not all, flavor features and can complement or substitute taste panel data for these sensory aspects.

Models can predict beer sensory profiles from chemical data

The rich datasets of chemical analyses, tasting panel assessments and public reviews gathered in the first part of this study provided us with a unique opportunity to develop predictive models that link chemical data to sensorial features. Given the complexity of beer flavor, basic statistical tools such as correlations or linear regression may not always be the most suitable for making accurate predictions. Instead, we applied different machine learning models that can model both simple linear and complex interactive relationships. Specifically, we constructed a set of regression models to predict (a) trained panel scores for beer flavor and quality and (b) public reviews’ appreciation scores from beer chemical profiles. We trained and tested 10 different models (Methods), 3 linear regression-based models (simple linear regression with first-order interactions (LR), lasso regression with first-order interactions (Lasso), partial least squares regressor (PLSR)), 5 decision tree models (AdaBoost regressor (ABR), extra trees (ET), gradient boosting regressor (GBR), random forest (RF) and XGBoost regressor (XGBR)), 1 support vector regression (SVR), and 1 artificial neural network (ANN) model.

To compare the performance of our machine learning models, the dataset was randomly split into a training and test set, stratified by beer style. After a model was trained on data in the training set, its performance was evaluated on its ability to predict the test dataset obtained from multi-output models (based on the coefficient of determination, see Methods). Additionally, individual-attribute models were ranked per descriptor and the average rank was calculated, as proposed by Korneva et al. 64 . Importantly, both ways of evaluating the models’ performance agreed in general. Performance of the different models varied (Table  1 ). It should be noted that all models perform better at predicting RateBeer results than results from our trained tasting panel. One reason could be that sensory data is inherently variable, and this variability is averaged out with the large number of public reviews from RateBeer. Additionally, all tree-based models perform better at predicting taste than aroma. Linear models (LR) performed particularly poorly, with negative R 2 values, due to severe overfitting (training set R 2  = 1). Overfitting is a common issue in linear models with many parameters and limited samples, especially with interaction terms further amplifying the number of parameters. L1 regularization (Lasso) successfully overcomes this overfitting, out-competing multiple tree-based models on the RateBeer dataset. Similarly, the dimensionality reduction of PLSR avoids overfitting and improves performance, to some extent. Still, tree-based models (ABR, ET, GBR, RF and XGBR) show the best performance, out-competing the linear models (LR, Lasso, PLSR) commonly used in sensory science 65 .

GBR models showed the best overall performance in predicting sensory responses from chemical information, with R 2 values up to 0.75 depending on the predicted sensory feature (Supplementary Table  S4 ). The GBR models predict consumer appreciation (RateBeer) better than our trained panel’s appreciation (R 2 value of 0.67 compared to R 2 value of 0.09) (Supplementary Table  S3 and Supplementary Table  S4 ). ANN models showed intermediate performance, likely because neural networks typically perform best with larger datasets 66 . The SVR shows intermediate performance, mostly due to the weak predictions of specific attributes that lower the overall performance (Supplementary Table  S4 ).

Model dissection identifies specific, unexpected compounds as drivers of consumer appreciation

Next, we leveraged our models to infer important contributors to sensory perception and consumer appreciation. Consumer preference is a crucial sensory aspects, because a product that shows low consumer appreciation scores often does not succeed commercially 25 . Additionally, the requirement for a large number of representative evaluators makes consumer trials one of the more costly and time-consuming aspects of product development. Hence, a model for predicting chemical drivers of overall appreciation would be a welcome addition to the available toolbox for food development and optimization.

Since GBR models on our RateBeer dataset showed the best overall performance, we focused on these models. Specifically, we used two approaches to identify important contributors. First, rankings of the most important predictors for each sensorial trait in the GBR models were obtained based on impurity-based feature importance (mean decrease in impurity). High-ranked parameters were hypothesized to be either the true causal chemical properties underlying the trait, to correlate with the actual causal properties, or to take part in sensory interactions affecting the trait 67 (Fig.  4A ). In a second approach, we used SHAP 68 to determine which parameters contributed most to the model for making predictions of consumer appreciation (Fig.  4B ). SHAP calculates parameter contributions to model predictions on a per-sample basis, which can be aggregated into an importance score.

A The impurity-based feature importance (mean deviance in impurity, MDI) calculated from the Gradient Boosting Regression (GBR) model predicting RateBeer appreciation scores. The top 15 highest ranked chemical properties are shown. B SHAP summary plot for the top 15 parameters contributing to our GBR model. Each point on the graph represents a sample from our dataset. The color represents the concentration of that parameter, with bluer colors representing low values and redder colors representing higher values. Greater absolute values on the horizontal axis indicate a higher impact of the parameter on the prediction of the model. C Spearman correlations between the 15 most important chemical properties and consumer overall appreciation. Numbers indicate the Spearman Rho correlation coefficient, and the rank of this correlation compared to all other correlations. The top 15 important compounds were determined using SHAP (panel B).

Both approaches identified ethyl acetate as the most predictive parameter for beer appreciation (Fig.  4 ). Ethyl acetate is the most abundant ester in beer with a typical ‘fruity’, ‘solvent’ and ‘alcoholic’ flavor, but is often considered less important than other esters like isoamyl acetate. The second most important parameter identified by SHAP is ethanol, the most abundant beer compound after water. Apart from directly contributing to beer flavor and mouthfeel, ethanol drastically influences the physical properties of beer, dictating how easily volatile compounds escape the beer matrix to contribute to beer aroma 69 . Importantly, it should also be noted that the importance of ethanol for appreciation is likely inflated by the very low appreciation scores of non-alcoholic beers (Supplementary Fig.  S4 ). Despite not often being considered a driver of beer appreciation, protein level also ranks highly in both approaches, possibly due to its effect on mouthfeel and body 70 . Lactic acid, which contributes to the tart taste of sour beers, is the fourth most important parameter identified by SHAP, possibly due to the generally high appreciation of sour beers in our dataset.

Interestingly, some of the most important predictive parameters for our model are not well-established as beer flavors or are even commonly regarded as being negative for beer quality. For example, our models identify methanethiol and ethyl phenyl acetate, an ester commonly linked to beer staling 71 , as a key factor contributing to beer appreciation. Although there is no doubt that high concentrations of these compounds are considered unpleasant, the positive effects of modest concentrations are not yet known 72 , 73 .

To compare our approach to conventional statistics, we evaluated how well the 15 most important SHAP-derived parameters correlate with consumer appreciation (Fig.  4C ). Interestingly, only 6 of the properties derived by SHAP rank amongst the top 15 most correlated parameters. For some chemical compounds, the correlations are so low that they would have likely been considered unimportant. For example, lactic acid, the fourth most important parameter, shows a bimodal distribution for appreciation, with sour beers forming a separate cluster, that is missed entirely by the Spearman correlation. Additionally, the correlation plots reveal outliers, emphasizing the need for robust analysis tools. Together, this highlights the need for alternative models, like the Gradient Boosting model, that better grasp the complexity of (beer) flavor.

Finally, to observe the relationships between these chemical properties and their predicted targets, partial dependence plots were constructed for the six most important predictors of consumer appreciation 74 , 75 , 76 (Supplementary Fig.  S7 ). One-way partial dependence plots show how a change in concentration affects the predicted appreciation. These plots reveal an important limitation of our models: appreciation predictions remain constant at ever-increasing concentrations. This implies that once a threshold concentration is reached, further increasing the concentration does not affect appreciation. This is false, as it is well-documented that certain compounds become unpleasant at high concentrations, including ethyl acetate (‘nail polish’) 77 and methanethiol (‘sulfury’ and ‘rotten cabbage’) 78 . The inability of our models to grasp that flavor compounds have optimal levels, above which they become negative, is a consequence of working with commercial beer brands where (off-)flavors are rarely too high to negatively impact the product. The two-way partial dependence plots show how changing the concentration of two compounds influences predicted appreciation, visualizing their interactions (Supplementary Fig.  S7 ). In our case, the top 5 parameters are dominated by additive or synergistic interactions, with high concentrations for both compounds resulting in the highest predicted appreciation.

To assess the robustness of our best-performing models and model predictions, we performed 100 iterations of the GBR, RF and ET models. In general, all iterations of the models yielded similar performance (Supplementary Fig.  S8 ). Moreover, the main predictors (including the top predictors ethanol and ethyl acetate) remained virtually the same, especially for GBR and RF. For the iterations of the ET model, we did observe more variation in the top predictors, which is likely a consequence of the model’s inherent random architecture in combination with co-correlations between certain predictors. However, even in this case, several of the top predictors (ethanol and ethyl acetate) remain unchanged, although their rank in importance changes (Supplementary Fig.  S8 ).

Next, we investigated if a combination of RateBeer and trained panel data into one consolidated dataset would lead to stronger models, under the hypothesis that such a model would suffer less from bias in the datasets. A GBR model was trained to predict appreciation on the combined dataset. This model underperformed compared to the RateBeer model, both in the native case and when including a dataset identifier (R 2  = 0.67, 0.26 and 0.42 respectively). For the latter, the dataset identifier is the most important feature (Supplementary Fig.  S9 ), while most of the feature importance remains unchanged, with ethyl acetate and ethanol ranking highest, like in the original model trained only on RateBeer data. It seems that the large variation in the panel dataset introduces noise, weakening the models’ performances and reliability. In addition, it seems reasonable to assume that both datasets are fundamentally different, with the panel dataset obtained by blind tastings by a trained professional panel.

Lastly, we evaluated whether beer style identifiers would further enhance the model’s performance. A GBR model was trained with parameters that explicitly encoded the styles of the samples. This did not improve model performance (R2 = 0.66 with style information vs R2 = 0.67). The most important chemical features are consistent with the model trained without style information (eg. ethanol and ethyl acetate), and with the exception of the most preferred (strong ale) and least preferred (low/no-alcohol) styles, none of the styles were among the most important features (Supplementary Fig.  S9 , Supplementary Table  S5 and S6 ). This is likely due to a combination of style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original models, as well as the low number of samples belonging to some styles, making it difficult for the model to learn style-specific patterns. Moreover, beer styles are not rigorously defined, with some styles overlapping in features and some beers being misattributed to a specific style, all of which leads to more noise in models that use style parameters.

Model validation

To test if our predictive models give insight into beer appreciation, we set up experiments aimed at improving existing commercial beers. We specifically selected overall appreciation as the trait to be examined because of its complexity and commercial relevance. Beer flavor comprises a complex bouquet rather than single aromas and tastes 53 . Hence, adding a single compound to the extent that a difference is noticeable may lead to an unbalanced, artificial flavor. Therefore, we evaluated the effect of combinations of compounds. Because Blond beers represent the most extensive style in our dataset, we selected a beer from this style as the starting material for these experiments (Beer 64 in Supplementary Data  1 ).

In the first set of experiments, we adjusted the concentrations of compounds that made up the most important predictors of overall appreciation (ethyl acetate, ethanol, lactic acid, ethyl phenyl acetate) together with correlated compounds (ethyl hexanoate, isoamyl acetate, glycerol), bringing them up to 95 th percentile ethanol-normalized concentrations (Methods) within the Blond group (‘Spiked’ concentration in Fig.  5A ). Compared to controls, the spiked beers were found to have significantly improved overall appreciation among trained panelists, with panelist noting increased intensity of ester flavors, sweetness, alcohol, and body fullness (Fig.  5B ). To disentangle the contribution of ethanol to these results, a second experiment was performed without the addition of ethanol. This resulted in a similar outcome, including increased perception of alcohol and overall appreciation.

Adding the top chemical compounds, identified as best predictors of appreciation by our model, into poorly appreciated beers results in increased appreciation from our trained panel. Results of sensory tests between base beers and those spiked with compounds identified as the best predictors by the model. A Blond and Non/Low-alcohol (0.0% ABV) base beers were brought up to 95th-percentile ethanol-normalized concentrations within each style. B For each sensory attribute, tasters indicated the more intense sample and selected the sample they preferred. The numbers above the bars correspond to the p values that indicate significant changes in perceived flavor (two-sided binomial test: alpha 0.05, n  = 20 or 13).

In a last experiment, we tested whether using the model’s predictions can boost the appreciation of a non-alcoholic beer (beer 223 in Supplementary Data  1 ). Again, the addition of a mixture of predicted compounds (omitting ethanol, in this case) resulted in a significant increase in appreciation, body, ester flavor and sweetness.

Predicting flavor and consumer appreciation from chemical composition is one of the ultimate goals of sensory science. A reliable, systematic and unbiased way to link chemical profiles to flavor and food appreciation would be a significant asset to the food and beverage industry. Such tools would substantially aid in quality control and recipe development, offer an efficient and cost-effective alternative to pilot studies and consumer trials and would ultimately allow food manufacturers to produce superior, tailor-made products that better meet the demands of specific consumer groups more efficiently.

A limited set of studies have previously tried, to varying degrees of success, to predict beer flavor and beer popularity based on (a limited set of) chemical compounds and flavors 79 , 80 . Current sensitive, high-throughput technologies allow measuring an unprecedented number of chemical compounds and properties in a large set of samples, yielding a dataset that can train models that help close the gaps between chemistry and flavor, even for a complex natural product like beer. To our knowledge, no previous research gathered data at this scale (250 samples, 226 chemical parameters, 50 sensory attributes and 5 consumer scores) to disentangle and validate the chemical aspects driving beer preference using various machine-learning techniques. We find that modern machine learning models outperform conventional statistical tools, such as correlations and linear models, and can successfully predict flavor appreciation from chemical composition. This could be attributed to the natural incorporation of interactions and non-linear or discontinuous effects in machine learning models, which are not easily grasped by the linear model architecture. While linear models and partial least squares regression represent the most widespread statistical approaches in sensory science, in part because they allow interpretation 65 , 81 , 82 , modern machine learning methods allow for building better predictive models while preserving the possibility to dissect and exploit the underlying patterns. Of the 10 different models we trained, tree-based models, such as our best performing GBR, showed the best overall performance in predicting sensory responses from chemical information, outcompeting artificial neural networks. This agrees with previous reports for models trained on tabular data 83 . Our results are in line with the findings of Colantonio et al. who also identified the gradient boosting architecture as performing best at predicting appreciation and flavor (of tomatoes and blueberries, in their specific study) 26 . Importantly, besides our larger experimental scale, we were able to directly confirm our models’ predictions in vivo.

Our study confirms that flavor compound concentration does not always correlate with perception, suggesting complex interactions that are often missed by more conventional statistics and simple models. Specifically, we find that tree-based algorithms may perform best in developing models that link complex food chemistry with aroma. Furthermore, we show that massive datasets of untrained consumer reviews provide a valuable source of data, that can complement or even replace trained tasting panels, especially for appreciation and basic flavors, such as sweetness and bitterness. This holds despite biases that are known to occur in such datasets, such as price or conformity bias. Moreover, GBR models predict taste better than aroma. This is likely because taste (e.g. bitterness) often directly relates to the corresponding chemical measurements (e.g., iso-alpha acids), whereas such a link is less clear for aromas, which often result from the interplay between multiple volatile compounds. We also find that our models are best at predicting acidity and alcohol, likely because there is a direct relation between the measured chemical compounds (acids and ethanol) and the corresponding perceived sensorial attribute (acidity and alcohol), and because even untrained consumers are generally able to recognize these flavors and aromas.

The predictions of our final models, trained on review data, hold even for blind tastings with small groups of trained tasters, as demonstrated by our ability to validate specific compounds as drivers of beer flavor and appreciation. Since adding a single compound to the extent of a noticeable difference may result in an unbalanced flavor profile, we specifically tested our identified key drivers as a combination of compounds. While this approach does not allow us to validate if a particular single compound would affect flavor and/or appreciation, our experiments do show that this combination of compounds increases consumer appreciation.

It is important to stress that, while it represents an important step forward, our approach still has several major limitations. A key weakness of the GBR model architecture is that amongst co-correlating variables, the largest main effect is consistently preferred for model building. As a result, co-correlating variables often have artificially low importance scores, both for impurity and SHAP-based methods, like we observed in the comparison to the more randomized Extra Trees models. This implies that chemicals identified as key drivers of a specific sensory feature by GBR might not be the true causative compounds, but rather co-correlate with the actual causative chemical. For example, the high importance of ethyl acetate could be (partially) attributed to the total ester content, ethanol or ethyl hexanoate (rho=0.77, rho=0.72 and rho=0.68), while ethyl phenylacetate could hide the importance of prenyl isobutyrate and ethyl benzoate (rho=0.77 and rho=0.76). Expanding our GBR model to include beer style as a parameter did not yield additional power or insight. This is likely due to style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original model, as well as the smaller sample size per style, limiting the power to uncover style-specific patterns. This can be partly attributed to the curse of dimensionality, where the high number of parameters results in the models mainly incorporating single parameter effects, rather than complex interactions such as style-dependent effects 67 . A larger number of samples may overcome some of these limitations and offer more insight into style-specific effects. On the other hand, beer style is not a rigid scientific classification, and beers within one style often differ a lot, which further complicates the analysis of style as a model factor.

Our study is limited to beers from Belgian breweries. Although these beers cover a large portion of the beer styles available globally, some beer styles and consumer patterns may be missing, while other features might be overrepresented. For example, many Belgian ales exhibit yeast-driven flavor profiles, which is reflected in the chemical drivers of appreciation discovered by this study. In future work, expanding the scope to include diverse markets and beer styles could lead to the identification of even more drivers of appreciation and better models for special niche products that were not present in our beer set.

In addition to inherent limitations of GBR models, there are also some limitations associated with studying food aroma. Even if our chemical analyses measured most of the known aroma compounds, the total number of flavor compounds in complex foods like beer is still larger than the subset we were able to measure in this study. For example, hop-derived thiols, that influence flavor at very low concentrations, are notoriously difficult to measure in a high-throughput experiment. Moreover, consumer perception remains subjective and prone to biases that are difficult to avoid. It is also important to stress that the models are still immature and that more extensive datasets will be crucial for developing more complete models in the future. Besides more samples and parameters, our dataset does not include any demographic information about the tasters. Including such data could lead to better models that grasp external factors like age and culture. Another limitation is that our set of beers consists of high-quality end-products and lacks beers that are unfit for sale, which limits the current model in accurately predicting products that are appreciated very badly. Finally, while models could be readily applied in quality control, their use in sensory science and product development is restrained by their inability to discern causal relationships. Given that the models cannot distinguish compounds that genuinely drive consumer perception from those that merely correlate, validation experiments are essential to identify true causative compounds.

Despite the inherent limitations, dissection of our models enabled us to pinpoint specific molecules as potential drivers of beer aroma and consumer appreciation, including compounds that were unexpected and would not have been identified using standard approaches. Important drivers of beer appreciation uncovered by our models include protein levels, ethyl acetate, ethyl phenyl acetate and lactic acid. Currently, many brewers already use lactic acid to acidify their brewing water and ensure optimal pH for enzymatic activity during the mashing process. Our results suggest that adding lactic acid can also improve beer appreciation, although its individual effect remains to be tested. Interestingly, ethanol appears to be unnecessary to improve beer appreciation, both for blond beer and alcohol-free beer. Given the growing consumer interest in alcohol-free beer, with a predicted annual market growth of >7% 84 , it is relevant for brewers to know what compounds can further increase consumer appreciation of these beers. Hence, our model may readily provide avenues to further improve the flavor and consumer appreciation of both alcoholic and non-alcoholic beers, which is generally considered one of the key challenges for future beer production.

Whereas we see a direct implementation of our results for the development of superior alcohol-free beverages and other food products, our study can also serve as a stepping stone for the development of novel alcohol-containing beverages. We want to echo the growing body of scientific evidence for the negative effects of alcohol consumption, both on the individual level by the mutagenic, teratogenic and carcinogenic effects of ethanol 85 , 86 , as well as the burden on society caused by alcohol abuse and addiction. We encourage the use of our results for the production of healthier, tastier products, including novel and improved beverages with lower alcohol contents. Furthermore, we strongly discourage the use of these technologies to improve the appreciation or addictive properties of harmful substances.

The present work demonstrates that despite some important remaining hurdles, combining the latest developments in chemical analyses, sensory analysis and modern machine learning methods offers exciting avenues for food chemistry and engineering. Soon, these tools may provide solutions in quality control and recipe development, as well as new approaches to sensory science and flavor research.

Beer selection

250 commercial Belgian beers were selected to cover the broad diversity of beer styles and corresponding diversity in chemical composition and aroma. See Supplementary Fig.  S1 .

Chemical dataset

Sample preparation.

Beers within their expiration date were purchased from commercial retailers. Samples were prepared in biological duplicates at room temperature, unless explicitly stated otherwise. Bottle pressure was measured with a manual pressure device (Steinfurth Mess-Systeme GmbH) and used to calculate CO 2 concentration. The beer was poured through two filter papers (Macherey-Nagel, 500713032 MN 713 ¼) to remove carbon dioxide and prevent spontaneous foaming. Samples were then prepared for measurements by targeted Headspace-Gas Chromatography-Flame Ionization Detector/Flame Photometric Detector (HS-GC-FID/FPD), Headspace-Solid Phase Microextraction-Gas Chromatography-Mass Spectrometry (HS-SPME-GC-MS), colorimetric analysis, enzymatic analysis, Near-Infrared (NIR) analysis, as described in the sections below. The mean values of biological duplicates are reported for each compound.

HS-GC-FID/FPD

HS-GC-FID/FPD (Shimadzu GC 2010 Plus) was used to measure higher alcohols, acetaldehyde, esters, 4-vinyl guaicol, and sulfur compounds. Each measurement comprised 5 ml of sample pipetted into a 20 ml glass vial containing 1.75 g NaCl (VWR, 27810.295). 100 µl of 2-heptanol (Sigma-Aldrich, H3003) (internal standard) solution in ethanol (Fisher Chemical, E/0650DF/C17) was added for a final concentration of 2.44 mg/L. Samples were flushed with nitrogen for 10 s, sealed with a silicone septum, stored at −80 °C and analyzed in batches of 20.

The GC was equipped with a DB-WAXetr column (length, 30 m; internal diameter, 0.32 mm; layer thickness, 0.50 µm; Agilent Technologies, Santa Clara, CA, USA) to the FID and an HP-5 column (length, 30 m; internal diameter, 0.25 mm; layer thickness, 0.25 µm; Agilent Technologies, Santa Clara, CA, USA) to the FPD. N 2 was used as the carrier gas. Samples were incubated for 20 min at 70 °C in the headspace autosampler (Flow rate, 35 cm/s; Injection volume, 1000 µL; Injection mode, split; Combi PAL autosampler, CTC analytics, Switzerland). The injector, FID and FPD temperatures were kept at 250 °C. The GC oven temperature was first held at 50 °C for 5 min and then allowed to rise to 80 °C at a rate of 5 °C/min, followed by a second ramp of 4 °C/min until 200 °C kept for 3 min and a final ramp of (4 °C/min) until 230 °C for 1 min. Results were analyzed with the GCSolution software version 2.4 (Shimadzu, Kyoto, Japan). The GC was calibrated with a 5% EtOH solution (VWR International) containing the volatiles under study (Supplementary Table  S7 ).

HS-SPME-GC-MS

HS-SPME-GC-MS (Shimadzu GCMS-QP-2010 Ultra) was used to measure additional volatile compounds, mainly comprising terpenoids and esters. Samples were analyzed by HS-SPME using a triphase DVB/Carboxen/PDMS 50/30 μm SPME fiber (Supelco Co., Bellefonte, PA, USA) followed by gas chromatography (Thermo Fisher Scientific Trace 1300 series, USA) coupled to a mass spectrometer (Thermo Fisher Scientific ISQ series MS) equipped with a TriPlus RSH autosampler. 5 ml of degassed beer sample was placed in 20 ml vials containing 1.75 g NaCl (VWR, 27810.295). 5 µl internal standard mix was added, containing 2-heptanol (1 g/L) (Sigma-Aldrich, H3003), 4-fluorobenzaldehyde (1 g/L) (Sigma-Aldrich, 128376), 2,3-hexanedione (1 g/L) (Sigma-Aldrich, 144169) and guaiacol (1 g/L) (Sigma-Aldrich, W253200) in ethanol (Fisher Chemical, E/0650DF/C17). Each sample was incubated at 60 °C in the autosampler oven with constant agitation. After 5 min equilibration, the SPME fiber was exposed to the sample headspace for 30 min. The compounds trapped on the fiber were thermally desorbed in the injection port of the chromatograph by heating the fiber for 15 min at 270 °C.

The GC-MS was equipped with a low polarity RXi-5Sil MS column (length, 20 m; internal diameter, 0.18 mm; layer thickness, 0.18 µm; Restek, Bellefonte, PA, USA). Injection was performed in splitless mode at 320 °C, a split flow of 9 ml/min, a purge flow of 5 ml/min and an open valve time of 3 min. To obtain a pulsed injection, a programmed gas flow was used whereby the helium gas flow was set at 2.7 mL/min for 0.1 min, followed by a decrease in flow of 20 ml/min to the normal 0.9 mL/min. The temperature was first held at 30 °C for 3 min and then allowed to rise to 80 °C at a rate of 7 °C/min, followed by a second ramp of 2 °C/min till 125 °C and a final ramp of 8 °C/min with a final temperature of 270 °C.

Mass acquisition range was 33 to 550 amu at a scan rate of 5 scans/s. Electron impact ionization energy was 70 eV. The interface and ion source were kept at 275 °C and 250 °C, respectively. A mix of linear n-alkanes (from C7 to C40, Supelco Co.) was injected into the GC-MS under identical conditions to serve as external retention index markers. Identification and quantification of the compounds were performed using an in-house developed R script as described in Goelen et al. and Reher et al. 87 , 88 (for package information, see Supplementary Table  S8 ). Briefly, chromatograms were analyzed using AMDIS (v2.71) 89 to separate overlapping peaks and obtain pure compound spectra. The NIST MS Search software (v2.0 g) in combination with the NIST2017, FFNSC3 and Adams4 libraries were used to manually identify the empirical spectra, taking into account the expected retention time. After background subtraction and correcting for retention time shifts between samples run on different days based on alkane ladders, compound elution profiles were extracted and integrated using a file with 284 target compounds of interest, which were either recovered in our identified AMDIS list of spectra or were known to occur in beer. Compound elution profiles were estimated for every peak in every chromatogram over a time-restricted window using weighted non-negative least square analysis after which peak areas were integrated 87 , 88 . Batch effect correction was performed by normalizing against the most stable internal standard compound, 4-fluorobenzaldehyde. Out of all 284 target compounds that were analyzed, 167 were visually judged to have reliable elution profiles and were used for final analysis.

Discrete photometric and enzymatic analysis

Discrete photometric and enzymatic analysis (Thermo Scientific TM Gallery TM Plus Beermaster Discrete Analyzer) was used to measure acetic acid, ammonia, beta-glucan, iso-alpha acids, color, sugars, glycerol, iron, pH, protein, and sulfite. 2 ml of sample volume was used for the analyses. Information regarding the reagents and standard solutions used for analyses and calibrations is included in Supplementary Table  S7 and Supplementary Table  S9 .

NIR analyses

NIR analysis (Anton Paar Alcolyzer Beer ME System) was used to measure ethanol. Measurements comprised 50 ml of sample, and a 10% EtOH solution was used for calibration.

Correlation calculations

Pairwise Spearman Rank correlations were calculated between all chemical properties.

Sensory dataset

Trained panel.

Our trained tasting panel consisted of volunteers who gave prior verbal informed consent. All compounds used for the validation experiment were of food-grade quality. The tasting sessions were approved by the Social and Societal Ethics Committee of the KU Leuven (G-2022-5677-R2(MAR)). All online reviewers agreed to the Terms and Conditions of the RateBeer website.

Sensory analysis was performed according to the American Society of Brewing Chemists (ASBC) Sensory Analysis Methods 90 . 30 volunteers were screened through a series of triangle tests. The sixteen most sensitive and consistent tasters were retained as taste panel members. The resulting panel was diverse in age [22–42, mean: 29], sex [56% male] and nationality [7 different countries]. The panel developed a consensus vocabulary to describe beer aroma, taste and mouthfeel. Panelists were trained to identify and score 50 different attributes, using a 7-point scale to rate attributes’ intensity. The scoring sheet is included as Supplementary Data  3 . Sensory assessments took place between 10–12 a.m. The beers were served in black-colored glasses. Per session, between 5 and 12 beers of the same style were tasted at 12 °C to 16 °C. Two reference beers were added to each set and indicated as ‘Reference 1 & 2’, allowing panel members to calibrate their ratings. Not all panelists were present at every tasting. Scores were scaled by standard deviation and mean-centered per taster. Values are represented as z-scores and clustered by Euclidean distance. Pairwise Spearman correlations were calculated between taste and aroma sensory attributes. Panel consistency was evaluated by repeating samples on different sessions and performing ANOVA to identify differences, using the ‘stats’ package (v4.2.2) in R (for package information, see Supplementary Table  S8 ).

Online reviews from a public database

The ‘scrapy’ package in Python (v3.6) (for package information, see Supplementary Table  S8 ). was used to collect 232,288 online reviews (mean=922, min=6, max=5343) from RateBeer, an online beer review database. Each review entry comprised 5 numerical scores (appearance, aroma, taste, palate and overall quality) and an optional review text. The total number of reviews per reviewer was collected separately. Numerical scores were scaled and centered per rater, and mean scores were calculated per beer.

For the review texts, the language was estimated using the packages ‘langdetect’ and ‘langid’ in Python. Reviews that were classified as English by both packages were kept. Reviewers with fewer than 100 entries overall were discarded. 181,025 reviews from >6000 reviewers from >40 countries remained. Text processing was done using the ‘nltk’ package in Python. Texts were corrected for slang and misspellings; proper nouns and rare words that are relevant to the beer context were specified and kept as-is (‘Chimay’,’Lambic’, etc.). A dictionary of semantically similar sensorial terms, for example ‘floral’ and ‘flower’, was created and collapsed together into one term. Words were stemmed and lemmatized to avoid identifying words such as ‘acid’ and ‘acidity’ as separate terms. Numbers and punctuation were removed.

Sentences from up to 50 randomly chosen reviews per beer were manually categorized according to the aspect of beer they describe (appearance, aroma, taste, palate, overall quality—not to be confused with the 5 numerical scores described above) or flagged as irrelevant if they contained no useful information. If a beer contained fewer than 50 reviews, all reviews were manually classified. This labeled data set was used to train a model that classified the rest of the sentences for all beers 91 . Sentences describing taste and aroma were extracted, and term frequency–inverse document frequency (TFIDF) was implemented to calculate enrichment scores for sensorial words per beer.

The sex of the tasting subject was not considered when building our sensory database. Instead, results from different panelists were averaged, both for our trained panel (56% male, 44% female) and the RateBeer reviews (70% male, 30% female for RateBeer as a whole).

Beer price collection and processing

Beer prices were collected from the following stores: Colruyt, Delhaize, Total Wine, BeerHawk, The Belgian Beer Shop, The Belgian Shop, and Beer of Belgium. Where applicable, prices were converted to Euros and normalized per liter. Spearman correlations were calculated between these prices and mean overall appreciation scores from RateBeer and the taste panel, respectively.

Pairwise Spearman Rank correlations were calculated between all sensory properties.

Machine learning models

Predictive modeling of sensory profiles from chemical data.

Regression models were constructed to predict (a) trained panel scores for beer flavors and quality from beer chemical profiles and (b) public reviews’ appreciation scores from beer chemical profiles. Z-scores were used to represent sensory attributes in both data sets. Chemical properties with log-normal distributions (Shapiro-Wilk test, p  <  0.05 ) were log-transformed. Missing chemical measurements (0.1% of all data) were replaced with mean values per attribute. Observations from 250 beers were randomly separated into a training set (70%, 175 beers) and a test set (30%, 75 beers), stratified per beer style. Chemical measurements (p = 231) were normalized based on the training set average and standard deviation. In total, three linear regression-based models: linear regression with first-order interaction terms (LR), lasso regression with first-order interaction terms (Lasso) and partial least squares regression (PLSR); five decision tree models, Adaboost regressor (ABR), Extra Trees (ET), Gradient Boosting regressor (GBR), Random Forest (RF) and XGBoost regressor (XGBR); one support vector machine model (SVR) and one artificial neural network model (ANN) were trained. The models were implemented using the ‘scikit-learn’ package (v1.2.2) and ‘xgboost’ package (v1.7.3) in Python (v3.9.16). Models were trained, and hyperparameters optimized, using five-fold cross-validated grid search with the coefficient of determination (R 2 ) as the evaluation metric. The ANN (scikit-learn’s MLPRegressor) was optimized using Bayesian Tree-Structured Parzen Estimator optimization with the ‘Optuna’ Python package (v3.2.0). Individual models were trained per attribute, and a multi-output model was trained on all attributes simultaneously.

Model dissection

GBR was found to outperform other methods, resulting in models with the highest average R 2 values in both trained panel and public review data sets. Impurity-based rankings of the most important predictors for each predicted sensorial trait were obtained using the ‘scikit-learn’ package. To observe the relationships between these chemical properties and their predicted targets, partial dependence plots (PDP) were constructed for the six most important predictors of consumer appreciation 74 , 75 .

The ‘SHAP’ package in Python (v0.41.0) was implemented to provide an alternative ranking of predictor importance and to visualize the predictors’ effects as a function of their concentration 68 .

Validation of causal chemical properties

To validate the effects of the most important model features on predicted sensory attributes, beers were spiked with the chemical compounds identified by the models and descriptive sensory analyses were carried out according to the American Society of Brewing Chemists (ASBC) protocol 90 .

Compound spiking was done 30 min before tasting. Compounds were spiked into fresh beer bottles, that were immediately resealed and inverted three times. Fresh bottles of beer were opened for the same duration, resealed, and inverted thrice, to serve as controls. Pairs of spiked samples and controls were served simultaneously, chilled and in dark glasses as outlined in the Trained panel section above. Tasters were instructed to select the glass with the higher flavor intensity for each attribute (directional difference test 92 ) and to select the glass they prefer.

The final concentration after spiking was equal to the within-style average, after normalizing by ethanol concentration. This was done to ensure balanced flavor profiles in the final spiked beer. The same methods were applied to improve a non-alcoholic beer. Compounds were the following: ethyl acetate (Merck KGaA, W241415), ethyl hexanoate (Merck KGaA, W243906), isoamyl acetate (Merck KGaA, W205508), phenethyl acetate (Merck KGaA, W285706), ethanol (96%, Colruyt), glycerol (Merck KGaA, W252506), lactic acid (Merck KGaA, 261106).

Significant differences in preference or perceived intensity were determined by performing the two-sided binomial test on each attribute.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this work are available in the Supplementary Data files and have been deposited to Zenodo under accession code 10653704 93 . The RateBeer scores data are under restricted access, they are not publicly available as they are property of RateBeer (ZX Ventures, USA). Access can be obtained from the authors upon reasonable request and with permission of RateBeer (ZX Ventures, USA).  Source data are provided with this paper.

Code availability

The code for training the machine learning models, analyzing the models, and generating the figures has been deposited to Zenodo under accession code 10653704 93 .

Tieman, D. et al. A chemical genetic roadmap to improved tomato flavor. Science 355 , 391–394 (2017).

Article   ADS   CAS   PubMed   Google Scholar  

Plutowska, B. & Wardencki, W. Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages – A review. Food Chem. 107 , 449–463 (2008).

Article   CAS   Google Scholar  

Legin, A., Rudnitskaya, A., Seleznev, B. & Vlasov, Y. Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie. Anal. Chim. Acta 534 , 129–135 (2005).

Loutfi, A., Coradeschi, S., Mani, G. K., Shankar, P. & Rayappan, J. B. B. Electronic noses for food quality: A review. J. Food Eng. 144 , 103–111 (2015).

Ahn, Y.-Y., Ahnert, S. E., Bagrow, J. P. & Barabási, A.-L. Flavor network and the principles of food pairing. Sci. Rep. 1 , 196 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bartoshuk, L. M. & Klee, H. J. Better fruits and vegetables through sensory analysis. Curr. Biol. 23 , R374–R378 (2013).

Article   CAS   PubMed   Google Scholar  

Piggott, J. R. Design questions in sensory and consumer science. Food Qual. Prefer. 3293 , 217–220 (1995).

Article   Google Scholar  

Kermit, M. & Lengard, V. Assessing the performance of a sensory panel-panellist monitoring and tracking. J. Chemom. 19 , 154–161 (2005).

Cook, D. J., Hollowood, T. A., Linforth, R. S. T. & Taylor, A. J. Correlating instrumental measurements of texture and flavour release with human perception. Int. J. Food Sci. Technol. 40 , 631–641 (2005).

Chinchanachokchai, S., Thontirawong, P. & Chinchanachokchai, P. A tale of two recommender systems: The moderating role of consumer expertise on artificial intelligence based product recommendations. J. Retail. Consum. Serv. 61 , 1–12 (2021).

Ross, C. F. Sensory science at the human-machine interface. Trends Food Sci. Technol. 20 , 63–72 (2009).

Chambers, E. IV & Koppel, K. Associations of volatile compounds with sensory aroma and flavor: The complex nature of flavor. Molecules 18 , 4887–4905 (2013).

Pinu, F. R. Metabolomics—The new frontier in food safety and quality research. Food Res. Int. 72 , 80–81 (2015).

Danezis, G. P., Tsagkaris, A. S., Brusic, V. & Georgiou, C. A. Food authentication: state of the art and prospects. Curr. Opin. Food Sci. 10 , 22–31 (2016).

Shepherd, G. M. Smell images and the flavour system in the human brain. Nature 444 , 316–321 (2006).

Meilgaard, M. C. Prediction of flavor differences between beers from their chemical composition. J. Agric. Food Chem. 30 , 1009–1017 (1982).

Xu, L. et al. Widespread receptor-driven modulation in peripheral olfactory coding. Science 368 , eaaz5390 (2020).

Kupferschmidt, K. Following the flavor. Science 340 , 808–809 (2013).

Billesbølle, C. B. et al. Structural basis of odorant recognition by a human odorant receptor. Nature 615 , 742–749 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Smith, B. Perspective: Complexities of flavour. Nature 486 , S6–S6 (2012).

Pfister, P. et al. Odorant receptor inhibition is fundamental to odor encoding. Curr. Biol. 30 , 2574–2587 (2020).

Moskowitz, H. W., Kumaraiah, V., Sharma, K. N., Jacobs, H. L. & Sharma, S. D. Cross-cultural differences in simple taste preferences. Science 190 , 1217–1218 (1975).

Eriksson, N. et al. A genetic variant near olfactory receptor genes influences cilantro preference. Flavour 1 , 22 (2012).

Ferdenzi, C. et al. Variability of affective responses to odors: Culture, gender, and olfactory knowledge. Chem. Senses 38 , 175–186 (2013).

Article   PubMed   Google Scholar  

Lawless, H. T. & Heymann, H. Sensory evaluation of food: Principles and practices. (Springer, New York, NY). https://doi.org/10.1007/978-1-4419-6488-5 (2010).

Colantonio, V. et al. Metabolomic selection for enhanced fruit flavor. Proc. Natl. Acad. Sci. 119 , e2115865119 (2022).

Fritz, F., Preissner, R. & Banerjee, P. VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds. Nucleic Acids Res 49 , W679–W684 (2021).

Tuwani, R., Wadhwa, S. & Bagler, G. BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules. Sci. Rep. 9 , 1–13 (2019).

Dagan-Wiener, A. et al. Bitter or not? BitterPredict, a tool for predicting taste from chemical structure. Sci. Rep. 7 , 1–13 (2017).

Pallante, L. et al. Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach. Sci. Rep. 12 , 1–11 (2022).

Malavolta, M. et al. A survey on computational taste predictors. Eur. Food Res. Technol. 248 , 2215–2235 (2022).

Lee, B. K. et al. A principal odor map unifies diverse tasks in olfactory perception. Science 381 , 999–1006 (2023).

Mayhew, E. J. et al. Transport features predict if a molecule is odorous. Proc. Natl. Acad. Sci. 119 , e2116576119 (2022).

Niu, Y. et al. Sensory evaluation of the synergism among ester odorants in light aroma-type liquor by odor threshold, aroma intensity and flash GC electronic nose. Food Res. Int. 113 , 102–114 (2018).

Yu, P., Low, M. Y. & Zhou, W. Design of experiments and regression modelling in food flavour and sensory analysis: A review. Trends Food Sci. Technol. 71 , 202–215 (2018).

Oladokun, O. et al. The impact of hop bitter acid and polyphenol profiles on the perceived bitterness of beer. Food Chem. 205 , 212–220 (2016).

Linforth, R., Cabannes, M., Hewson, L., Yang, N. & Taylor, A. Effect of fat content on flavor delivery during consumption: An in vivo model. J. Agric. Food Chem. 58 , 6905–6911 (2010).

Guo, S., Na Jom, K. & Ge, Y. Influence of roasting condition on flavor profile of sunflower seeds: A flavoromics approach. Sci. Rep. 9 , 11295 (2019).

Ren, Q. et al. The changes of microbial community and flavor compound in the fermentation process of Chinese rice wine using Fagopyrum tataricum grain as feedstock. Sci. Rep. 9 , 3365 (2019).

Hastie, T., Friedman, J. & Tibshirani, R. The Elements of Statistical Learning. (Springer, New York, NY). https://doi.org/10.1007/978-0-387-21606-5 (2001).

Dietz, C., Cook, D., Huismann, M., Wilson, C. & Ford, R. The multisensory perception of hop essential oil: a review. J. Inst. Brew. 126 , 320–342 (2020).

CAS   Google Scholar  

Roncoroni, Miguel & Verstrepen, Kevin Joan. Belgian Beer: Tested and Tasted. (Lannoo, 2018).

Meilgaard, M. Flavor chemistry of beer: Part II: Flavor and threshold of 239 aroma volatiles. in (1975).

Bokulich, N. A. & Bamforth, C. W. The microbiology of malting and brewing. Microbiol. Mol. Biol. Rev. MMBR 77 , 157–172 (2013).

Dzialo, M. C., Park, R., Steensels, J., Lievens, B. & Verstrepen, K. J. Physiology, ecology and industrial applications of aroma formation in yeast. FEMS Microbiol. Rev. 41 , S95–S128 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Datta, A. et al. Computer-aided food engineering. Nat. Food 3 , 894–904 (2022).

American Society of Brewing Chemists. Beer Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A.).

Olaniran, A. O., Hiralal, L., Mokoena, M. P. & Pillay, B. Flavour-active volatile compounds in beer: production, regulation and control. J. Inst. Brew. 123 , 13–23 (2017).

Verstrepen, K. J. et al. Flavor-active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Meilgaard, M. C. Flavour chemistry of beer. part I: flavour interaction between principal volatiles. Master Brew. Assoc. Am. Tech. Q 12 , 107–117 (1975).

Briggs, D. E., Boulton, C. A., Brookes, P. A. & Stevens, R. Brewing 227–254. (Woodhead Publishing). https://doi.org/10.1533/9781855739062.227 (2004).

Bossaert, S., Crauwels, S., De Rouck, G. & Lievens, B. The power of sour - A review: Old traditions, new opportunities. BrewingScience 72 , 78–88 (2019).

Google Scholar  

Verstrepen, K. J. et al. Flavor active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Snauwaert, I. et al. Microbial diversity and metabolite composition of Belgian red-brown acidic ales. Int. J. Food Microbiol. 221 , 1–11 (2016).

Spitaels, F. et al. The microbial diversity of traditional spontaneously fermented lambic beer. PLoS ONE 9 , e95384 (2014).

Blanco, C. A., Andrés-Iglesias, C. & Montero, O. Low-alcohol Beers: Flavor Compounds, Defects, and Improvement Strategies. Crit. Rev. Food Sci. Nutr. 56 , 1379–1388 (2016).

Jackowski, M. & Trusek, A. Non-Alcohol. beer Prod. – Overv. 20 , 32–38 (2018).

Takoi, K. et al. The contribution of geraniol metabolism to the citrus flavour of beer: Synergy of geraniol and β-citronellol under coexistence with excess linalool. J. Inst. Brew. 116 , 251–260 (2010).

Kroeze, J. H. & Bartoshuk, L. M. Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiol. Behav. 35 , 779–783 (1985).

Mennella, J. A. et al. A spoonful of sugar helps the medicine go down”: Bitter masking bysucrose among children and adults. Chem. Senses 40 , 17–25 (2015).

Wietstock, P., Kunz, T., Perreira, F. & Methner, F.-J. Metal chelation behavior of hop acids in buffered model systems. BrewingScience 69 , 56–63 (2016).

Sancho, D., Blanco, C. A., Caballero, I. & Pascual, A. Free iron in pale, dark and alcohol-free commercial lager beers. J. Sci. Food Agric. 91 , 1142–1147 (2011).

Rodrigues, H. & Parr, W. V. Contribution of cross-cultural studies to understanding wine appreciation: A review. Food Res. Int. 115 , 251–258 (2019).

Korneva, E. & Blockeel, H. Towards better evaluation of multi-target regression models. in ECML PKDD 2020 Workshops (eds. Koprinska, I. et al.) 353–362 (Springer International Publishing, Cham, 2020). https://doi.org/10.1007/978-3-030-65965-3_23 .

Gastón Ares. Mathematical and Statistical Methods in Food Science and Technology. (Wiley, 2013).

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on tabular data? Preprint at http://arxiv.org/abs/2207.08815 (2022).

Gries, S. T. Statistics for Linguistics with R: A Practical Introduction. in Statistics for Linguistics with R (De Gruyter Mouton, 2021). https://doi.org/10.1515/9783110718256 .

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Ickes, C. M. & Cadwallader, K. R. Effects of ethanol on flavor perception in alcoholic beverages. Chemosens. Percept. 10 , 119–134 (2017).

Kato, M. et al. Influence of high molecular weight polypeptides on the mouthfeel of commercial beer. J. Inst. Brew. 127 , 27–40 (2021).

Wauters, R. et al. Novel Saccharomyces cerevisiae variants slow down the accumulation of staling aldehydes and improve beer shelf-life. Food Chem. 398 , 1–11 (2023).

Li, H., Jia, S. & Zhang, W. Rapid determination of low-level sulfur compounds in beer by headspace gas chromatography with a pulsed flame photometric detector. J. Am. Soc. Brew. Chem. 66 , 188–191 (2008).

Dercksen, A., Laurens, J., Torline, P., Axcell, B. C. & Rohwer, E. Quantitative analysis of volatile sulfur compounds in beer using a membrane extraction interface. J. Am. Soc. Brew. Chem. 54 , 228–233 (1996).

Molnar, C. Interpretable Machine Learning: A Guide for Making Black-Box Models Interpretable. (2020).

Zhao, Q. & Hastie, T. Causal interpretations of black-box models. J. Bus. Econ. Stat. Publ. Am. Stat. Assoc. 39 , 272–281 (2019).

Article   MathSciNet   Google Scholar  

Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning. (Springer, 2019).

Labrado, D. et al. Identification by NMR of key compounds present in beer distillates and residual phases after dealcoholization by vacuum distillation. J. Sci. Food Agric. 100 , 3971–3978 (2020).

Lusk, L. T., Kay, S. B., Porubcan, A. & Ryder, D. S. Key olfactory cues for beer oxidation. J. Am. Soc. Brew. Chem. 70 , 257–261 (2012).

Gonzalez Viejo, C., Torrico, D. D., Dunshea, F. R. & Fuentes, S. Development of artificial neural network models to assess beer acceptability based on sensory properties using a robotic pourer: A comparative model approach to achieve an artificial intelligence system. Beverages 5 , 33 (2019).

Gonzalez Viejo, C., Fuentes, S., Torrico, D. D., Godbole, A. & Dunshea, F. R. Chemical characterization of aromas in beer and their effect on consumers liking. Food Chem. 293 , 479–485 (2019).

Gilbert, J. L. et al. Identifying breeding priorities for blueberry flavor using biochemical, sensory, and genotype by environment analyses. PLOS ONE 10 , 1–21 (2015).

Goulet, C. et al. Role of an esterase in flavor volatile variation within the tomato clade. Proc. Natl. Acad. Sci. 109 , 19009–19014 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Borisov, V. et al. Deep Neural Networks and Tabular Data: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 1–21 https://doi.org/10.1109/TNNLS.2022.3229161 (2022).

Statista. Statista Consumer Market Outlook: Beer - Worldwide.

Seitz, H. K. & Stickel, F. Molecular mechanisms of alcoholmediated carcinogenesis. Nat. Rev. Cancer 7 , 599–612 (2007).

Voordeckers, K. et al. Ethanol exposure increases mutation rate through error-prone polymerases. Nat. Commun. 11 , 3664 (2020).

Goelen, T. et al. Bacterial phylogeny predicts volatile organic compound composition and olfactory response of an aphid parasitoid. Oikos 129 , 1415–1428 (2020).

Article   ADS   Google Scholar  

Reher, T. et al. Evaluation of hop (Humulus lupulus) as a repellent for the management of Drosophila suzukii. Crop Prot. 124 , 104839 (2019).

Stein, S. E. An integrated method for spectrum extraction and compound identification from gas chromatography/mass spectrometry data. J. Am. Soc. Mass Spectrom. 10 , 770–781 (1999).

American Society of Brewing Chemists. Sensory Analysis Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A., 1992).

McAuley, J., Leskovec, J. & Jurafsky, D. Learning Attitudes and Attributes from Multi-Aspect Reviews. Preprint at https://doi.org/10.48550/arXiv.1210.3926 (2012).

Meilgaard, M. C., Carr, B. T. & Carr, B. T. Sensory Evaluation Techniques. (CRC Press, Boca Raton). https://doi.org/10.1201/b16452 (2014).

Schreurs, M. et al. Data from: Predicting and improving complex beer flavor through machine learning. Zenodo https://doi.org/10.5281/zenodo.10653704 (2024).

Download references

Acknowledgements

We thank all lab members for their discussions and thank all tasting panel members for their contributions. Special thanks go out to Dr. Karin Voordeckers for her tremendous help in proofreading and improving the manuscript. M.S. was supported by a Baillet-Latour fellowship, L.C. acknowledges financial support from KU Leuven (C16/17/006), F.A.T. was supported by a PhD fellowship from FWO (1S08821N). Research in the lab of K.J.V. is supported by KU Leuven, FWO, VIB, VLAIO and the Brewing Science Serves Health Fund. Research in the lab of T.W. is supported by FWO (G.0A51.15) and KU Leuven (C16/17/006).

Author information

These authors contributed equally: Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni.

Authors and Affiliations

VIB—KU Leuven Center for Microbiology, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni, Lloyd Cool, Beatriz Herrera-Malaver, Florian A. Theßeling & Kevin J. Verstrepen

CMPG Laboratory of Genetics and Genomics, KU Leuven, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Leuven Institute for Beer Research (LIBR), Gaston Geenslaan 1, B-3001, Leuven, Belgium

Laboratory of Socioecology and Social Evolution, KU Leuven, Naamsestraat 59, B-3000, Leuven, Belgium

Lloyd Cool, Christophe Vanderaa & Tom Wenseleers

VIB Bioinformatics Core, VIB, Rijvisschestraat 120, B-9052, Ghent, Belgium

Łukasz Kreft & Alexander Botzki

AB InBev SA/NV, Brouwerijplein 1, B-3000, Leuven, Belgium

Philippe Malcorps & Luk Daenen

You can also search for this author in PubMed   Google Scholar

Contributions

S.P., M.S. and K.J.V. conceived the experiments. S.P., M.S. and K.J.V. designed the experiments. S.P., M.S., M.R., B.H. and F.A.T. performed the experiments. S.P., M.S., L.C., C.V., L.K., A.B., P.M., L.D., T.W. and K.J.V. contributed analysis ideas. S.P., M.S., L.C., C.V., T.W. and K.J.V. analyzed the data. All authors contributed to writing the manuscript.

Corresponding author

Correspondence to Kevin J. Verstrepen .

Ethics declarations

Competing interests.

K.J.V. is affiliated with bar.on. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Florian Bauer, Andrew John Macintosh and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, supplementary data 2, supplementary data 3, supplementary data 4, supplementary data 5, supplementary data 6, supplementary data 7, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schreurs, M., Piampongsant, S., Roncoroni, M. et al. Predicting and improving complex beer flavor through machine learning. Nat Commun 15 , 2368 (2024). https://doi.org/10.1038/s41467-024-46346-0

Download citation

Received : 30 October 2023

Accepted : 21 February 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41467-024-46346-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Read our research on: Gun Policy | International Conflict | Election 2024

Regions & Countries

What public k-12 teachers want americans to know about teaching.

Illustrations by Hokyoung Kim

At a time when most teachers are feeling stressed and overwhelmed in their jobs, we asked 2,531 public K-12 teachers this open-ended question:

If there’s one thing you’d want the public to know about teachers, what would it be?

We also asked Americans what they think about teachers to compare with teachers’ perceptions of how the public views them.

Related: What’s It Like To Be a Teacher in America Today?

Pew Research Center conducted this analysis to better understand what public K-12 teachers would like Americans to know about their profession. We also wanted to learn how the public thinks about teachers.

For the open-end question, we surveyed 2,531 U.S. public K-12 teachers from Oct. 17 to Nov. 14, 2023. The teachers surveyed are members of RAND’s American Teacher Panel, a nationally representative panel of public K-12 school teachers recruited through MDR Education. Survey data is weighted to state and national teacher characteristics to account for differences in sampling and response to ensure they are representative of the target population.

Overall, 96% of surveyed teachers provided an answer to the open-ended question. Center researchers developed a coding scheme categorizing the responses, coded all responses, and then grouped them into the six themes explored in the data essay.

For the questions for the general public, we surveyed 5,029 U.S. adults from Nov. 9 to Nov. 16, 2023. The adults surveyed are members of the Ipsos KnowledgePanel, a nationally representative online survey panel. Panel members are randomly recruited through probability-based sampling, and households are provided with access to the Internet and hardware if needed. To ensure that the results of this survey reflect a balanced cross section of the nation, the data is weighted to match the U.S. adult population by gender, age, education, race and ethnicity and other categories.

Here are the questions used for this analysis , along with responses, the teacher survey methodology and the general public survey methodology .

Most of the responses to the open-ended question fell into one of these six themes:

Teaching is a hard job

About half of teachers (51%) said they want the public to know that teaching is a difficult job and that teachers are hardworking. Within this share, many mentioned that they have roles and responsibilities in the classroom besides teaching, which makes the job stressful. Many also talked about working long hours, beyond those they’re contracted for.

“Teachers serve multiple roles other than being responsible for teaching curriculum. We are counselors, behavioral specialists and parents for students who need us to fill those roles. We sacrifice a lot to give all of ourselves to the role as teacher.”

– Elementary school teacher

“The amount of extra hours that teachers have to put in beyond the contractual time is ridiculous. Arriving 30 minutes before and leaving an hour after is just the tip of the iceberg. … And as far as ‘having summers off,’ most of August is taken up with preparing materials for the upcoming school year or attending three, four, seven days’ worth of unpaid development training.”

– High school teacher

Teachers care about their students

The next most common theme: 22% of teachers brought up how fulfilling teaching is and how much teachers care about their students. Many gave examples of the hardships of teaching but reaffirmed that they do their job because they love the kids and helping them succeed. 

“We are passionate about what we do. Every child we teach is important to us and we look out for them like they are our own.”

– Middle school teacher

“We are in it for the kids, and the most incredible moments are when children make connections with learning.”

Teachers are undervalued and disrespected

Some 17% of teachers want the public to know that they feel undervalued and disrespected, and that they need more public support. Some mentioned that they are well-educated professionals but are not treated as such. And many teachers in this category responded with a general plea for support from the public, which they don’t feel they’re getting now.

“We feel undervalued. The public and many parents of my students treat me and my peers as if we do not know as much as they do, as if we are uneducated.”

“The public attitudes toward teachers have been degrading, and it is making it impossible for well-qualified teachers to be found. People are simply not wanting to go into the profession because of public sentiments.”

Teachers are underpaid

A similar share of teachers (15%) want the public to know that teachers are underpaid. Many teachers said their salary doesn’t account for the effort and care they put into their students’ education and believe that their pay should reflect this.

“We are sorely underpaid for the amount of hours we work and the education level we have attained.”

Teachers need support and resources from government and administrators

About one-in-ten teachers (9%) said they need more support from the government, their administrators and other key stakeholders. Many mentioned working in understaffed schools, not having enough funding and paying for supplies out of pocket. Some teachers also expressed that they have little control over the curriculum that they teach.

“The world-class education we used to be proud of does not exist because of all the red tape we are constantly navigating. If you want to see real change in the classroom, advocate for smaller class sizes for your child, push your district to cap class sizes at a reasonable level and have real, authentic conversations with your child’s teacher about what is going on in the classroom if you’re curious.”

Teachers need more support from parents

Roughly the same share of teachers (8%) want the public to know that teachers need more support from parents, emphasizing that the parent-teacher relationship is strained. Many view parents as partners in their child’s education and believe that a strong relationship improves kids’ overall social and emotional development.

“Teachers help students to reach their potential. However, that job is near impossible if parents/guardians do not take an active part in their student’s education.”

How the U.S. public views teachers

While the top response from teachers in the open-ended question is that they want the public to know that teaching is a hard job, most Americans already see it that way. Two-thirds of U.S. adults say being a public K-12 teacher is harder than most other jobs, with 33% saying it’s a lot harder.

And about three-quarters of Americans (74%) say teachers should be paid more than they are now, including 39% who say teachers should be paid a lot more.

Americans are about evenly divided on whether the public generally looks up to (32%) or down on (30%) public K-12 teachers. Some 37% say Americans neither look up to or down on public K-12 teachers.

In addition to the open-ended question about what they want the public to know about them, we asked teachers how much they think most Americans trust public K-12 teachers to do their job well. We also asked the public how much they trust teachers. Answers differ considerably.

Nearly half of public K-12 teachers (47%) say most Americans don’t trust teachers much or at all. A third say most Americans trust teachers some, and 18% say the public trusts teachers a great deal or a fair amount.

In contrast, a majority of Americans (57%) say they do trust public K-12 teachers to do their job well a great deal or a fair amount. About a quarter (26%) say they trust teachers some, and 17% say they don’t trust teachers much or at all.

Related: About half of Americans say public K-12 education is going in the wrong direction

How the public’s views differ by party

There are sizable party differences in Americans’ views of teachers. In particular, Democrats and Democratic-leaning independents are more likely than Republicans and Republican leaners to say:

• They trust teachers to do their job well a great deal or a fair amount (70% vs. 44%)
• Teaching is a lot or somewhat harder when compared with most other jobs (77% vs. 59%)
• Teachers should be paid a lot or somewhat more than they are now (86% vs. 63%)

In their own words

Below, we have a selection of quotes that describe what teachers want the public to know about them and their profession.

Social Trends Monthly Newsletter

Sign up to to receive a monthly digest of the Center's latest research on the attitudes and behaviors of Americans in key realms of daily life

About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts .

Help | Advanced Search

Computer Science > Machine Learning

Title: pissa: principal singular values and singular vectors adaptation of large language models.

Abstract: As the parameters of LLMs expand, the computational cost of fine-tuning the entire model becomes prohibitive. To address this challenge, we introduce a PEFT method, Principal Singular values and Singular vectors Adaptation (PiSSA), which optimizes a significantly reduced parameter space while achieving or surpassing the performance of full-parameter fine-tuning. PiSSA is inspired by Intrinsic SAID, which suggests that pre-trained, over-parametrized models inhabit a space of low intrinsic dimension. Consequently, PiSSA represents a matrix W within the model by the product of two trainable matrices A and B, plus a residual matrix $W^{res}$ for error correction. SVD is employed to factorize W, and the principal singular values and vectors of W are utilized to initialize A and B. The residual singular values and vectors initialize the residual matrix $W^{res}$, which keeps frozen during fine-tuning. Notably, PiSSA shares the same architecture with LoRA. However, LoRA approximates Delta W through the product of two matrices, A, initialized with Gaussian noise, and B, initialized with zeros, while PiSSA initializes A and B with principal singular values and vectors of the original matrix W. PiSSA can better approximate the outcomes of full-parameter fine-tuning at the beginning by changing the essential parts while freezing the "noisy" parts. In comparison, LoRA freezes the original matrix and updates the "noise". This distinction enables PiSSA to convergence much faster than LoRA and also achieve better performance in the end. Due to the same architecture, PiSSA inherits many of LoRA's advantages, such as parameter efficiency and compatibility with quantization. Leveraging a fast SVD method, the initialization of PiSSA takes only a few seconds, inducing negligible cost of switching LoRA to PiSSA.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .