clinical critical thinking skills test

Validation of a clinical critical thinking skills test in nursing

Affiliations.

• 1 Department of Nursing, College of Medicine, Soonchunhyang University, Cheonan, Korea.
• 2 Division of Nursing Science, College of Health Science, Ewha Woman's University, Seoul, Korea.
• PMID: 25622716
• PMCID: PMC4352692
• DOI: 10.3352/jeehp.2015.12.1

Purpose: The purpose of this study was to develop a revised version of the clinical critical thinking skills test (CCTS) and to subsequently validate its performance.

Methods: This study is a secondary analysis of the CCTS. Data were obtained from a convenience sample of 284 college students in June 2011. Thirty items were analyzed using item response theory and test reliability was assessed. Test-retest reliability was measured using the results of 20 nursing college and graduate school students in July 2013. The content validity of the revised items was analyzed by calculating the degree of agreement between instrument developer intention in item development and the judgments of six experts. To analyze response process validity, qualitative data related to the response processes of nine nursing college students obtained through cognitive interviews were analyzed.

Results: Out of initial 30 items, 11 items were excluded after the analysis of difficulty and discrimination parameter. When the 19 items of the revised version of the CCTS were analyzed, levels of item difficulty were found to be relatively low and levels of discrimination were found to be appropriate or high. The degree of agreement between item developer intention and expert judgments equaled or exceeded 50%.

Conclusion: From above results, evidence of the response process validity was demonstrated, indicating that subjects respondeds as intended by the test developer. The revised 19-item CCTS was found to have sufficient reliability and validity and will therefore represents a more convenient measurement of critical thinking ability.

Keywords: Intention; Judgment; Nursing students; Reproducibility of results; Thinking.

California Critical Thinking Skills Test

Get data for Admissions Evaluating Critical Thinking Skills   Student Success Advising National Benchmarking Accreditation Educational Research

The  California Critical Thinking Skills Test (CCTST)  is an educational assessment that measures all the core reasoning skills needed for reflective decision-making. The CCTST provides valid and reliable data on critical thinking skills of individuals and of groups.  It is designed for use with undergraduate and graduate students. It is available in many languages and its OVERALL skills score can be benchmarked using one of many percentile comparisons. Clients most commonly use the  CCTST for admissions, advising and retention, studies of curriculum effectiveness, accreditation, and the documentation of student learning outcomes.

For assessment specs, administration, metrics reported, and more, scroll down. Contact us by using the “Request A Quote” button to ask a question. Or phone us at 650-697-5628 to speak with an assessment services client support specialist.

Seamless Testing. Results You Can Trust.

Higher education.

CCTST is calibrated for undergraduate and graduate level college students across the full spectrum of disciplines and fields of study.

Administration

Administered online with a secure, multi-lingual interface, it’s user-friendly and accessible anywhere.

Support Materials

User Manual includes all needed information about administering the assessment and interpreting the resulting individual and group scores.

Assessment Specs

55 minutes timed administration; 40 engaging, scenario-based questions

Deliverables

Group graphics with statistical summary of scores; Excel spreadsheet of responses to all custom demographic questions, and all scores for each person tested. Optional individual score reports for administrators and/or test takers.

Results Reported

Metrics include scores for 8 critical skills, plus an OVERALL rating. Population percentile scores are available for benchmarking.

All of the CCTST metrics are on a 100-point scale with a corresponding qualitative rating (Superior, Strong, Moderate, Weak, Not Manifested).

Available in English, Arabic, Chinese Simplified, Chinese Traditional, Dutch, French, German, Indonesian-Bahasa, Italian, Japanese, Korean, Norwegian, Portuguese, Spanish, Swedish, Thai, Turkish, and Vietnamese languages.

CCTST provides 8  cognitive skill scores to focus future development and training. Items are drawn from a scientifically developed and tested item pool.

• OVERALL Critical Thinking Skills – Sustained use of critical thinking to form reasoned judgments
• Analysis  – Accurate identification of the problem and decision-critical elements
• Interpretation  – Discovering and determining significance and contextual meaning
• Inference –  Drawing warranted and logical conclusions from reasons and evidence
• Evaluation  – Assessing credibility of claims and the strength of arguments
• Explanation  – Providing the evidence, reasons, assumptions, or rationale for judgments and decisions
• Induction  – Reasoned judgment in ambiguous, risky, and uncertain contexts
• Deduction  – Reasoned judgment in precisely defined, logically rigorous contexts
• Numeracy  – Sustained use of critical thinking skills in quantitative contexts (quantitative reasoning)

The  California Critical Thinking Skills Test (CCTST)  Report Package includes an individual test-taker report for each person assessed and group summary reports for each group and sub-group in the sample.

Reports are generated immediately after the conclusion of testing and are available for clients to download making real time assessment possible. Read more about how our  customer support specialists work with clients to select their reporting options on our Services tab or contact us for a consultation.

Group Analytics

• Clients can generate and download Excel spreadsheet files of all scores (OVERALL, Percentile ranking and all cognitive score metrics). At the option of the client, these also include the responses to custom demographic questions added by the client to the assessment profile, and percentile scores corresponding to the external comparison group selected by the client.
• Presentation-ready tables and graphic representations of the score distribution for OVERALL critical thinking skills and for the additional cognitive skill metrics.
• Customers who have added custom demographic questions can generate sub-group reports for these variables, or for specific testing sessions or time periods.

Optional Individual Test-Taker Reports

• An overall score of critical thinking skills (OVERALL Score). OVERALL is reported on a 100-point scale accompanied by a qualitative rating (Superior, Strong, Moderate, Weak, Not Manifested), and a comparison percentile score.
• Scores for each cognitive skill metric. These metrics are scored on a 100-point scale and are accompanied by a categorical interpretation of the strength of the score indicating areas of strength and areas for future development.
• The Individual Test Taker Report can be pushed to an email address of the client’s choosing (for example, to an admissions office email, institutional assessment email, dean’s office email, etc.).
• The client controls whether individual reports are made available to the test-taker.

Need to expedite your project?  We can have your first online testing assignment available for your students within 24 hours.  Request a Quote or get started by calling 650-697-5628 and speaking with one of our assessment specialists today.

Unlock your exclusive access to our resource library, training tools.

 Understand the depth of our metrics with hands-on tools designed to elucidate the reasoning behind our results. These tools empower you to interpret and apply our data in your professional journey.

Published Articles

Explore how our products are used in real-world scenarios, with comprehensive studies that showcase their impact and effectiveness.

Analytical Reports

Delve into critical thinking insights from our founders, offering fresh perspectives and groundbreaking approaches.

Your subscription includes access to our resource library and periodic emails that keep you informed and ahead in your field. Your privacy is important to us. We promise to keep your information safe and never spam you. You can unsubscribe at any time.

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Back to Entry
• Entry Contents
• Entry Bibliography
• Academic Tools
• Friends PDF Preview
• Author and Citation Info
• Back to Top

Supplement to Critical Thinking

How can one assess, for purposes of instruction or research, the degree to which a person possesses the dispositions, skills and knowledge of a critical thinker?

In psychometrics, assessment instruments are judged according to their validity and reliability.

Roughly speaking, an instrument is valid if it measures accurately what it purports to measure, given standard conditions. More precisely, the degree of validity is “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (American Educational Research Association 2014: 11). In other words, a test is not valid or invalid in itself. Rather, validity is a property of an interpretation of a given score on a given test for a specified use. Determining the degree of validity of such an interpretation requires collection and integration of the relevant evidence, which may be based on test content, test takers’ response processes, a test’s internal structure, relationship of test scores to other variables, and consequences of the interpretation (American Educational Research Association 2014: 13–21). Criterion-related evidence consists of correlations between scores on the test and performance on another test of the same construct; its weight depends on how well supported is the assumption that the other test can be used as a criterion. Content-related evidence is evidence that the test covers the full range of abilities that it claims to test. Construct-related evidence is evidence that a correct answer reflects good performance of the kind being measured and an incorrect answer reflects poor performance.

An instrument is reliable if it consistently produces the same result, whether across different forms of the same test (parallel-forms reliability), across different items (internal consistency), across different administrations to the same person (test-retest reliability), or across ratings of the same answer by different people (inter-rater reliability). Internal consistency should be expected only if the instrument purports to measure a single undifferentiated construct, and thus should not be expected of a test that measures a suite of critical thinking dispositions or critical thinking abilities, assuming that some people are better in some of the respects measured than in others (for example, very willing to inquire but rather closed-minded). Otherwise, reliability is a necessary but not a sufficient condition of validity; a standard example of a reliable instrument that is not valid is a bathroom scale that consistently under-reports a person’s weight.

Assessing dispositions is difficult if one uses a multiple-choice format with known adverse consequences of a low score. It is pretty easy to tell what answer to the question “How open-minded are you?” will get the highest score and to give that answer, even if one knows that the answer is incorrect. If an item probes less directly for a critical thinking disposition, for example by asking how often the test taker pays close attention to views with which the test taker disagrees, the answer may differ from reality because of self-deception or simple lack of awareness of one’s personal thinking style, and its interpretation is problematic, even if factor analysis enables one to identify a distinct factor measured by a group of questions that includes this one (Ennis 1996). Nevertheless, Facione, Sánchez, and Facione (1994) used this approach to develop the California Critical Thinking Dispositions Inventory (CCTDI). They began with 225 statements expressive of a disposition towards or away from critical thinking (using the long list of dispositions in Facione 1990a), validated the statements with talk-aloud and conversational strategies in focus groups to determine whether people in the target population understood the items in the way intended, administered a pilot version of the test with 150 items, and eliminated items that failed to discriminate among test takers or were inversely correlated with overall results or added little refinement to overall scores (Facione 2000). They used item analysis and factor analysis to group the measured dispositions into seven broad constructs: open-mindedness, analyticity, cognitive maturity, truth-seeking, systematicity, inquisitiveness, and self-confidence (Facione, Sánchez, and Facione 1994). The resulting test consists of 75 agree-disagree statements and takes 20 minutes to administer. A repeated disturbing finding is that North American students taking the test tend to score low on the truth-seeking sub-scale (on which a low score results from agreeing to such statements as the following: “To get people to agree with me I would give any reason that worked”. “Everyone always argues from their own self-interest, including me”. “If there are four reasons in favor and one against, I’ll go with the four”.) Development of the CCTDI made it possible to test whether good critical thinking abilities and good critical thinking dispositions go together, in which case it might be enough to teach one without the other. Facione (2000) reports that administration of the CCTDI and the California Critical Thinking Skills Test (CCTST) to almost 8,000 post-secondary students in the United States revealed a statistically significant but weak correlation between total scores on the two tests, and also between paired sub-scores from the two tests. The implication is that both abilities and dispositions need to be taught, that one cannot expect improvement in one to bring with it improvement in the other.

A more direct way of assessing critical thinking dispositions would be to see what people do when put in a situation where the dispositions would reveal themselves. Ennis (1996) reports promising initial work with guided open-ended opportunities to give evidence of dispositions, but no standardized test seems to have emerged from this work. There are however standardized aspect-specific tests of critical thinking dispositions. The Critical Problem Solving Scale (Berman et al. 2001: 518) takes as a measure of the disposition to suspend judgment the number of distinct good aspects attributed to an option judged to be the worst among those generated by the test taker. Stanovich, West and Toplak (2011: 800–810) list tests developed by cognitive psychologists of the following dispositions: resistance to miserly information processing, resistance to myside thinking, absence of irrelevant context effects in decision-making, actively open-minded thinking, valuing reason and truth, tendency to seek information, objective reasoning style, tendency to seek consistency, sense of self-efficacy, prudent discounting of the future, self-control skills, and emotional regulation.

It is easier to measure critical thinking skills or abilities than to measure dispositions. The following eight currently available standardized tests purport to measure them: the Watson-Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), the Cornell Critical Thinking Tests Level X and Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), the Ennis-Weir Critical Thinking Essay Test (Ennis & Weir 1985), the California Critical Thinking Skills Test (Facione 1990b, 1992), the Halpern Critical Thinking Assessment (Halpern 2016), the Critical Thinking Assessment Test (Center for Assessment & Improvement of Learning 2017), the Collegiate Learning Assessment (Council for Aid to Education 2017), the HEIghten Critical Thinking Assessment (https://territorium.com/heighten/), and a suite of critical thinking assessments for different groups and purposes offered by Insight Assessment (https://www.insightassessment.com/products). The Critical Thinking Assessment Test (CAT) is unique among them in being designed for use by college faculty to help them improve their development of students’ critical thinking skills (Haynes et al. 2015; Haynes & Stein 2021). Also, for some years the United Kingdom body OCR (Oxford Cambridge and RSA Examinations) awarded AS and A Level certificates in critical thinking on the basis of an examination (OCR 2011). Many of these standardized tests have received scholarly evaluations at the hands of, among others, Ennis (1958), McPeck (1981), Norris and Ennis (1989), Fisher and Scriven (1997), Possin (2008, 2013a, 2013b, 2013c, 2014, 2020) and Hatcher and Possin (2021). Their evaluations provide a useful set of criteria that such tests ideally should meet, as does the description by Ennis (1984) of problems in testing for competence in critical thinking: the soundness of multiple-choice items, the clarity and soundness of instructions to test takers, the information and mental processing used in selecting an answer to a multiple-choice item, the role of background beliefs and ideological commitments in selecting an answer to a multiple-choice item, the tenability of a test’s underlying conception of critical thinking and its component abilities, the set of abilities that the test manual claims are covered by the test, the extent to which the test actually covers these abilities, the appropriateness of the weighting given to various abilities in the scoring system, the accuracy and intellectual honesty of the test manual, the interest of the test to the target population of test takers, the scope for guessing, the scope for choosing a keyed answer by being test-wise, precautions against cheating in the administration of the test, clarity and soundness of materials for training essay graders, inter-rater reliability in grading essays, and clarity and soundness of advance guidance to test takers on what is required in an essay. Rear (2019) has challenged the use of standardized tests of critical thinking as a way to measure educational outcomes, on the grounds that  they (1) fail to take into account disputes about conceptions of critical thinking, (2) are not completely valid or reliable, and (3) fail to evaluate skills used in real academic tasks. He proposes instead assessments based on discipline-specific content.

There are also aspect-specific standardized tests of critical thinking abilities. Stanovich, West and Toplak (2011: 800–810) list tests of probabilistic reasoning, insights into qualitative decision theory, knowledge of scientific reasoning, knowledge of rules of logical consistency and validity, and economic thinking. They also list instruments that probe for irrational thinking, such as superstitious thinking, belief in the superiority of intuition, over-reliance on folk wisdom and folk psychology, belief in “special” expertise, financial misconceptions, overestimation of one’s introspective powers, dysfunctional beliefs, and a notion of self that encourages egocentric processing. They regard these tests along with the previously mentioned tests of critical thinking dispositions as the building blocks for a comprehensive test of rationality, whose development (they write) may be logistically difficult and would require millions of dollars.

A superb example of assessment of an aspect of critical thinking ability is the Test on Appraising Observations (Norris & King 1983, 1985, 1990a, 1990b), which was designed for classroom administration to senior high school students. The test focuses entirely on the ability to appraise observation statements and in particular on the ability to determine in a specified context which of two statements there is more reason to believe. According to the test manual (Norris & King 1985, 1990b), a person’s score on the multiple-choice version of the test, which is the number of items that are answered correctly, can justifiably be given either a criterion-referenced or a norm-referenced interpretation.

On a criterion-referenced interpretation, those who do well on the test have a firm grasp of the principles for appraising observation statements, and those who do poorly have a weak grasp of them. This interpretation can be justified by the content of the test and the way it was developed, which incorporated a method of controlling for background beliefs articulated and defended by Norris (1985). Norris and King synthesized from judicial practice, psychological research and common-sense psychology 31 principles for appraising observation statements, in the form of empirical generalizations about tendencies, such as the principle that observation statements tend to be more believable than inferences based on them (Norris & King 1984). They constructed items in which exactly one of the 31 principles determined which of two statements was more believable. Using a carefully constructed protocol, they interviewed about 100 students who responded to these items in order to determine the thinking that led them to choose the answers they did (Norris & King 1984). In several iterations of the test, they adjusted items so that selection of the correct answer generally reflected good thinking and selection of an incorrect answer reflected poor thinking. Thus they have good evidence that good performance on the test is due to good thinking about observation statements and that poor performance is due to poor thinking about observation statements. Collectively, the 50 items on the final version of the test require application of 29 of the 31 principles for appraising observation statements, with 13 principles tested by one item, 12 by two items, three by three items, and one by four items. Thus there is comprehensive coverage of the principles for appraising observation statements. Fisher and Scriven (1997: 135–136) judge the items to be well worked and sound, with one exception. The test is clearly written at a grade 6 reading level, meaning that poor performance cannot be attributed to difficulties in reading comprehension by the intended adolescent test takers. The stories that frame the items are realistic, and are engaging enough to stimulate test takers’ interest. Thus the most plausible explanation of a given score on the test is that it reflects roughly the degree to which the test taker can apply principles for appraising observations in real situations. In other words, there is good justification of the proposed interpretation that those who do well on the test have a firm grasp of the principles for appraising observation statements and those who do poorly have a weak grasp of them.

To get norms for performance on the test, Norris and King arranged for seven groups of high school students in different types of communities and with different levels of academic ability to take the test. The test manual includes percentiles, means, and standard deviations for each of these seven groups. These norms allow teachers to compare the performance of their class on the test to that of a similar group of students.

Copyright © 2022 by David Hitchcock < hitchckd @ mcmaster . ca >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2023 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Healthcare (Basel)

The Outcome-Present State Test Model of Clinical Reasoning to Promote Critical Thinking in Psychiatric Nursing Practice among Nursing Students: A Mixed Research Study

Associated data.

The data presented in this study are available on request from the corresponding author.

This study determined whether teaching intervention using the outcome-present state test (OPT) clinical reasoning model can effectively improve critical thinking in nursing students during a psychiatry internship. In addition, it evaluates the experiences of the students using this model in clinical practice. Methods: In this interventional study, 19 students were taught critical thinking skills using the OPT clinical reasoning model during a psychiatry clinical practice. Work-learning forms were used in daily 1 h individual and group discussions with students. The critical thinking disposition scale was completed by every student before and after the intervention. Moreover, the students were asked to the complete reflection experience forms. Results: The average critical thinking disposition pre-intervention score was 95.21, whereas the average post-intervention score was 97.05, indicating an increase of 1.84. There was a significant increase in the fourth dimension of open-mindedness (z = −2.80, p < 0.01). The learning experience has been likened to a process of clearing the fog, and it involves the use of limited known conditions, thinking outside the box, and adaptation to complex care issues. Conclusion: Using the OPT clinical reasoning model as a teaching strategy during a psychiatric nursing internship significantly improved the open-mindedness dimension among the students. The student reflective experience of talking to teachers as peers helped students identify clues and reframe problems related to clinical care. Additionally, the students reported that this led to more harmonious interactions with their teachers.

1. Introduction

Clinical practice is a core module of nursing education. During this practice, students acquire knowledge and learn practical skills in the clinical field. They also have the opportunity to apply what they have learned. In clinical practice, the role of the teacher is to optimize students’ learning. Thus, the teachers must find ways to promote proactive learning among their students. Notably, the critical thinking skills of nurses can affect care outcomes. Therefore, cultivating critical thinking skills in nursing students and integrating them into nursing education is necessary for students to gain sufficient professional responsibility [ 1 ].

Critical thinking is a complex activity, and its development requires education, time, and personal commitment [ 2 ]. The teachers’ ability to assess students is important for measuring critical thinking skills and monitoring their development [ 3 ]. Clinical reasoning as an analytical process used to solve clinical problems and patient care [ 4 ]. Although many explanations have been reported in the literature, there is a consensus that critical thinking is oriented thinking involving reflective reasoning before having a conclusion that leads to a clinical decision. Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and evaluating information [ 5 ]. The review of tools to measure critical thinking in nursing and midwifery students noted that the measurement of critical thinking in some studies reviewed may have been influenced by the impact of culture on different learning environments [ 6 ]. Of the 53 papers that were reviewed, the majority (n = 38) measured critical thinking. The other 48 papers focused on describing and/or validating a measurement tool or model. Multiple tools were used to measure critical thinking. Of the commercially developed tools, the most common were the California Critical Thinking Skills Test (CCTST) and California Critical Thinking Disposition Inventory (CCTDI) and variations of CCTST and CCTDI. The CCTST and the CCTDI were the only commercially developed tools that were translated into other languages [ 7 ].

Critical thinking is of great importance, but studies have showed that nursing students lack critical thinking dispositions and skills [ 8 ]. The improvement in critical thinking skills among nursing students can positively affect self-reflection and care behavior [ 9 ]. Researchers suggested that educational interventions including use of explicit teaching strategies and higher-order questions are to be promoted in advancing students’ critical thinking [ 10 , 11 ]. In the hospital, nurses are required to make important and quick decisions that have consequences on the health of their patients. Therefore, quick thinking and sufficient foresight to accurately predict the possible outcomes of intervention choices are central aspects of nursing. Thus, critical thinking skills need to be integrated in university nursing education. In addition, teaching methods and strategies should be changed to include reflection and development of clinical judgment for effective and ethical problem solving [ 12 ].

The Singapura study was to explore nursing students’ perceptions of critical thinking and clinical reasoning. The nursing students in this study felt that critical thinking and clinical reasoning were essential skills for providing high-quality patient care [ 13 ]. In nursing education, critical thinking is an essential skill for clinical reasoning and optimal nursing practice to improve patient care, and, thus, has become one of the most highly valued educational outcomes recently [ 14 ]. The study findings showed that nursing instructors must implement teaching strategies to develop critical thinking among students [ 15 ]. These teaching strategies should incorporate critical thinking into overall learning to encourage its use in clinical practice [ 16 , 17 ].

Pesut and Herman (1998) proposed the outcome-present state test (OPT) model to provide students with clinical reasoning tools to be used in complex scenarios [ 18 ]. The OPT model maps the relationships between the patient’s disease diagnosis and the associated care issues. This map is created by summarizing the patient’s background information and incorporating the knowledge of the nurses. A previous study reported that the clinical reasoning web can visualize the relationship between patients’ diagnoses and their care needs, allowing students to prioritize care problems as well as determine dynamic, interactive, and causal relationships between the contributing variables [ 19 ]. The study found that the use of the clinical reasoning web concept map resulted in significantly better problem solving, support, and understanding among nursing students. The web concept map to improve critical thinking in nursing students [ 20 ].

Research has shown that the critical thinking disposition is positively associated with effective clinical decision-making and competence among the clinical nurses [ 21 ], which subsequently improves patient safety and enhances patient outcomes [ 22 ]. Our study was conducted with nursing students through the learning strategies using the self-regulated learning (SRL) model and the OPT model of reflective clinical reasoning. SRL is a model of learning situated in social cognitive theory that views learners as active participants in their learning. The current study determines whether the OPT model can effectively improve critical thinking in nursing students during a psychiatry nursing practice and to understand students’ reflection experiences of using this clinical reasoning model.

2.1. Study Design

In this interventional study, 19 nursing students were taught to use the OPT model during a psychiatry nursing practice. Signed consent forms were obtained from all participants. The critical thinking disposition scale was completed by the nursing students before and after the practice.

2.2. Subjects and Setting

Overall, 19 third-year students from a nursing university participated in this study. Participants were also required to complete a psychiatric nursing practice. The practice was in the Tzu Chi Hospital psychiatry ward. These participants were classified into three groups of 5–7 students. The practice duration was 3 weeks. The practice instructor was also the researcher in this study.

2.3. Intervention

During the first week, in order to think about the patient problem, we used a clinical reasoning web worksheet. A group discussion was then conducted in which the students shared the contents of their worksheets and discussed the background of their patients. Students were encouraged to help each other identify clues, such as the patient’s major complaints, test results, and symptoms, and to use these to illustrate what they had learned about the patient’s history. They were also encouraged to help other to present their patient’s diagnosis.

In the second week, we used the clinical reasoning web worksheet, including a concept map, to help students think about the following: 1. Current evidence, 2. The relationships between the evidence and the nursing diagnosis, 3. How the nursing diagnosis was reached and how important it was, and 4. A topic summarizing the patient’s background history.

In the third week, the teacher and students completed the clinical reasoning web worksheet together. During the group discussion, the teacher encouraged the students to think about the following: 1. The significance of the tests being administered (current/pending results), 2. How the patient defined their current condition, 3. The gap between the patient’s expected outcomes and their current condition; 4. How these gaps might be filled, and 5. Whether thoughts about care measures or clue symptoms should be reviewed.

In addition, the students completed a reflection log as well as worksheets. A human pictogram was used to assist the students in completing the patients’ clinical care reports.

2.4. Data Collection

2.4.1. critical thinking disposition scale.

The Critical Thinking Scale is a self-development questionnaire by Yeh (1999) and was used to measure the critical thinking disposition of students. This scale is a 20-item, six-point Likert-type scale (ranging from 1 = No to 5 = Always). The Critical Thinking Scale was four dimensions: systematicity and analytical skills (9 items), open-mindedness and empathy (4 items), knowledge and curiosity (3 items), and overall reflection (4 items). Scores from the 20 items can be added up to obtain a total score, ranging from 20 to 120. The Cronbach’s alpha of the overall scale is 0.86 and subscale α values ranging from 0.53 to 0.76 [ 23 ].

2.4.2. Reflection

Reflection homework was completed by the participants, and the students were required to answer three questions.

• What was the most striking thing about the patient that you cared for in this internship and why?
• During the internship period, how has the clinical reasoning web and OPT clinical reasoning model worksheet helped you/your patient?
• What difficulties did you encounter when using the clinical reasoning web and OPT clinical reasoning model worksheet during the internship?

2.5. Data Analysis

2.5.1. quantitative data.

SPSS Statistics for Windows version 22.0 (IBM Corp. Armonk, NY, USA) was used for the statistical analysis of data. Parametric data were presented as mean and standard deviation, whereas nonparametric data were presented as median and percentage. Wilcoxon tests were used to identify pre- to post-intervention changes in critical thinking.

2.5.2. Qualitative Analysis

The students’ learning experiences were organized, and the data analysis was started with the transcription of the interview recordings and the preliminary organization of the transcripts. The investigator carefully read the transcripts as well as extracted descriptions of clinical scenarios and the reasoning used to form meaningful units. Units with similar contents were grouped into themes, and themes with similar concepts were grouped into categories. Furthermore, the investigator repeatedly reviewed the transcript data to increase the depth and breadth of the categories. The rigor of the data was trustworthiness, credibility, transferability, and conformability [ 24 ]. During the interviews, the investigator adopted a sincere nonjudgmental attitude to encourage the respondent to freely disclose his/her thoughts. The content of the interview was faithfully transcribed, and respondents were encouraged to describe their experiences in as much detail as possible. The students were considered coinvestigators, and the results were sent to the student for confirmation before the study was completed. In addition, the investigator described his/her own experiences, background, and thoughts to ensure that the entire study process was documented in detail. The researchers with peer discussions were repeated during the data analysis. Throughout the study, the researcher continued to study the relevant literature, reflecting on his/her experiences, and discussing his/her perspectives.

2.6. Ethical Considerations

This study was approved by the Institutional Review Board of Tzu Chi Hospital (IRB-110-158-B) and was conducted in accordance with the principles of the 2013 revision of the Declaration of Helsinki. Participants were invited to participate in the OPT clinical reasoning model and informed that their learning rights would not be compromised if they left the study early. All participants signed an informed consent form.

3.1. Demographics

The study included 19 participants; of these, 13 (69%) were women and 6 (31%) were men. The mean age of the participants was 21.4 years, and the mean course satisfaction score was 4.99.

3.2. Quantitative Research Results

As shown in Table 1 , the average pre-investigation critical thinking disposition score of the 19 students was 95.21, and the average post-investigation score was 97.05, indicating an increase of 1.84. Among the four critical thinking disposition dimensions, the post-investigation score for systematicity and analytical skills decreased by 1.21 from the pre-investigation score, but the difference was not significant. The score for open-mindedness had increased by 1.39 post-investigation, and the difference was significant (z = −2.80, p < 0.01). Additionally, the scores for curiosity and overall reflection increased by 0.9 and 0.73, respectively, and there were no differences between the pre- and post-investigation scores.

Critical thinking about differences in various aspects pre-post psychiatric practice ( n = 19).

As shown in Figure 1 , the worst scoring items on the critical thinking pre-test were item 2 “I attempted to use some new viewpoints or concepts,” and item 17 “When others proposed a viewpoint, I attempted to identify the implicit main arguments in the viewpoint.” The critical thinking scale item 8 that showed the maximum improvement was “When handling problems, I first define the problem clearly,” and the average score increased from 4.42 to 4.89.

The average score for each item in the critical thinking scale (n = 19).

3.3. Qualitative Research Results

Nursing students are unfamiliar with clinical care scenarios during their internship, are unaware of what to do, and may feel scared and anxious. This learning process is like entering a fog ( Table 2 ). Using appropriate teaching strategies, the internship instructor can guide students in clinical reasoning discussions and teach them to make multiple connections using limited information and identify their focus of care.

Categories and subcategories of the study.

3.3.1. Using Limited Known Conditions

The internship process is like solving a puzzle. Students learn to gather information about the clinical care of the case such as the patient’s chief complaint or test results and try making several connections. Therefore, any piece of information may pose a nursing issue that could be overlooked if neglected.

Identifying Problems from Scraps of Scattered Information

I had to identify important information based on scraps of information collected from the patient and collate it into the reasoning web and OPT worksheet. Problems could have occurred if the wrong key points were extracted during this transformation process. (B3).

I feel that there is a need to link the patient’s symptoms with lifestyle habits before admission or at initial disease onset. I had to frequently ask the patient about his/her medical history or family and work information again. (C3)

I am unclear about the integration of observation data and symptoms, and this may affect subsequent nursing problems and measures. There will be omissions in nursing problems if the observation data are extremely few. (K3)

Multidimensional Connections

I should try making multidimensional connections. Every piece of information may be a nursing problem, and omission of this will lead to overlooking the problem. If the problem is not resolved, the objective of this hospitalization will be lost. (E3)

I feel that using the OPT worksheet can allow me to better connect the patient’s problems and find correlations between the problems, which allows me to reflect on which management measures are better. Additionally, I would think of any difficulties the patient is facing, such as, measuring blood sugar at home, and then think about other methods to solve this problem. (L2)

3.3.2. Thinking outside the Box

Students used several problems and clues to learn how to clarify the context of a problem. During this process, the teacher and students worked together to identify different viewpoints and problem causality.

Clarifying the Context of the Problem

I can better analyze and understand the details omitted by me, more comprehensively think about problems, and identify blind spots and overcome them. Discussion with the teacher allowed me to comprehensively analyze nursing problems. (F2)

I feel that the reasoning web and OPT worksheet can help me better clarify the context of the problem. Simultaneously, I can better confirm nursing problems that match the patient’s condition and decrease problems that lead to misjudgment errors and increased length of hospitalization. In addition, this method allows for efficient and effective provision of nursing measures. (D2)

Adjusting One’s Thoughts

The main part of this theme is to adjust one’s own thoughts. This is because the information received is sometimes scattered and cannot be networked, resulting in an inability to use it for the patient. When the teacher asked questions, I kept repeating and going in circles and was unable to rearrange the patient care framework and derive new care interventions. (G2)

The main assistance is that it organizes thoughts, identifies primary and secondary problems, and allows me to think about the ways to provide suitable treatment. (J2)

3.3.3. Adapting to Complex Care Issues

Students became adept at using the OPT worksheet and implementing the strategies taught. After learning to adjust their thinking, they were better able to understand their patients’ clinical care problems and made fewer care omissions. The reframing proposed by the students was difficulties in learning.

Accurate Care Focus

I feel that these two reasoning tools allow me to clearly understand the priority and causality of the patient’s problems such that I could efficiently and accurately understand the focus of care, thereby determining the correct nursing problems and measures. (A2)

I feel that the greatest assistance rendered is in recognizing the patient’s current symptoms, and there is a clear care focus on implementation. (H2)

Decreasing Care Omissions

This form can help determine my patient’s current condition and problems and to examine them individually by analyzing various aspects. The form also allows me to know what I have omitted in addition to identifying the main issues. (E2)

Reframe Learning

I feel that this is a reframing, and it is difficult to understand as I need to consider the patient’s difficulties before thinking of other suitable methods. However, I became enlightened after discussing it with my teacher. (L3)

3.4. Compilation of Qualitative and Quantitative Results

The critical thinking scores were increased. This indicates that the OPT model teaching strategy helped improve critical thinking among nursing students. The students also reported that the learning process improved collaborative learning and their interactions with teachers. The students’ reflections on their experiences revealed that thinking outside the box and adjusting thoughts to become more problem-focused were effective. This result is consistent with the significant improvements in open-mindedness observed using the critical thinking disposition test.

In addition, nursing students reported that they could better adapt to complex nursing problems using the OPT model to quickly identify their patients’ nursing problems and that they could reduce their nursing care omissions. This explains why the students indicated that the clinical reasoning model provided a deeper understanding and improvement in their critical thinking after the internship was completed. The nursing students’ reflections on their experiences indicated that they viewed the OPT teaching method as a student-centered learning experience that required them to identify cues, think about those cues in context, connect variables within problems, and think outside the box. This learning experience inspired deeper learning and enhanced their cognitive abilities. This was demonstrated by the increase in the scores for critical thinking disposition in the quantitative results.

4. Discussion

Our study was conducted with nursing students through the learning strategies using the SRL model and the OPT model of reflective clinical reasoning. The SRL model in nursing is proposed as a theoretical structure that explains how clinical reasoning skills can be acquired through attention to reflective thinking and critical thinking skills. This approach aligns with nurse education’s underpinning philosophical approach to learning in fostering critical thinking and to be independent and lifelong learners [ 25 ]. Evidence suggests that effective and efficient clinical reasoning is a consequence of intentional reflection supported by self-regulation [ 26 ]. A study findings included the high level of reported motivational and learning strategies used by students in their approach to learning, and in their teachers as partner [ 27 ].

The critical thinking scores of nursing students improved after using the OPT clinical reasoning model as a teaching strategy during their psychiatric internship. This result is consistent with the result of a previous study that reported that improved critical thinking can help nursing students to more accurately use it in nursing internships and to become better at dealing with complex and challenging clinical scenarios [ 8 ]. In this study, worksheets were collaboratively completed by the teacher and students, and the students were guided to think about the connections between the clues and problems. This encouraged them to express their thoughts and learn to think contextually about their patient’s health. Nursing students perceived critical thinking and clinical reasoning as essential for nursing practices and described these skills as linking theory to practice. Strategies such as simulation, case studies, real clinical experiences, and guidance from clinical instructors/preceptors were found to stimulate critical thinking and clinical reasoning for the students. It is necessary to train and enhance smooth transition from theoretical to practical healthcare practices [ 13 ].

Reflections on the students’ experiences generally indicated that they found the OPT model worksheet helpful in accurately identifying the focus of the care situation, but they had difficulty in reframing the learning. When students were guided to think about the differences between patient-expected solutions and current care, whether the revision in care interventions was needed and whether they felt able to engage patients in discussions about their expectations, they generally found it difficult to answer such questions. Some students have proposed the challenges of critical thinking. The results of this study, which are consistent with earlier literature, show that students have difficulty in learning critical thinking processes, which can hinder the development of their self-confidence in critical thinking [ 28 ]. One explanation may be that students have been used to the Chinese conventional pedagogy that values rote-memorization and respecting authoritative figures (teachers) influenced by the Confucian culture. In the Chinese educational evaluation system that is often exam-oriented, there is less encouragement for having questions from students [ 29 ]. The study corroborates the assertion that students’ understanding and practice regarding critical thinking are shaped by social, cultural, and educational contexts [ 30 ].

The care culture in Taiwan was traditionally dominated by medical personnel, with patients expecting these professionals to effectively manage their healthcare and medical treatment. These cultural factors contribute to an individual’s critical thinking disposition [ 31 ]. The study of university students from New Zealand and Japan found that culture-related factors do influence students’ critical thinking use. However, the variations in these factors may not always correspond to differences in critical thinking. Their findings showed that there were no appreciable variations in the reported use of critical thinking between students from Western and Asian cultural backgrounds. Students’ use of critical thinking is more significantly impacted by the educational environment [ 32 ]. Therefore, nursing education in Taiwan must change. The students must learn to consider the patient’s perspective on patient outcomes and embrace patient-centered nursing practice.

The reflections of the nursing students in this present study reported that the use of the OPT worksheet helped them understand clinical nursing problems from the patient’s perspective and to reduce care omissions. Clues and concepts can be linked and discussed at different stages of the nursing process to strengthen the clinical preparation of such students. Students also learned to plan their care strategies and prioritize the most important items. This finding is consistent with previous research results indicating that critical thinking facilitates the nursing process [ 33 ]. In the study, lecturers presented several patient problems to the group and guided students to work from initial hypotheses based on prior knowledge or experience, and then collected data for further synthesis and analysis of health problem solutions and nursing intervention. The learning outcome design of the course tools were introduced to help students develop critical thinking, such as reflective journals of this course. These tools could cultivate students’ elaboration of critical thinking [ 14 ]. This methodology, such as interactive teaching strategies and student active participation, together contributed to the improvement of students’ critical thinking and creative self-efficacy [ 28 ]. Additionally, implementing a teaching strategy for critical thinking in clinical scenarios can enhance students’ theoretical and clinical competencies and skills as well as better prepare them for their future careers as professional nurses.

5. Conclusions

The OPT clinical reasoning model as a teaching strategy in a psychiatric nursing internship significantly improved critical thinking among the students. Teachers encouraged students to find clues and reframe clinical care problems. Students reported that they felt a sense of accomplishment and experienced more harmonious interactions with their teachers. We also found that students had difficulty understanding critical thinking as an abstract concept, but they could use worksheets and images to discuss and present concept map findings. The students found that this helped in easier understanding.

6. Limitations

This study has some limitations. The sample size was small, and generalizability was insufficient. Therefore, further studies on the OPT clinical reasoning model and critical thinking are needed to validate our study results.

Based on our findings, we propose two suggestions. First, we recommend that teachers introduce the OPT clinical reasoning worksheet during nursing internships and use images to facilitate discussion. This can effectively improve students’ understanding of the critical thinking process. Second, we recommend that the development and use of measure tools include care situations of critical thinking ability in future research. Although the critical thinking scale was used in this study, the questions excluded clinical scenarios that nursing students may face, leading to some discrepancy between the scale and the clinical learning content. The ward culture in a clinical setting is a significant aspect that influences how students learn to use clinical reasoning and critical thinking. The results of this study did not bring up the critical thinking discussions related to the interaction process with the nursing staff. Future research can explore the clinical reasoning learning experience of nursing students and clinical nurses during the practice process.

Funding Statement

This research was funded by Taiwan Ministry of Education grant number PMN1100565.

Author Contributions

Y.-C.M.: Conceptualization, Methodology, Investigation, Data Analysis, Writing—Review and Editing. J.-L.J.: Methodology, Formal Analysis, Writing—Review Y.-C.L.: Investigation, Formal Analysis. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The institutional review board reviewed and approved the study (IRB-110-158-B).

Informed Consent Statement

Not applicable.

Data Availability Statement

Conflicts of interest.

The authors have no competing interests to declare.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

We place functional and analytical cookies that are necessary for the website to work properly. We and third parties also place marketing and tracking cookies for social logins and to show personalized ads tailored to your (internet) behavior, if you consent. Choose "Consent" or read more in our cookie statement first. Reject CONSENT

Critical Thinking test

By 123test team . Updated May 12, 2023

Critical Thinking test reviews

This Critical Thinking test measures your ability to think critically and draw logical conclusions based on written information. Critical Thinking tests are often used in job assessments in the legal sector to assess a candidate's  analytical critical  thinking skills. A well known example of a critical thinking test is the Watson-Glaser Critical Thinking Appraisal .

Need more practice?

Score higher on your critical thinking test.

The test comprises of the following five sections with a total of 10 questions:

• Analysing Arguments
• Assumptions
• Interpreting Information

Instructions Critical Thinking test

Each question presents one or more paragraphs of text and a question about the information in the text. It's your job to figure out which of the options is the correct answer.

Below is a statement that is followed by an argument. You should consider this argument to be true. It is then up to you to determine whether the argument is strong or weak. Do not let your personal opinion about the statement play a role in your evaluation of the argument.

Statement: It would be good if people would eat vegetarian more often. Argument: No, because dairy also requires animals to be kept that will have to be eaten again later.

Is this a strong or weak argument?

Strong argument Weak argument

Statement: Germany should no longer use the euro as its currency Argument: No, because that means that the 10 billion Deutschmark that the introduction of the euro has cost is money thrown away.

Overfishing is the phenomenon that too much fish is caught in a certain area, which leads to the disappearance of the fish species in that area. This trend can only be reversed by means of catch reduction measures. These must therefore be introduced and enforced.

Assumption: The disappearance of fish species in areas of the oceans is undesirable.

Is the assumption made from the text?

Assumption is made Assumption is not made

As a company, we strive for satisfied customers. That's why from now on we're going to keep track of how quickly our help desk employees pick up the phone. Our goal is for that phone to ring for a maximum of 20 seconds.

Assumption: The company has tools or ways to measure how quickly help desk employees pick up the phone.

• All reptiles lay eggs
• All reptiles are vertebrates
• All snakes are reptiles
• All vertebrates have brains
• Some reptiles hatch their eggs themselves
• Most reptiles have two lungs
• Many snakes only have one lung
• Cobras are poisonous snakes
• All reptiles are animals

Conclusion: Some snakes hatch their eggs themselves.

Does the conclusion follow the statements?

Conclusion follows Conclusion does not follow

(Continue with the statements from question 5.)

Conclusion: Some animals that lay eggs only have one lung.

In the famous 1971 Stanford experiment, 24 normal, healthy male students were randomly assigned as 'guards' (12) or 'prisoners' (12). The guards were given a uniform and instructed to keep order, but not to use force. The prisoners were given prison uniforms. Soon after the start of the experiment, the guards made up all kinds of sentences for the prisoners. Insurgents were shot down with a fire extinguisher and public undressing or solitary confinement was also a punishment. The aggression of the guards became stronger as the experiment progressed. At one point, the abuses took place at night, because the guards thought that the researchers were not watching. It turned out that some guards also had fun treating the prisoners very cruelly. For example, prisoners got a bag over their heads and were chained to their ankles. Originally, the experiment would last 14 days. However, after six days the experiment was stopped.

The students who took part in the research did not expect to react the way they did in such a situation.

To what extent is this conclusion true, based on the given text?

True Probably true More information required Probably false False

(Continue with the text from 'Stanford experiment' in question 7.)

The results of the experiment support the claim that every young man (or at least some young men) is capable of turning into a sadist fairly quickly.

• A flag is a tribute to the nation and should therefore not be hung outside at night. Hoisting the flag therefore happens at sunrise, bringing it down at sunset. Only when a country flag is illuminated by spotlights on both sides, it may remain hanging after sunset. There is a simple rule of thumb for the time of bringing down the flag. This is the moment when there is no longer any visible difference between the individual colors of the flag.
• A flag may not touch the ground.
• On the Dutch flag, unless entitled to do so, no decorations or other additions should be made. Also the use of a flag purely for decoration should be avoided. However, flag cloth may be used for decoration - for example in the form of drapes.
• The orange pennant is only used on birthdays of members of the Royal House and on King's Day. The orange pennant should be as long or slightly longer than the diagonal of the flag.

Conclusion: One can assume that no Dutch flag will fly at government buildings at night, unless it is illuminated by spotlights on both sides.

Does the conclusion follow, based on the given text?

(Continue with the text from 'Dutch flag protocol' in question 9.)

Conclusion: If the protocol is followed, the orange pennant will always be longer than the horizontal bands/stripes of the flag.

Please answer the questions below. Not all questions are required but it will help us improve this test.

My educational level is

-- please select -- primary school high school college university PhD other

Spring Offer 2023

Seven comprehensive reports of career tests and iq tests with a whopping 70% discount..

Seven comprehensive reports of career tests and IQ tests with a whopping 70% discount:

• DISC Personality test
• Jung Personality test
• Big Five Personality test
• Work values test
• Competency test
• Classical IQ test
• Culture fair intelligence test

Het Werkboek Loopbaanbegeleiding helpt je de resultaten bij elkaar te brengen en een actieplan te maken.

Zijn je baan en je loopbaan belangrijk voor je?

With this unique offer, you get access to almost all the comprehensive products for one very competitive price.

Normally $84.00, now for $24.95

You will receive 7 ticket codes after purchase that you can use to start the tests from your personal account. This allows you to take the tests at your own pace and order.