research paper in bibliography

How to Write a Bibliography for a Research Paper

Do not try to “wow” your instructor with a long bibliography when your instructor requests only a works cited page. It is tempting, after doing a lot of work to research a paper, to try to include summaries on each source as you write your paper so that your instructor appreciates how much work you did. That is a trap you want to avoid. MLA style, the one that is most commonly followed in high schools and university writing courses, dictates that you include only the works you actually cited in your paper—not all those that you used.

Academic Writing, Editing, Proofreading, And Problem Solving Services

Get 10% off with 24start discount code, assembling bibliographies and works cited.

• If your assignment calls for a bibliography, list all the sources you consulted in your research.
• If your assignment calls for a works cited or references page, include only the sources you quote, summarize, paraphrase, or mention in your paper.
• If your works cited page includes a source that you did not cite in your paper, delete it.
• All in-text citations that you used at the end of quotations, summaries, and paraphrases to credit others for their ideas,words, and work must be accompanied by a cited reference in the bibliography or works cited. These references must include specific information about the source so that your readers can identify precisely where the information came from.The citation entries on a works cited page typically include the author’s name, the name of the article, the name of the publication, the name of the publisher (for books), where it was published (for books), and when it was published.

The good news is that you do not have to memorize all the many ways the works cited entries should be written. Numerous helpful style guides are available to show you the information that should be included, in what order it should appear, and how to format it. The format often differs according to the style guide you are using. The Modern Language Association (MLA) follows a particular style that is a bit different from APA (American Psychological Association) style, and both are somewhat different from the Chicago Manual of Style (CMS). Always ask your teacher which style you should use.

A bibliography usually appears at the end of a paper on its own separate page. All bibliography entries—books, periodicals, Web sites, and nontext sources such radio broadcasts—are listed together in alphabetical order. Books and articles are alphabetized by the author’s last name.

Most teachers suggest that you follow a standard style for listing different types of sources. If your teacher asks you to use a different form, however, follow his or her instructions. Take pride in your bibliography. It represents some of the most important work you’ve done for your research paper—and using proper form shows that you are a serious and careful researcher.

Bibliography Entry for a Book

A bibliography entry for a book begins with the author’s name, which is written in this order: last name, comma, first name, period. After the author’s name comes the title of the book. If you are handwriting your bibliography, underline each title. If you are working on a computer, put the book title in italicized type. Be sure to capitalize the words in the title correctly, exactly as they are written in the book itself. Following the title is the city where the book was published, followed by a colon, the name of the publisher, a comma, the date published, and a period. Here is an example:

Format : Author’s last name, first name. Book Title. Place of publication: publisher, date of publication.

• A book with one author : Hartz, Paula.  Abortion: A Doctor’s Perspective, a Woman’s Dilemma . New York: Donald I. Fine, Inc., 1992.
• A book with two or more authors : Landis, Jean M. and Rita J. Simon.  Intelligence: Nature or Nurture?  New York: HarperCollins, 1998.

Bibliography Entry for a Periodical

A bibliography entry for a periodical differs slightly in form from a bibliography entry for a book. For a magazine article, start with the author’s last name first, followed by a comma, then the first name and a period. Next, write the title of the article in quotation marks, and include a period (or other closing punctuation) inside the closing quotation mark. The title of the magazine is next, underlined or in italic type, depending on whether you are handwriting or using a computer, followed by a period. The date and year, followed by a colon and the pages on which the article appeared, come last. Here is an example:

Format:  Author’s last name, first name. “Title of the Article.” Magazine. Month and year of publication: page numbers.

• Article in a monthly magazine : Crowley, J.E.,T.E. Levitan and R.P. Quinn.“Seven Deadly Half-Truths About Women.”  Psychology Today  March 1978: 94–106.
• Article in a weekly magazine : Schwartz, Felice N.“Management,Women, and the New Facts of Life.”  Newsweek  20 July 2006: 21–22.
• Signed newspaper article : Ferraro, Susan. “In-law and Order: Finding Relative Calm.”  The Daily News  30 June 1998: 73.
• Unsigned newspaper article : “Beanie Babies May Be a Rotten Nest Egg.”  Chicago Tribune  21 June 2004: 12.

Bibliography Entry for a Web Site

For sources such as Web sites include the information a reader needs to find the source or to know where and when you found it. Always begin with the last name of the author, broadcaster, person you interviewed, and so on. Here is an example of a bibliography for a Web site:

Format : Author.“Document Title.” Publication or Web site title. Date of publication. Date of access.

Example : Dodman, Dr. Nicholas. “Dog-Human Communication.”  Pet Place . 10 November 2006.  23 January 2014 < http://www.petplace.com/dogs/dog-human-communication-2/page1.aspx >

After completing the bibliography you can breathe a huge sigh of relief and pat yourself on the back. You probably plan to turn in your work in printed or handwritten form, but you also may be making an oral presentation. However you plan to present your paper, do your best to show it in its best light. You’ve put a great deal of work and thought into this assignment, so you want your paper to look and sound its best. You’ve completed your research paper!

Back to  How To Write A Research Paper .

ORDER HIGH QUALITY CUSTOM PAPER

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

How to Write a Bibliography in APA Format

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

• APA Bibliography
• How to Create One
• Why You Need It

Sample Bibliography

An APA format bibliography lists all of the sources that might be used in a paper. A bibliography can be a great tool to help you keep track of information during the research and writing process. In some cases, your instructor may require you to include a bibliography as part of your assignment.

At a Glance

A well-written APA format bibliography can help you keep track of information and sources as you research and write your psychology paper. To create a bibliography, gather up all of the sources that you might use in your paper. Create an APA format reference for each source and then write a brief annotation. Your annotation should be a brief summary of what each reference is about. You can quickly refer to these annotations When writing your paper and determine which to include.

What Is an APA Format Bibliography?

An APA format bibliography is an alphabetical listing of all sources that might be used to write an academic paper, essay, article, or research paper—particularly work that is covering psychology or psychology-related topics. APA format is the official style of the American Psychological Association (APA). This format is used by many psychology professors, students, and researchers.

Even if it is not a required part of your assignment, writing a bibliography can help you keep track of your sources and make it much easier to create your final reference page in proper APA format.

Creating an APA Bibliography

A bibliography is similar in many ways to a reference section , but there are some important differences. While a reference section includes every source that was actually used in your paper, a bibliography may include sources that you considered using but may have dismissed because they were irrelevant or outdated.

Bibliographies can be a great way to keep track of information you might want to use in your paper and to organize the information that you find in different sources. The following are four steps you can follow to create your APA format bibliography.

Start on a New Page

Your working bibliography should be kept separate from the rest of your paper. Start it on a new page, with the title "Bibliography" centered at the top and in bold text. Some people use the title "References" instead, so it's best to check with your professor or instructor about which they prefer you to use.

Gather Your Sources

Compile all the sources you might possibly use in your paper. While you might not use all of these sources in your paper, having a complete list will make it easier later on when you prepare your reference section.

Gathering your sources can be particularly helpful when outlining and writing your paper.

By quickly glancing through your working bibliography, you will be able to get a better idea of which sources will be the most appropriate to support your thesis and main points.

Reference Each Source

Your references should be listed alphabetically by the author’s last name, and they should be double-spaced. The first line of each reference should be flush left, while each additional line of a single reference should be a few spaces to the right of the left margin, which is known as a hanging indent.

The format of each source is as follows for academic journals:

• Last name of first author (followed by their first initial)
• The year the source was published in parentheses
• The title of the source
• The journal that published the source (in italics)
• The volume number, if applicable (in italics)
• The issue number, if applicable
• Page numbers (in parentheses)
• The URL or "doi" in lowercase letters followed by a colon and the doi number, if applicable

The following examples are scholarly articles in academic journals, cited in APA format:

• Kulacaoglu, F., & Kose, S. (2018). Borderline personality disorder (BPD): In the midst of vulnerability, chaos, and awe.  Brain sciences ,  8 (11), 201. doi:10.3390/brainsci8110201
• Cattane, N., Rossi, R., & Lanfredi, M. (2017). Borderline personality disorder and childhood trauma: exploring the affected biological systems and mechanisms.  BMC Psychiatry,   18 (221). doi:10.1186/s12888-017-1383-2

Visit the American Psychological Association's website for more information on citing other types of sources including online media, audiovisual media, and more.

Create an Annotation for Each Source

Normally a bibliography contains only references' information, but in some cases you might decide to create an annotated bibliography. An annotation is a summary or evaluation of the source.

An annotation is a brief description of approximately 150 words describing the information in the source, your evaluation of its credibility, and how it pertains to your topic. Writing one of these for each piece of research will make your writing process faster and easier.

This step helpful in determining which sources to ultimately use in your paper. Your instructor may also require it as part of the assignment so they can assess your thought process and understanding of your topic.

Reasons to Write a Bibliography

One of the biggest reasons to create an APA format bibliography is simply to make the research and writing process easier.

If you do not have a comprehensive list of all of your references, you might find yourself scrambling to figure out where you found certain bits of information that you included in your paper.

A bibliography is also an important tool that your readers can use to access your sources.

While writing an annotated bibliography might not be required for your assignment, it can be a very useful step. The process of writing an annotation helps you learn more about your topic, develop a deeper understanding of the subject, and become better at evaluating various sources of information.

The following is an example of an APA format bibliography by the website EasyBib:

There are many online resources that demonstrate different formats of bibliographies, including the American Psychological Association website . Purdue University's Online Writing Lab also has examples of formatting an APA format bibliography.

Check out this video on their YouTube channel which provides detailed instructions on formatting an APA style bibliography in Microsoft Word.

You can check out the Purdue site for more information on writing an annotated APA bibliography as well.

What This Means For You

If you are taking a psychology class, you may be asked to create a bibliography as part of the research paper writing process. Even if your instructor does not expressly require a bibliography, creating one can be a helpful way to help structure your research and make the writing process more manageable.

For psychology majors , it can be helpful to save any bibliographies you have written throughout your studies so that you can refer back to them later when studying for exams or writing papers for other psychology courses.

American Psychological Association. Publication Manual of the American Psychological Association . 7th Edition. Washington, DC: American Psychological Association; 2020.

Masic I. The importance of proper citation of references in biomedical articles.   Acta Inform Med . 2013;21(3):148–155. doi:10.5455/aim.2013.21.148-155

American Psychological Association. How do you format a bibliography in APA Style?

Cornell University Library. How to prepare an annotated bibliography: The annotated bibliography .

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• Try for free

How to Write a Bibliography (MLA, APA Examples)

Learn how to easily write a bibliography by following the format outlined in this article.

This resource will help your students properly cite different resources in the bibliography of a research paper, and how to format those citations, for books, encyclopedias, films, websites, and people.

What is a bibliography?

According to Infoplease.com, A bibliography is a list of the types of sources you used to get information for your report. It is included at the end of your report, on the last page (or last few pages).

What are the types of bibliography styles (MLA, APA, etc.)?

The 3 most common bibliography/citation styles are:

• MLA Style: The Modern Language Association works cited page style
• APA Style: The American Psychological Association style
• Chicago Style: The bibliography style defined by the Chicago Manual of Style

We’ll give examples of how to create bibliography entries in various styles further down in this article. 

What sources do you put in a bibliography?

An annotated bibliography should include a reference list of any sources you use in writing a research paper. Any printed sources from which you use a text citation, including books, websites, newspaper articles, journal articles, academic writing, online sources (such as PDFs), and magazines should be included in a reference list. In some cases, you may need or want to cite conversations or interviews, works of art, visual works such as movies, television shows, or documentaries - these (and many others) can also be included in a reference list.

How to get started writing your bibliography

You will find it easier to prepare your MLA, APA, or Chicago annotated bibliography if you keep track of each book, encyclopedia, journal article, webpage or online source you use as you are reading and taking notes. Start a preliminary, or draft, bibliography by listing on a separate sheet of paper all your sources. Note down the full title, author’s last name, place of publication, web address, publisher, and date of publication for each source.

Haven't started your paper yet and need an outline? These sample essay outlines include a research paper outline from an actual student paper.

How to write a bibliography step-by-step (with examples)

General Format: Author (last name first). Title of the book. Publisher, Date of publication.

MLA Style: Sibley, David Allen. What It’s Like to Be a Bird. From Flying to Nesting, Eating to Singing, What Birds Are Doing, and Why. Alfred A. Knopf, 2020.

APA Style: Sibley, D.A. (2020). What It’s Like to Be a Bird. From Flying to Nesting, Eating to Singing, What Birds Are Doing, and Why . Alfred A. Knopf.

Notes: Use periods, not commas, to separate the data in the entry. Use a hanging indent if the entry is longer than one line. For APA style, do not use the full author’s first name.

Websites or webpages:

  MLA Style: The SB Nation Family of Sites. Pension Plan Puppets: A Toronto Maple Leafs Blog, 2022, www.pensionplanpuppets.com. Accessed 15 Feb. 2022.

APA Style: American Heart Association. (2022, April 11). How to keep your dog’s heart healthy. https://www.heart.org/en/news/2022/04/11/how-to-keep-your-dogs-heart-healthy

Online news article from a newspaper site:

APA Style: Duehren, A. (2022, April 9). Janet Yellen faces challenge to keep pressure on Russia. Wall Street Journal. https://www.wsj.com/articles/janet-yellen-faces-challenge-to-keep-pressure-on-russia-while-addressing-global-consequences-11650366000

Print journal articles:

MLA Style: Booch, Grady. "Patterns in Object-Oriented Design." IEEE Software Engineering, vol. 6, no. 6, 2006, pp. 31-50.

APA Style: Booch, G. (2006). Patterns in object-oriented design. IEEE Software Engineering, 6(6), 31–50.

Note: It is suggested that you include a DOI and a webpage address when referencing either a printed journal article, and electronic journal article, or an journal article that appears in both formats. 

MLA Style: Gamma, Eric, and Peter A. Coad. “Exceptions to the Unified Modeling Language in Python Patterns.” IEEE Software Engineering, vol. 2, no. 6, 8 Mar. 2006, pp. 190-194. O’Reilly Software Engineering Library, https://doi.org/10.1006/se.20061. Accessed 26 May 2009.

APA Style: Masters, H., Barron, J., & Chanda, L. (2017). Motivational interviewing techniques for adolescent populations in substance abuse counseling. NAADAC Notes, 7(8), 7–13. https://www.naadac.com/notes/adolescent-techniques

ML:A Style: @Grady_Booch. “That’s a bold leap over plain old battery power cars.” Twitter, 13 Mar. 2013, 12:06 p.m., https://twitter.com/Grady_Booch/status/1516379006727188483.

APA Style: Westborough Library [@WestboroughLib]. (2022, April 12). Calling all 3rd through 5th grade kids! Join us for the Epic Writing Showdown! Winner receives a prize! Space is limited so register, today. loom.ly/ypaTG9Q [Tweet; thumbnail link to article]. Twitter. https://twitter.com/WestboroughLib/status/1516373550415896588.

Print magazine articles:

General format: Author (last name first), "Article Title." Name of magazine. Volume number, (Date): page numbers.

MLA Style: Stiteler, Sharon. "Tracking Red-Breasted Grosbeak Migration." Minnesota Bird Journal, 7 Sept. 2019, pp. 7-11.

APA Style: Jordan, Jennifer, "Filming at the Top of the World." Museum of Science Magazine. Volume 47, No. 1, (Winter 1998): p. 11.

Print newspaper articles:

General format: Author (last name first), "Article Title." Name of newspaper, city, state of publication. (date): edition if available, section, page number(s).

MLA Style: Adelman, Martin. "Augustus Announces Departure from City Manager Post." New York Times, late ed., 15 February 2020, p. A1

APA Style: Adelman, M. (2020, February 15). Augustus announced departure from city manager post. New York Times, A1.

Encyclopedias:

General Format: Encyclopedia Title, Edition Date. Volume Number, "Article Title," page numbers.

MLA Style: “Gorillas.” The Encyclopedia Brittanica. 15th ed. 2010.

APA Style: Encyclopedia Brittanica, Inc. (1997.) Gorillas. In The Encyclopedia Brittanica (15th ed., pp. 50-51). Encyclopedia Brittanica, Inc.

Personal interviews:

General format: Full name (last name first). Personal Interview. (Occupation.) Date of interview.

MLA Style: Smithfield, Joseph. Personal interview. 19 May 2014.

APA Style: APA does not require a formal citation for a personal interview. Published interviews from other sources should be cited accordingly.

Films and movies:

General format: Title, Director, Distributor, Year.

MLA Style: Fury. Directed by David Ayer, performances by Brad Pitt, Shia LaBeouf, Jon Bernthal, Sony Pictures, 2014.

APA Style: Ayer, D. (Director). (2014). Fury [Film]. Sony Pictures.

Featured High School Resources

Related Resources

About the author

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Referencing
• Harvard Style Bibliography | Format & Examples

Harvard Style Bibliography | Format & Examples

Published on 1 May 2020 by Jack Caulfield . Revised on 7 November 2022.

In Harvard style , the bibliography or reference list provides full references for the sources you used in your writing.

• A reference list consists of entries corresponding to your in-text citations .
• A bibliography sometimes also lists sources that you consulted for background research, but did not cite in your text.

The two terms are sometimes used interchangeably. If in doubt about which to include, check with your instructor or department.

The information you include in a reference varies depending on the type of source, but it usually includes the author, date, and title of the work, followed by details of where it was published. You can automatically generate accurate references using our free reference generator:

Harvard Reference Generator

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Formatting a harvard style bibliography, harvard reference examples, referencing sources with multiple authors, referencing sources with missing information, frequently asked questions about harvard bibliographies.

Sources are alphabetised by author last name. The heading ‘Reference list’ or ‘Bibliography’ appears at the top.

Each new source appears on a new line, and when an entry for a single source extends onto a second line, a hanging indent is used:

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Reference list or bibliography entries always start with the author’s last name and initial, the publication date and the title of the source. The other information required varies depending on the source type. Formats and examples for the most common source types are given below.

• Entire book
• Book chapter
• Translated book
• Edition of a book

Journal articles

• Print journal
• Online-only journal with DOI
• Online-only journal without DOI
• General web page
• Online article or blog
• Social media post

Newspapers and magazines

• Newspaper article
• Magazine article

When a source has up to three authors, list all of them in the order their names appear on the source. If there are four or more, give only the first name followed by ‘ et al. ’:

Sometimes a source won’t list all the information you need for your reference. Here’s what to do when you don’t know the publication date or author of a source.

Some online sources, as well as historical documents, may lack a clear publication date. In these cases, you can replace the date in the reference list entry with the words ‘no date’. With online sources, you still include an access date at the end:

When a source doesn’t list an author, you can often list a corporate source as an author instead, as with ‘Scribbr’ in the above example. When that’s not possible, begin the entry with the title instead of the author:

Though the terms are sometimes used interchangeably, there is a difference in meaning:

• A reference list only includes sources cited in the text – every entry corresponds to an in-text citation .
• A bibliography also includes other sources which were consulted during the research but not cited.

In Harvard referencing, up to three author names are included in an in-text citation or reference list entry. When there are four or more authors, include only the first, followed by ‘ et al. ’

In Harvard style referencing , to distinguish between two sources by the same author that were published in the same year, you add a different letter after the year for each source:

• (Smith, 2019a)
• (Smith, 2019b)

Add ‘a’ to the first one you cite, ‘b’ to the second, and so on. Do the same in your bibliography or reference list .

To create a hanging indent for your bibliography or reference list :

• Highlight all the entries
• Click on the arrow in the bottom-right corner of the ‘Paragraph’ tab in the top menu.
• In the pop-up window, under ‘Special’ in the ‘Indentation’ section, use the drop-down menu to select ‘Hanging’.
• Then close the window with ‘OK’.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Caulfield, J. (2022, November 07). Harvard Style Bibliography | Format & Examples. Scribbr. Retrieved 25 March 2024, from https://www.scribbr.co.uk/referencing/harvard-bibliography/

Is this article helpful?

Jack Caulfield

Other students also liked, a quick guide to harvard referencing | citation examples, harvard in-text citation | a complete guide & examples, referencing books in harvard style | templates & examples, scribbr apa citation checker.

An innovative new tool that checks your APA citations with AI software. Say goodbye to inaccurate citations!

The Quintessence of Basic and Clinical Research and Scientific Publishing pp 657–665 Cite as

How to Create a Bibliography

• Rohan Reddy 4 ,
• Samuel Sorkhi 4 ,
• Saager Chawla 4 &
• Mahadevan Raj Rajasekaran 5  
• First Online: 01 October 2023

606 Accesses

This chapter describes the fundamental principles and practices of referencing sources in scientific writing and publishing. Understanding plagiarism and improper referencing of the source material is paramount to producing original work that contains an authentic voice. Citing references helps authors to avoid plagiarism, give credit to the original author, and allow potential readers to refer to the legitimate sources and learn more information. Furthermore, quality references serve as an invaluable resource that can enlighten future research in a field. This chapter outlines fundamental aspects of referencing as well as how these sources are formatted as per recommended citation styles. Appropriate referencing is an important tool that can be utilized to develop the credibility of the author and the arguments presented. Additionally, online software can be useful in helping the author organize their sources and promote proper collaboration in scientific writing.

• Referencing
• Harvard and Vancouver styles
• Referencing software
• Scientific writing

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

AWELU (2022) The functions of references. Lund University. https://www.awelu.lu.se/referencing/the-functions-of-references/ . Accessed 28 Dec 2022

Masic I (2013) The importance of proper citation of references in biomedical articles. Acta Inform Med 21(3):148–155

Article   PubMed   PubMed Central   Google Scholar  

Neville C (2012) Referencing: principles, practice and problems. RGUHS J Pharm Sci 2(2):1–8

Google Scholar  

Horkoff T (2015) Writing for success 1st Canadian edition: BCcampus. Chapter 7. Sources: choosing the right ones. https://opentextbc.ca/writingforsuccess/chapter/chapter-7-sources-choosing-the-right-ones/ . Accessed 28 Dec 2022

A guide to database and catalog searching: bibliographic elements. Northwestern State University. https://libguides.nsula.edu/howtosearch/bibelements . Updated 5 Aug 2021; Accessed 28 Dec 2022

Research process: step 7: citing and keeping track of sources. University of Rio Grande. https://libguides.rio.edu/c.php?g=620382&p=4320145 . Updated 12 Dec 2022; Accessed 28 Dec 2022

Pears R, Shields G (2022) Cite them right. Bloomsbury Publishing

Harvard system (1999) Bournemouth University. http://ibse.hk/Harvard_System.pdf . Accessed 28 Dec 2022

Gitanjali B (2004) Reference styles and common problems with referencing: Medknow. https://www.jpgmonline.com/documents/author/24/11_Gitanjali_3.pdf . Accessed 28 Dec 2022

Williams K, Spiro J, Swarbrick N (2008) How to reference Harvard referencing for Westminster Institute students. Westminster Institute of Education Oxford Brookes University

Harvard: reference list and bibliography: University of Birmingham. 2022. https://intranet.birmingham.ac.uk/as/libraryservices/library/referencing/icite/harvard/referencelist.aspx . Accessed 28 Dec 2022

Limited SN (2017) Responsible referencing. Nat Methods 14(3):209

Article   Google Scholar  

Hensley MK (2011) Citation management software: features and futures. Ref User Serv Q 50(3):204–208

Ramesh A, Rajasekaran MR (2014) Zotero: open-source bibliography management software. RGUHS J Pharm Sci 4:3–6

Shapland M (2000) Evaluation of reference management software on NT (comparing Papyrus with Procite, Reference Manager, Endnote, Citation, GetARef, Biblioscape, Library Master, Bibliographica, Scribe, Refs). University of Bristol

EndNote 20. Clarivate. 2022. https://endnote.com/product-details . Accessed 28 Dec 2022

Angélil-Carter S (2014) Stolen language? Plagiarism in writing. Routledge

Book   Google Scholar  

Howard RM (1995) Plagiarisms, authorships, and the academic death penalty. Coll Engl 57(7):788–806

Barrett R, Malcolm J (2006) Embedding plagiarism education in the assessment process. Int J Educ Integr 2(1):38–45

Introna L, Hayes N, Blair L, Wood E (2003) Cultural attitudes towards plagiarism. University of Lancaster, Lancaster, pp 1–57

Neville C (2016) The complete guide to referencing and avoiding plagiarism. McGraw-Hill Education (UK), London, p 43

Campion EW, Anderson KR, Drazen JM (2001) Internet-only publication. N Engl J Med 345(5):365

Li X, Crane N (1996) Electronic styles: a handbook for citing electronic information. Information Today, Inc.

APA (2005) Concise rules of APA style. APA, Washington, DC

Snyder PJ, Peterson A (2002) The referencing of internet web sites in medical and scientific publications. Brain Cogn 50(2):335–337

Article   PubMed   Google Scholar  

Barroga EF (2014) Reference accuracy: authors’, reviewers’, editors’, and publishers’ contributions. J Korean Med Sci 29(12):1587–1589

Standardization IOf (1997) Information and documentation—bibliographic references—part 2: electronic documents or parts thereof, ISO 690-2: 1997. ISO

Download references

Conflict of Interest

None declared.

Author information

Authors and affiliations.

Department of Urology, University of California at San Diego, San Diego, CA, USA

Rohan Reddy, Samuel Sorkhi & Saager Chawla

Department of Urology, University of California and VA San Diego Healthcare System, San Diego, CA, USA

Mahadevan Raj Rajasekaran

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mahadevan Raj Rajasekaran .

Editor information

Editors and affiliations.

Retired Senior Expert Pharmacologist at the Office of Cardiology, Hematology, Endocrinology, and Nephrology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA

Gowraganahalli Jagadeesh

Professor & Director, Research Training and Publications, The Office of Research and Development, Periyar Maniammai Institute of Science & Technology (Deemed to be University), Vallam, Tamil Nadu, India

Pitchai Balakumar

Division Cardiology & Nephrology, Office of Cardiology, Hematology, Endocrinology and Nephrology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA

Fortunato Senatore

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Cite this chapter.

Reddy, R., Sorkhi, S., Chawla, S., Rajasekaran, M.R. (2023). How to Create a Bibliography. In: Jagadeesh, G., Balakumar, P., Senatore, F. (eds) The Quintessence of Basic and Clinical Research and Scientific Publishing. Springer, Singapore. https://doi.org/10.1007/978-981-99-1284-1_39

Download citation

DOI : https://doi.org/10.1007/978-981-99-1284-1_39

Published : 01 October 2023

Publisher Name : Springer, Singapore

Print ISBN : 978-981-99-1283-4

Online ISBN : 978-981-99-1284-1

eBook Packages : Biomedical and Life Sciences Biomedical and Life Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Home / Guides / Citation Guides / MLA Format / Creating an MLA Bibliography

Creating an MLA Bibliography

If you write a research paper in MLA format, then you will need to include a Works Cited page according to the current 9th edition of the Modern Language Association (MLA) guidelines. Along with citing your sources within the body of your paper, you also need to include full citations of all sources at the end of your paper. The references in a bibliography are formatted in the same way as they would be in a Works Cited page. However, a bibliography refers to all works that you have consulted in your research, even if you did not use their information directly in your paper.

When you use the correct MLA bibliography format, it shows the reader what sources you consulted, makes finding your sources easier for the reader, and gives credibility to your work as a researcher and writer. This MLA sample paper will show you how the bibliography is incorporated into the rest of your paper. We also have a guide on APA reference pages , if you are following APA style in your paper.

Works cited or bibliography?

You may be wondering, what is a bibliography, and how is it different from a Works Cited page? The difference between the two is that while a bibliography refers to any source you consulted to write your research paper, a Works Cited page only includes full citations of the sources you quoted or paraphrased within your paper.

Typically, when someone says, “MLA bibliography” they really mean a Works Cited page, since the MLA format usually uses a Works Cited page instead of a bibliography.

A bibliography in MLA format may also refer to a Works Consulted page. If you used other sources that you did not directly quote or paraphrase within the paper, you will need to create a Works Consulted/Additional Resources page. A Works Consulted page starts on a separate page and follows the Works Cited page. It follows the same formatting guidelines as a Works Cited page, but you will use Works Consulted (or Additional Resources) as the title.

If you’re unsure of what to include in your citations list (works cited, works consulted, or both), ask your instructor. For the rest of this article, we will refer to this page as the MLA bibliography.

MLA bibliography formatting guidelines

These are the formatting rules you need to follow to create your bibliography according to MLA’s current edition guidelines. Your first page(s) will be your Works Cited page(s) and include the references that you directly refer to in your paper. Usually, this is all that is needed. If your instructor wants you to also include the works you consulted but did not include in your paper (more like a bibliography), then add Works Consulted or Additional Resources page for these sources.

• Your MLA Works Cited (and Works Consulted or Additional Resources pages) should begin on a separate page or pages at the end of your essay.
• Your essay should have a header on every page that includes your last name and the page number.
• The last name/page number header should be on the top right of each page with a ½ inch margin from the top of the page.
• One-inch margins.
• Title the page Works Cited (no italicization or quotation marks) unless otherwise instructed. Center the title. The top should look like this:

• Only center the Works Cited title; all citations should be left-justified.
• Double-space citations.
• Do not add an additional space between citations.
• After the first line, use a hanging indent of ½ inch on all additional lines of a citation. The hanging indent should look like this:

• Typically, this is the author’s last name, but sometimes it could be the title of the source if the author’s name is not available.

If you have a Works Consulted or Additional Resources page after your Works Cited page, format it in the same way, but with the title of Works Consulted or Additional Resources instead of Works Cited. Alternatively, your instructor may require a bibliography. If this is the case, all your sources, whether they are cited in your paper are not, are listed on the same page.

MLA citation guidelines

These are the rules you need to follow to create citations for an MLA bibliography. This section contains information on how to correctly use author names, punctuation, capitalization, fonts, page numbers, DOIs, and URLS in the citations on your MLA bibliography.

Author names

After the title Works Cited, the last name of the author of a source should be the first thing to appear on your page.

List the author’s last name followed by a comma, then the first name followed by the middle name or middle initial if applicable, without a comma separating the first and middle names. Add a period after the name.

Rowling, J.K.

Smith, Alexander McCall.

• Do not include titles such as Dr., Mrs., etc. or professional qualifications such as PhD, M.S., etc. with author names.
• Include suffixes such as Jr. or III after the author’s first name. Separate the first name and the suffix by a comma unless the suffix is a numeral. For example, to cite an author named John Smith, Jr., you would type Smith, John, Jr.

Sources with two authors

For a source with two authors, list the author names in your citation in the order they appear on the source, not alphabetically.

Type the last name of the first author listed on the source followed by a comma, then the first author’s first name followed by a comma. Then type the word “and” then list the second author’s first name and last name in the standard order. Follow the second name with a period.

Include middle names or initials and suffixes when applicable according to the guidelines for one author as listed above.

1st Author’s Last Name, First Name, and 2nd Author’s First Name Last Name.

Lutz, Lisa, and David Hayward.

Clark, Mary Higgins, and Alafair Burke.

Sources with three or more authors

For a source with three or more authors, only type the last and first name of the first author listed in the source, followed by a comma and the phrase et al., which is Latin for “and others.” Be sure to always place a period after the al in et al. but never after the et.

1st Author’s Last Name, First Name, et al.

Charaipotra, Sona, et al.

Williams, Beatriz, et al. All the Ways We Said Goodbye . HarperLuxe, 2020.

Organizations and corporations as authors

For sources with organizations or corporations listed as the author, type the name of the corporation in place of an author’s name. If the organization begins with an article like a, an, or the, it should be excluded in the Works Cited entry.

Modern Language Association of America. MLA Handbook . 2016.

*Note: If the organization is listed as both the author and the publisher, begin the citation with the title and include the organization’s name within the publisher field instead. 

For a source with no author listed, simply omit the author’s name and begin the citation with the title of the source. Use the first letter of the title when considering alphabetical order in your MLA bibliography.

Capitalization

Use MLA title case when citing titles of sources.

• Nouns, pronouns, verbs, adverbs, adjectives, and subordinating conjunctions should be capitalized.
• Articles, prepositions, and coordinating conjunctions should not be capitalized.

Font formatting

• Italicize the titles of larger works such as magazines and books. Also, italicize database and website names.
• Instead of italicization, use quotation marks around titles of shorter works such as poems, short stories, and articles.
• End all bibliography citations with a period.

Page numbers

Include page numbers in your full citations whenever possible. This helps the reader find the information you cited more quickly than if you just cited the entire source and lends more credibility to your argument. If you cite different pages from the same source within your paper, you should cite the entire source on your MLA bibliography instead of listing all of the page numbers you used.

When including page numbers in a citation, use the abbreviation p. to cite one page and the abbreviation pp. to cite multiple pages with a hyphen between the page numbers.

p. 25 or pp. 16-37

When citing page numbers in MLA, omit the first set of repeated digits.

pp. 365-69, not pp. 365-369

DOIs and URLs

A Digital Object Identifier (DOI) is used to locate and identify an online source. While URLs may change or web pages might be edited or updated, a DOI is permanent and therefore more useful in a source citation.

• Use a DOI (digital object identifier) whenever possible. Otherwise use a permalink or URL.
• DOIs should be formatted with “https://doi.org/” before the DOI number.
• Do not include “http://” or “https://” in your URLs.
• As either one will be the last part of your citation, place a period after the DOI or URL. (Note that this period is not part of the DOI or URL.)

Butarbutar, R, et al. “Analyzing of Puzzle Local Culture-Based in Teaching English for Young Learners.” IOP Conference Series: Earth and Environmental Science , vol. 343, 2019, https://doi.org/10.1088/1755-1315/343/1/012208.

Accessed dates

Since the previous 8th edition of the MLA Handbook was published, you do NOT need to list an accessed date for a stable source (e.g., online newspaper article, journal article, photograph, etc.). However, including an access date is good to include when a source does not have a publishing date, and some instructors will request that accessed dates be included for all sources.

If you do include an access date, here’s how to format it:

• Place it at the end of the citation without “http://” or “https://”.
• Write “Accessed” first, followed by the date accessed.
• The date accessed should be formatted as Day Month (abbreviated) Year.

Butarbutar, R, et al. “IOPscience.” IOP Conference Series: Earth and Environmental Science , IOP Publishing, 1 Oct. 2019, iopscience.iop.org/article/10.1088/1755-1315/343/1/012208/meta. Accessed 8 Oct. 2020.

Note: If you choose to list an accessed date after a DOI, the accessed date part of the citation will follow the period after the DOI and will end with a period at the end of the citation

Butarbutar, R, et al. “Analyzing of Puzzle Local Culture-Based in Teaching English for Young Learners.” IOP Conference Series: Earth and Environmental Science , vol. 343, 2019, https://doi.org/10.1088/1755-1315/343/1/012208. Accessed 8 Oct. 2020.

MLA 8 th edition vs MLA 9 th edition

The 9 th edition of the MLA handbook re-introduces guidelines regarding paper formatting (which were not present in the 8 th edition). The guidance in the 9 th addition is consistent with the guidance in previous editions and expands on the formatting of tables, figures/illustrations, and lists. The 9 th edition also offers new guidance in areas like annotated bibliographies, inclusive language, and footnotes/endnotes.

Many of the differences between the 8 th edition and 9 th edition have to do with the formatting of the core elements in reference list entries. Some of the main changes include:

Written by Grace Turney , freelance writer and artist. Grace is a former librarian and has a Master’s degree in Library Science and Information Technology. 

MLA Formatting Guide

MLA Formatting

Annotated Bibliography

Bibliography

• Block Quotes
• et al Usage
• In-text Citations
• Paraphrasing
• Page Numbers
• Sample Paper
• Works Cited
• MLA 8 Updates
• MLA 9 Updates
• View MLA Guide

Citation Examples

• Book Chapter
• Journal Article
• Magazine Article
• Newspaper Article
• Website (no author)
• View all MLA Examples

How useful was this post?

Click on a star to rate it!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

An MLA bibliography is similar to the Works Cited list that you include at the end of your paper. The only difference between a Works Cited list and a bibliography is that for the former, you need to include the entries for only the sources you cited in the text, whereas for the latter you can also include the sources you consulted to write your paper but didn’t directly cite in your writing. MLA generally prefers Works Cited lists to bibliographies.

If your instructor advises you to create an MLA bibliography, follow the same guidelines you would follow for creating an MLA Works Cited list.

The bibliography list appears at the end of the paper, after any endnotes if they are present.

All margins (top, bottom, left, and right) should be set at 1 inch.

Write the running head in the top right of the page at 0.5 inch from the top. Use the running head “Surname Page #.”

The font should be clear enough to read. Use Times New Roman font of size 12 points.

Entries should be double-spaced. If any entry runs over more than a line, indent the subsequent lines of the entry 0.5 inch from the left margin.

Bibliographic entries are arranged alphabetically according to the first item in each entry.

Title your bibliography as “Bibliography.”

Braidotti, Rosi. The Posthuman . Polity, 2013.

Brisini, Travis. “Phytomorphizing Performance: Plant Performance in an Expanded Field.” Text and Performance Quarterly , vol. 39, 2019,            pp. 1–2.

Riccio, Thomas. “Reimagining Yup’ik and Inupiat Performance.” Northwest Theatre Review , vol. 12, no. 1, 1999, pp. 1–30.

General rules for creating an annotated bibliography

The annotation is given after the source entry and is generally about 100-150 words in length. The annotation should be indented 1 inch from the left margin to distinguish it from the hanging indent within the citation entry.

The annotation, in general, should be written as short phrases. However, you may use full sentences as well.

The annotation for each source is usually no longer than one paragraph. However, if multiple paragraphs are included, indent the second and subsequent paragraphs without any extra line space between them.

The annotation provides basic information about the source, but does not include details about the source, quotes from the author, etc. The information can be descriptive (by generally describing what the source covers) or evaluative (by evaluating the source’s usefulness to the argument in your paper).

Example annotated bibliography

The below is an example of an annotated bibliography:

Morritt, Robert D. Beringia: Archaic Migrations into North America . Cambridge Scholars Pub, 2011.

The author studies the migration of cultures from Asia to North America. The connection between the North American Athabaskan language family and Siberia is presented, together with comparisons and examinations of the implications of linguistics from anthropological, archaeological, and folklore perspectives. This book explores the origins of the earliest people in the Americas, including Siberian, Dene, and Navajo Creation myths; linguistic comparisons between Siberian Ket Navajo and Western Apache; and comparisons between indigenous groups that appear to share the same origin.

MLA Citation Examples

Writing Tools

Citation Generators

Other Citation Styles

Plagiarism Checker

Upload a paper to check for plagiarism against billions of sources and get advanced writing suggestions for clarity and style.

Get Started

How to Write a Research Paper: Annotated Bibliography

• Anatomy of a Research Paper
• Developing a Research Focus
• Background Research Tips
• Searching Tips
• Scholarly Journals vs. Popular Journals
• Thesis Statement
• Annotated Bibliography
• Citing Sources
• Evaluating Sources
• Literature Review
• Academic Integrity
• Scholarship as Conversation
• Understanding Fake News
• Data, Information, Knowledge

What is an Annotated Bibliography?

UMary Writing Center

Check out the resources available from the  Writing Center . 

Write an Annotated Bibliography

What is an annotated bibliography?

It is a list of citations for various books, articles, and other sources on a topic. 

An annotation is a short summary and/or critical evaluation of a source.

Annotated bibliographies answer the question: "What would be the most relevant, most useful, or most up-to-date sources for this topic?"

 Annotated bibliographies can be part of a larger research project, or can be a stand-alone report in itself. 

Annotation versus abstracts 

An abstract is a paragraph at the beginning of the paper that discusses the main point of the original work. They typically do not include evaluation comments. 

Annotations can either be descriptive or evaluative. The annotated bibliography looks like a works cited page but includes an annotation after each source cited. 

Types of Annotations: 

Descriptive Annotations: Focuses on description. Describes the source by answering the following questions. 

Who wrote the document?

What does the document discuss?

When and where was the document written? 

Why was the document produced?

How was it provided to the public?

Evaluative Annotations: Focuses on description and evaluation. Includes a summary and critically assess the work for accuracy, relevance, and quality. 

Evaluative annotations help you learn about your topic, develop a thesis statement, decide if a specific source will be useful for your assignment, and determine if there is enough valid information available to complete your project.

What does the annotation include?

Depending on your assignment and style guide, annotations may include some or all of the following information. 

• Should be no more than 150 words or 4 to 6 sentences long. 
• What is the main focus or purpose of the work?
• Who is the intended audience?
• ​How useful or relevant was the article to your topic?
• Was there any unique features that useful to you?
• What is the background and credibility of the author?
• What are any conclusions or observations that your reached about the article?

Which citation style to use?

There are many styles manuals with specific instructions on how to format your annotated bibliography. This largely depends on what your instructor prefers or your subject discipline. Check out our citation guides for more information. 

Additional Information

Why doesn't APA have an official APA-approved format for annotated bibliographies?

Always consult your instructor about the format of an annotated bibliography for your class assignments. These guides provide you with examples of various styles for annotated bibliographies and they may not be in the format required by your instructor. 

Citation Examples and Annotations

Book Citation with Descriptive Annotation

Liroff, R. A., & G. G. Davis. (1981). Protecting open space: Land use control in the Adirondack Park. Cambridge, MA: Ballinger.

This book describes the implementation of regional planning and land use regulation in the Adirondack Park in upstate New York. The authors provide program evaluations of the Adirondack Park Agency’s regulatory and local planning assistance programs.

Journal Article Citation with Evaluative Annotation

Gottlieb, P. D. (1995). The “golden egg” as a natural resource: Toward a normative theory of growth management. Society and Natural Resources, 8, (5): 49-56.

This article explains the dilemma faced by North American suburbs, which demand both preservation of local amenities (to protect quality of life) and physical development (to expand the tax base). Growth management has been proposed as a policy solution to this dilemma. An analogy is made between this approach and resource economics. The author concludes that the growth management debate raises legitimate issues of sustainability and efficiency.

Examples were taken from http://lib.calpoly.edu/support/how-to/write-an-annotated-bibliography/#samples

Book Citation

Lee, Seok-hoon, Yong-pil Kim, Nigel Hemmington, and Deok-kyun Yun. “Competitive Service Quality Improvement (CSQI): A Case Study in the Fast-Food Industry.” Food Service Technology 4 (2004): 75-84.

In this highly technical paper, three industrial engineering professors in Korea and one services management professor in the UK discuss the mathematical limitations of the popular SERVQUAL scales. Significantly, they also aim to measure service quality in the fast-food industry, a neglected area of study. Unfortunately, the paper’s sophisticated analytical methods make it inaccessible to all but the most expert of researchers.

Battle, Ken. “Child Poverty: The Evolution and Impact of Child Benefits.”  A Question of Commitment: Children's Rights in Canada . Ed. Katherine Covell and R.Brian Howe. Waterloo, ON: Wilfrid Laurier University Press. 2007. 21-44.

             Ken Battle draws on a close study of government documents, as well as his own research as an extensively-published policy analyst, to explain Canadian child benefit programs.  He outlines some fundamental assumptions supporting the belief that all society members should contribute to the upbringing of children.  His comparison of child poverty rates in a number of countries is a useful wake-up to anyone assuming Canadian society is doing a good job of protecting children.  Battle pays particular attention to the National Child Benefit (NCB), arguing that it did not deserve to be criticized by politicians and journalists.  He outlines the NCB’s development, costs, and benefits, and laments that the Conservative government scaled it back in favour of the inferior Universal Child Care Benefit (UCCB).  However, he relies too heavily on his own work; he is the sole or primary author of almost half the sources in his bibliography.  He could make this work stronger by drawing from others' perspectives and analyses.  However, Battle does offer a valuable source for this essay, because the chapter provides a concise overview of government-funded assistance currently available to parents.  This offers context for analyzing the scope and financial reality of child poverty in Canada.

Journal Article Example

  Kerr, Don and Roderic Beaujot. “Child Poverty and Family Structure in Canada, 1981-1997.”  Journal of Comparative Family Studies  34.3 (2003): 321-335.

             Sociology professors Kerr and Beaujot analyze the demographics of impoverished families.  Drawing on data from Canada’s annual Survey of Consumer Finances, the authors consider whether each family had one or two parents, the age of single parents, and the number of children in each household.  They analyze child poverty rates in light of both these demographic factors and larger economic issues.  Kerr and Beaujot use this data to argue that. 

Examples were taken from  http://libguides.enc.edu/writing_basics/ annotatedbib/mla

Check out these resources for more information about Annotated Bibliographies. 

• Purdue Owl- Annotated Bibliographies
• University of North Carolina at Chapel Hill- Annotated Bibliographies
• << Previous: Thesis Statement
• Next: Citing Sources >>
• Last Updated: Feb 27, 2024 11:57 AM
• URL: https://libguide.umary.edu/researchpaper

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

Bibliography.

• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides

In addition to the sources cited in the individual text boxes of this writing guide, listed below are books about how to effectively write and organize a college-level research paper or dissertation [which frequently have good advice on writing, regardless of their purpose]. Enter the title of the book in the USC Libraries' search engine to check the availability of these or any other books on college-level writing and research. If the library does not own a particular title, you can request to borrow the book free of charge from the USC Libraries' interlibrary loan department .

• Abbott, Andrew. Digital Paper: A Manual for Research and Writing with Library and Internet Materials . Chicago, IL: University of Chicago Press, 2014.
• Anders, Marian. The Practical Grammar Handbook for College Writers . 2nd edition. Durham, NC: Carolina Academic Press, 2015.
• Aveyard, Helen. Doing a Literature Review in Health and Social Care: A Practical Guide . 3rd edition. New York: McGraw-Hill Education, 2014.
• Axelrod, Rise B. and Charles Raymond Cooper. The St. Martin's Guide to Writing . 10th edition. Boston, MA: Bedford/St. Martins, 2013.
• Badke, William. Research Strategies: Finding Your Way through the Information Fog . 5th edition. Bloomington, IN: iUniverse, LLC, 2014.
• Ballenger, Bruce P. The Curious Researcher: A Guide to Writing Research Papers . 8th edition. Boston, MA: Pearson, 2015.
• Barnet, Sylvan, Pat Bellanca, and Marcia Stubbs. A Short Guide to College Writing . 5th edition. Boston, MA: Pearson, 2013.
• Becker, Lucinda. Writing Successful Reports and Dissertations . Los Angeles, CA: Sage, 2014.
• Birkenstein, Cathy and Gerald Graff. “They Say/I Say”: The Moves That Matter in Academic Writing . 4th edition. New York: W.W. Norton and Company, 2018.
• Cahn, Steven M. and Victor Cahn. Polishing Your Prose: How to Turn First Drafts Into Finished Work . New York: Columbia University Press, 2013.

Carter, Caron, editor. Successful Dissertations: The Complete Guide for Education, Childhood and Early Childhood Studies Students . New York: Bloomsbury Academic, 2018.

• Channell, Carolyn. Engaging Questions: A Guide to Writing . 2nd edition. New York: McGraw-Hill Higher Education, 2016.
• Cioffi, Frank L. The Imaginative Argument: A Practical Manifesto for Writers . Princeton, NJ: Princeton University Press, 2017.
• Clark, Roy Peter. Help! for Writers: 210 Solutions to the Problems Every Writer Faces . New York: Little, Brown, 2011.
• Clouse, Barbara Fine. A Troubleshooting Guide for Writers: Strategies and Process . 7th edition. New York: McGraw-Hill, 2013.
• Cottrell, Stella. Dissertations and Project Reports: A Step by Step Guide . Basingstoke, UK: Palgrave Macmillan, 2014.
• Cutts, Martin. Oxford Guide to Plain English . 4th edition. Oxford, UK: Oxford University Press, 2013.
• Eby, Erika. The College Student's Guide to Writing a Great Research Paper: 101 Easy Tips and Tricks to Make Your Work Stand Out . Ocala, FL: Atlantic Pub. Group, 2012.
• Edwards, Mark. Writing in Sociology . Thousand Oaks, CA: Sage Publications, 2014.
• Evans, David, Paul Gruba, and Justin Zobel. How to Write a Better Thesis . 3rd edition. Melbourne, Australia: Melbourne University. Publishing, 2011.
• Faigley, Lester. Writing: A Guide for College and Beyond . 4th edition. Boston, MA: Pearson Higher Ed, 2016.
• Fiske, Robert Hartwell. To the Point: A Dictionary of Concise Writing . New York: W.W. Norton and Company, 2014.

Floyd, Randy G., editor. Publishing in School Psychology and Related Fields: An Insider's Guide . New York: Routledge, Taylor and Francis Group, 2018.

• Gelman, Howard. Everyone Can Write: A Guide to Get You Started . Wollombi, NSW, Australia: Exisle Publishing, 2014.
• Gill, Charlene. Essential Writing Skills for College and Beyond . Cincinnati, OH: Writer's Digest Books, 2014.
• Giltrow, Janet, Richard Gooding, Daniel Burgoyne, and Marlene Sawatsky. Academic Writing: An Introduction . 3rd edition. Tonawanda, NY: Broadview Press, 2014.
• Gibson, Twyla, and Mark Lipton. Research, Write, Create: Connecting Scholarship and Digital Media . Ontario, Canada: Oxford University Press, 2014.
• Glaser, Joe. Understanding Style: Practical Ways to Improve Your Writing . 3rd edition. New York: Oxford University Press, 2016.
• Goodson, Patricia. Becoming an Academic Writer: 50 Exercises for Paced, Productive, and Powerful Writing . 2nd edition. Thousand Oaks, CA: Sage Publications, 2016.

Graff, Gerald and Cathy Birkenstein. "They Say / I Say": The Moves that Matter in Academic Writing . New York: W.W. Norton and Company, 2017.

• Hacker, Diana and Nancy Sommers. Rules for Writers . 9th edition. Boston, MA: Bedford/St. Martin's, 2018.
• Hart, Chris. Doing a Literature Review: Releasing the Research Imagination . 2nd edition. Thousand Oaks, CA: Sage, 2018.
• Harvey, Michael. The Nuts and Bolts of College Writing . Indianapolis, IN: Hackett Publishing, 2013.
• Hjortshoj, Keith. The Transition to College Writing . Boston, MA: Bedford/St. Martins, 2009.
• Howard, Rebecca Moore. Writing Matters: A Handbook for Writing and Research . 2nd edition. New York: McGraw-Hill Higher Education, 2013.
• Johnson, William A. A Criminal Justice Student's Writer's Manual . 6th edition. Boston, MA: Pearson, 2015.
• Johnson Jr, William A., Gregory M. Scott, and Stephen M. Garrison. The Sociology Student Writer's Manual and Reader's Guide . 7th edition. Lanham, MD: Rowman and Littlefield, 2017.
• Joyner, Randy L., William A. Rouse, and Allan A. Glatthorn. Writing the Winning Thesis or Dissertation: A Step-by-step Guide . 3rd edition. Thousand Oaks, CA: Corwin Press, 2013.
• Kirszner, Laurie G. and Stephen R. Mandell. Patterns for College Writing: A Rhetorical Reader and Guide . 12th edition. Boston, MA: Bedford/St. Martins, 2012.
• Koerber, Duncan and Guy Allen. Clear, Precise, Direct: Strategies for Writing . Ontario, Canada: Oxford University Press, 2015.
• Langan, John. Exploring Writing: Paragraphs and Essays . 3rd edition. Dubuque, IA: McGraw-Hill, 2013.
• Lee, Kooi Cheng. Effective College Writing: A Process-Genre Approach . 2nd edition. Singapore: McGraw-Hill, 2009.
• Lerych, Lynne and Allison DeBoer Criswell. Everything You Need to Know about College Writing . Boston, MA: Bedford/St. Martin's, 2016.
• Lester, James D. and James D. Lester. Writing Research Papers: A Complete Guide . 14th edition. Boston, MA: Pearson, 2012.
• Lindemann, Kurt. Composing Research, Communicating Results: Writing the Communication Research Paper . Hoboken, NJ: John Wiley and Sons, 2017.
• Marsen, Sky. Professional Writing . 3rd edition. New York: Palgrave Macmillan, 2013.
• McMillan, Kathleen and Jonathan Weyers. How to Improve Your Critical Thinking and Reflective Skills . Harlow, UK: Pearson Education Limited, 2013.
• McNiff, Jean. Writing Up Your Action Research Project . New York: Routledge, 2015.
• Miller, Scott A. Writing in Psychology . New York: Routledge, 2014.
• Muller, Jake. Writing in the Social Sciences: A Guide for Term Papers and Book Reviews . 2nd edition. Ontario, Canada: Oxford University Press Canada, 2015.
• Northey, Margot, Dianne Draper, and David B. Knight. Making Sense: A Student's Guide to Research and Writing: Geography and Environmental Sciences . 6th edition. Oxford: Oxford University Press, 2015.
• Oliver, Paul. Writing Your Thesis . 3rd edition. London: Sage, 2013.
• Parsons, Tony, and Peter G. Knight. How to Do Your Dissertation in Geography and Related Disciplines . New York: Routledge, 2015.
• Pyrczak, Fred. Writing Empirical Research Reports: A Basic Guide for Students of the Social and Behavioral Sciences . 8th edition. Glendale, CA: Pyrczak Publishing, 2014.
• Raimes, Ann and Susan K. Miller-Cochran. Keys for Writers . 7th edition. Boston, MA: Wadsworth Publishing, 2014.
• Reinking, James A. Strategies for Successful Writing: A Rhetoric, Research Guide, Reader, and Handbook . 1oth edition. Boston, MA: Pearson, 2014.
• Reynolds, Nedra and Elizabeth Davis. Portfolio Keeping: A Guide for Students. 3rd edition. Boston, MA: Bedford/St. Martins, 2013.
• Ridley, Diana. The Literature Review: A Step-by-step Guide for Students . 2nd edition. Los Angeles, CA: Sage, 2012.
• Roen, Duane, Gregory Glau, and Barry Maid. The McGraw-Hill Guide: Writing for College, Writing for Life . 4th edition. New York: McGraw-Hill Education, 2018.
• Rudestam, Kjell Erik and Rae R. Newton. Surviving Your Dissertation: A Comprehensive Guide to Content and Process . 4th edition. Thousand Oaks, CA: Sage Publications, 2014.
• Scott, Gregory M. and Stephen M. Garrison. The Political Science Student Writer's Manual and Reader's Guide . 8th edition. Lanham, MD: Rowman and Littlefield, 2017.
• Spencer, Linda. Writing Well in the 21st Century: The Five Essentials . Lanham, MD: Rowman and Littlefield, 2014.
• Starr, Douglas Perret and Deborah Williams Dunsford. Working the Story: A Guide to Reporting and News Writing for Journalists and Public Relations Professionals . Lanham, MD: Rowman and Littlefield, 2014.
• Szuchman, Lenore T. Writing with Style: APA Style Made Easy . 6th edition. Belmont, CA: Wadsworth Cengage Learning, 2014.
• Thurman, Susan and Larry Shea. The Only Grammar Book You'll Ever Need: A One-Stop Source for Every Writing Assignment . Avon, MA: Adams Media, 2015.
• Turabian, Kate L. A Manual for Writers of Research Papers, Theses, and Dissertations: Chicago Style for Students and Researchers . 9th edition. Chicago, IL: University of Chicago Press, 2018.
• Wallwork, Adrian. English for Research: Usage, Style, and Grammar . New York: Springer Science and Business Media, 2012.
• Wang, Gabe T. and Keumjae Park. Student Research and Report Writing: From Topic Selection to the Complete Paper . Malden, MA: Wiley Blackwell, 2016.
• Warner, John. The Writer's Practice: Building Confidence in Your Non-Fiction Writing . New York: Penguin/Random House, 2019.
• Watt, Jane. Report Writing for Social Workers . Los Angeles, CA: Sage, 2012.
• Wyrick, Jean. Steps to Writing Well . 12th edition. Boston, MA: Cengage Learning, 2014.
• Wyse, Dominic and Kate Cowan. The Good Writing Guide for Education Students . 4th edition. London: Sage, 2017.
• Yagoda, Ben. How to Not Write Bad: The Most Common Writing Problems and the Best Ways to Avoid Them . New York: Riverhead Books, 2013.
• << Previous: USC Libraries Tutorials and Other Guides
• Next: Last Word >>
• Last Updated: Mar 26, 2024 10:40 AM
• URL: https://libguides.usc.edu/writingguide

authorsafterdark.org

research paper writing tips from the top experts

How to Write a Bibliography for a Research Paper: Guide & Samples

A bibliography is undoubtedly one of the most essential parts of every research paper. A research paper without a bibliography equals poorly conducted research. As such, the importance of a bibliography in a research paper cannot be overemphasized.

To produce well-constructed research, you must consult other authors and use their materials to support your argument. This is the only way to give these authors their credit; by listing them out in your bibliography.

Acknowledging the sources used in writing the topic is an effective approach to avoiding plagiarism in your research. It is also a great way to provide and inform other scholars with sources they may not be aware of. This article seeks to provide the format of bibliography and how to write a good bibliography for your research paper.

What Is a Bibliography in a Research Paper?

Simply put, a bibliography is a list of works used in writing a research paper. Every research paper must contain a list of sources the author used in preparing the research paper. Your source can range from books to scholarly papers, speeches, private records, interviews, letters, websites, and other sources.

Your research paper was not created in a vacuum; you must have consulted other authors or sourced to create enriched content. Therefore, at the end of your research, you must attach a list of all the sources used.

How to Write Bibliography for Research Paper

Writing a bibliography isn’t so hard; all you need is a list of sources used and a format for documenting them. To make things easier for you, prepare a draft comprising all of the sources you must have used. Ensure you include the book’s full title, the author, place of publication, publication date, and publisher. All sources must be listed out alphabetically using the authors’ names.

It is also important to note that there are different formats for writing a bibliography. As such, you must decide which format to employ in your research paper. You can format your work in Modern Language Association (MLA) format, American Psychological Association (APA), or Chicago Manual of Style (CMS). Regardless of which you choose, ensure that it is done correctly.

How to Format a Bibliography

Once you have an idea of how to format a bibliography, most of the work is completed. However, when writing it you need to pay close attention to each format and its content to avoid mixing them up. For printed sources, the bibliography of a research paper should provide the following:

• Author’s name
• Title of publication (provide the title of the article if it’s a journal, magazine, or encyclopedia)
• Date of publication
• Place of Publication of a book
• The publishing house of a book
• Volume number of magazine or encyclopedia
• The page numbers

To document sources from a website, you need to document the following:

• The author or editor’s name if there’s any
• Title of the page
• The organization of the webpage
• The URL of the website
• The date the information was gotten from the website

The common formats for writing a bibliography in a research paper are MLA and APA style. A bibliography is known as “Works Cited” in MLA and on the other hand, it is called “References List” in APA. Though both formats contain similar information about the sources used, there are still slight differences in formatting style. Here’s  what MLA and APA bibliography styles should look like.

How to Write a Bibliography APA

Your research paper’s bibliography must be attached at the end of it with the tag “References” at the center. In addition to this, you need to pay attention to the basics of APA style, such as capitalization, abbreviation, punctuation, underlining or italics, hanging indentation, and others. Here’s a research paper bibliography example in APA style:

Author’s last name, first initial. (Publication date). Book title . City of Publication: Publishing company.

• For encyclopedia:

Author’s last name, first initial. (Date). Title of Article. Title of Encyclopedia (Volumes, pages). City of Publication: Publishing Company.

• For magazine and newspaper articles:

Author’s last name, first initial. (Publication Date). Article title. Periodical title, Volume number (issue number if there’s any), page numbers.

How to Write a Bibliography MLA

Bibliography in MLA format is called “Works cited” and must be arranged in alphabetical order according to the author’s name. Here’s the required format for MLA style for varying sources:

“Author’s last name, first name. Book title. Publication City: Publishing company, publication date.”

• Encyclopedia & dictionary:

“Author’s last name, first name. “Title of Article.” Encyclopedia’s Title. Date.”

• Magazine & newspaper:

“Author’s last name, first name. “Article title.” Periodical title volume Date: inclusive pages.”

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

• Grades 6-12
• School Leaders

FREE Book Bracket Template. For March and Beyond!

How To Write a Bibliography (Three Styles, Plus Examples)

Give credit where credit is due.

Writing a research paper involves a lot of work. Students need to consult a variety of sources to gather reliable information and ensure their points are well supported. Research papers include a bibliography, which can be a little tricky for students. Learn how to write a bibliography in multiple styles and find basic examples below.

IMPORTANT: Each style guide has its own very specific rules, and they often conflict with one another. Additionally, each type of reference material has many possible formats, depending on a variety of factors. The overviews shown here are meant to guide students in writing basic bibliographies, but this information is by no means complete. Students should always refer directly to the preferred style guide to ensure they’re using the most up-to-date formats and styles.

What is a bibliography?

When you’re researching a paper, you’ll likely consult a wide variety of sources. You may quote some of these directly in your work, summarize some of the points they make, or simply use them to further the knowledge you need to write your paper. Since these ideas are not your own, it’s vital to give credit to the authors who originally wrote them. This list of sources, organized alphabetically, is called a bibliography.

A bibliography should include all the materials you consulted in your research, even if you don’t quote directly from them in your paper. These resources could include (but aren’t limited to):

• Books and e-books
• Periodicals like magazines or newspapers
• Online articles or websites
• Primary source documents like letters or official records

Bibliography vs. References

These two terms are sometimes used interchangeably, but they actually have different meanings. As noted above, a bibliography includes all the materials you used while researching your paper, whether or not you quote from them or refer to them directly in your writing.

A list of references only includes the materials you cite throughout your work. You might use direct quotes or summarize the information for the reader. Either way, you must ensure you give credit to the original author or document. This section can be titled “List of Works Cited” or simply “References.”

Your teacher may specify whether you should include a bibliography or a reference list. If they don’t, consider choosing a bibliography, to show all the works you used in researching your paper. This can help the reader see that your points are well supported, and allow them to do further reading on their own if they’re interested.

Bibliography vs. Citations

Citations refer to direct quotations from a text, woven into your own writing. There are a variety of ways to write citations, including footnotes and endnotes. These are generally shorter than the entries in a reference list or bibliography. Learn more about writing citations here.

What does a bibliography entry include?

Depending on the reference material, bibliography entries include a variety of information intended to help a reader locate the material if they want to refer to it themselves. These entries are listed in alphabetical order, and may include:

• Author/s or creator/s
• Publication date
• Volume and issue numbers
• Publisher and publication city
• Website URL

These entries don’t generally need to include specific page numbers or locations within the work (except for print magazine or journal articles). That type of information is usually only needed in a footnote or endnote citation.

What are the different bibliography styles?

In most cases, writers use one of three major style guides: APA (American Psychological Association), MLA (Modern Language Association), or The Chicago Manual of Style . There are many others as well, but these three are the most common choices for K–12 students.

Many teachers will state their preference for one style guide over another. If they don’t, you can choose your own preferred style. However, you should also use that guide for your entire paper, following their recommendations for punctuation, grammar, and more. This will ensure you are consistent throughout.

Below, you’ll learn how to write a simple bibliography using each of the three major style guides. We’ve included details for books and e-books, periodicals, and electronic sources like websites and videos. If the reference material type you need to include isn’t shown here, refer directly to the style guide you’re using.

APA Style Bibliography and Examples

Source: Verywell Mind

Technically, APA style calls for a list of references instead of a bibliography. If your teacher requires you to use the APA style guide , you can limit your reference list only to items you cite throughout your work.

How To Write a Bibliography (References) Using APA Style

Here are some general notes on writing an APA reference list:

• Title your bibliography section “References” and center the title on the top line of the page.
• Do not center your references; they should be left-aligned. For longer items, subsequent lines should use a hanging indent of 1/2 inch.
• Include all types of resources in the same list.
• Alphabetize your list by author or creator, last name first.
• Do not spell out the author/creator’s first or middle name; only use their initials.
• If there are multiple authors/creators, use an ampersand (&) before the final author/creator.
• Place the date in parentheses.
• Capitalize only the first word of the title and subtitle, unless the word would otherwise be capitalized (proper names, etc.).
• Italicize the titles of books, periodicals, or videos.
• For websites, include the full site information, including the http:// or https:// at the beginning.

Books and E-Books APA Bibliography Examples

For books, APA reference list entries use this format (only include the publisher’s website for e-books).

Last Name, First Initial. Middle Initial. (Publication date). Title with only first word capitalized . Publisher. Publisher’s website

• Wynn, S. (2020). City of London at war 1939–45 . Pen & Sword Military. https://www.pen-and-sword.co.uk/City-of-London-at-War-193945-Paperback/p/17299

Periodical APA Bibliography Examples

For journal or magazine articles, use this format. If you viewed the article online, include the URL at the end of the citation.

Last Name, First Initial. Middle Initial. (Publication date). Title of article. Magazine or Journal Title (Volume number) Issue number, page numbers. URL

• Bell, A. (2009). Landscapes of fear: Wartime London, 1939–1945. Journal of British Studies (48) 1, 153–175. https://www.jstor.org/stable/25482966

Here’s the format for newspapers. For print editions, include the page number/s. For online articles, include the full URL.

Last Name, First Initial. Middle Initial. (Year, Month Date) Title of article. Newspaper title. Page number/s. URL

• Blakemore, E. (2022, November 12) Researchers track down two copies of fossil destroyed by the Nazis.  The Washington Post. https://www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/

Electronic APA Bibliography Examples

For articles with a specific author on a website, use this format.

Last Name, First Initial. Middle Initial. (Year, Month Date). Title . Site name. URL

• Wukovits, J. (2023, January 30). A World War II survivor recalls the London Blitz . British Heritage . https://britishheritage.com/history/world-war-ii-survivor-london-blitz

When an online article doesn’t include a specific author or date, list it like this:

Title . (Year, Month Date). Site name. Retrieved Month Date, Year, from URL

• Growing up in the Second World War . (n.d.). Imperial War Museums. Retrieved May 12, 2023, from https://www.iwm.org.uk/history/growing-up-in-the-second-world-war

When you need to list a YouTube video, use the name of the account that uploaded the video, and format it like this:

Name of Account. (Upload year, month day). Title [Video]. YouTube. URL

• War Stories. (2023, January 15). How did London survive the Blitz during WW2? | Cities at war: London | War stories [Video]. YouTube. https://youtu.be/uwY6JlCvbxc

For more information on writing APA bibliographies, see the APA Style Guide website.

APA Bibliography (Reference List) Example Pages

Source: Simply Psychology

More APA example pages:

• Western Australia Library Services APA References Example Page
• Ancilla College APA References Page Example
• Scribbr APA References Page Example

MLA Style Bibliography Examples

Source: PressBooks

MLA style calls for a Works Cited section, which includes all materials quoted or referred to in your paper. You may also include a Works Consulted section, including other reference sources you reviewed but didn’t directly cite. Together, these constitute a bibliography. If your teacher requests an MLA Style Guide bibliography, ask if you should include Works Consulted as well as Works Cited.

How To Write a Bibliography (Works Cited and Works Consulted) in MLA Style

For both MLA Works Cited and Works Consulted sections, use these general guidelines:

• Start your Works Cited list on a new page. If you include a Works Consulted list, start that on its own new page after the Works Cited section.
• Center the title (Works Cited or Works Consulted) in the middle of the line at the top of the page.
• Align the start of each source to the left margin, and use a hanging indent (1/2 inch) for the following lines of each source.
• Alphabetize your sources using the first word of the citation, usually the author’s last name.
• Include the author’s full name as listed, last name first.
• Capitalize titles using the standard MLA format.
• Leave off the http:// or https:// at the beginning of a URL.

Books and E-Books MLA Bibliography Examples

For books, MLA reference list entries use this format. Add the URL at the end for e-books.

Last Name, First Name Middle Name. Title . Publisher, Date. URL

• Wynn, Stephen. City of London at War 1939–45 . Pen & Sword Military, 2020. www.pen-and-sword.co.uk/City-of-London-at-War-193945-Paperback/p/17299

Periodical MLA Bibliography Examples

Here’s the style format for magazines, journals, and newspapers. For online articles, add the URL at the end of the listing.

For magazines and journals:

Last Name, First Name. “Title: Subtitle.” Name of Journal , volume number, issue number, Date of Publication, First Page Number–Last Page Number.

• Bell, Amy. “Landscapes of Fear: Wartime London, 1939–1945.” Journal of British Studies , vol. 48, no. 1, pp. 153–175. www.jstor.org/stable/25482966

When citing newspapers, include the page number/s for print editions or the URL for online articles.

Last Name, First Name. “Title of article.” Newspaper title. Page number/s. Year, month day. Page number or URL

• Blakemore, Erin. “Researchers Track Down Two Copies of Fossil Destroyed by the Nazis.” The Washington Post. 2022, Nov. 12. www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/

Electronic MLA Bibliography Examples

Last Name, First Name. Year. “Title.” Month Day, Year published. URL

• Wukovits, John. 2023. “A World War II Survivor Recalls the London Blitz.” January 30,   2023. https://britishheritage.com/history/world-war-ii-survivor-london-blitz

Website. n.d. “Title.” Accessed Day Month Year. URL.

• Imperial War Museum. n.d. “Growing Up in the Second World War.” Accessed May 9, 2023. https://www.iwm.org.uk/history/growing-up-in-the-second-world-war.

Here’s how to list YouTube and other online videos.

Creator, if available. “Title of Video.” Website. Uploaded by Username, Day Month Year. URL.

• “How did London survive the Blitz during WW2? | Cities at war: London | War stories.” YouTube . Uploaded by War Stories, 15 Jan. 2023. youtu.be/uwY6JlCvbxc.

For more information on writing MLA style bibliographies, see the MLA Style website.

MLA Bibliography (Works Cited) Example Pages

Source: The Visual Communication Guy

More MLA example pages:

• Writing Commons Sample Works Cited Page
• Scribbr MLA Works Cited Sample Page
• Montana State University MLA Works Cited Page

Chicago Manual of Style Bibliography Examples

The Chicago Manual of Style (sometimes called “Turabian”) actually has two options for citing reference material : Notes and Bibliography and Author-Date. Regardless of which you use, you’ll need a complete detailed list of reference items at the end of your paper. The examples below demonstrate how to write that list.

How To Write a Bibliography Using The Chicago Manual of Style

Source: South Texas College

Here are some general notes on writing a Chicago -style bibliography:

• You may title it “Bibliography” or “References.” Center this title at the top of the page and add two blank lines before the first entry.
• Left-align each entry, with a hanging half-inch indent for subsequent lines of each entry.
• Single-space each entry, with a blank line between entries.
• Include the “http://” or “https://” at the beginning of URLs.

Books and E-Books Chicago Manual of Style Bibliography Examples

For books, Chicago -style reference list entries use this format. (For print books, leave off the information about how the book was accessed.)

Last Name, First Name Middle Name. Title . City of Publication: Publisher, Date. How e-book was accessed.

• Wynn, Stephen. City of London at War 1939–45 . Yorkshire: Pen & Sword Military, 2020. Kindle edition.

Periodical Chicago Manual of Style Bibliography Examples

For journal and magazine articles, use this format.

Last Name, First Name. Year of Publication. “Title: Subtitle.” Name of Journal , Volume Number, issue number, First Page Number–Last Page Number. URL.

• Bell, Amy. 2009. “Landscapes of Fear: Wartime London, 1939–1945.” Journal of British Studies, 48 no. 1, 153–175. https://www.jstor.org/stable/25482966.

When citing newspapers, include the URL for online articles.

Last Name, First Name. Year of Publication. “Title: Subtitle.” Name of Newspaper , Month day, year. URL.

• Blakemore, Erin. 2022. “Researchers Track Down Two Copies of Fossil Destroyed by the Nazis.” The Washington Post , November 12, 2022. https://www.washingtonpost.com/science/2022/11/12/ichthyosaur-fossil-images-discovered/.

Electronic Chicago Manual of Style Bibliography Examples

Last Name, First Name Middle Name. “Title.” Site Name . Year, Month Day. URL.

• Wukovits, John. “A World War II Survivor Recalls the London Blitz.” British Heritage. 2023, Jan. 30. britishheritage.com/history/world-war-ii-survivor-london-blitz.

“Title.” Site Name . URL. Accessed Day Month Year.

• “Growing Up in the Second World War.” Imperial War Museums . www.iwm.org.uk/history/growing-up-in-the-second-world-war. Accessed May 9, 2023.

Creator or Username. “Title of Video.” Website video, length. Month Day, Year. URL.

• War Stories. “How Did London Survive the Blitz During WW2? | Cities at War: London | War Stories.” YouTube video, 51:25. January 15, 2023. https://youtu.be/uwY6JlCvbxc.

For more information on writing Chicago -style bibliographies, see the Chicago Manual of Style website.

Chicago Manual of Style Bibliography Example Pages

Source: Chicago Manual of Style

More Chicago example pages:

• Scribbr Chicago Style Bibliography Example
• Purdue Online Writing Lab CMOS Bibliography Page
• Bibcitation Sample Chicago Bibliography

Now that you know how to write a bibliography, take a look at the Best Websites for Teaching & Learning Writing .

Plus, get all the latest teaching tips and ideas when you sign up for our free newsletters .

You Might Also Like

Sign Up For Our Job Hunting Course!

Just for teachers! Continue Reading

Copyright © 2023. All rights reserved. 5335 Gate Parkway, Jacksonville, FL 32256

• Utility Menu

fa3d988da6f218669ec27d6b6019a0cd

A publication of the harvard college writing program.

Harvard Guide to Using Sources 

• The Honor Code
• Sample Bibliography

Below you’ll find a Bibliography adapted from a research paper written by Aishani Aatresh for her Technology, Environment, and Society course.

• Citation Management Tools
• In-Text Citations
• Bibliography
• Examples of Commonly Cited Sources
• Frequently Asked Questions about Citing Sources in Chicago Format

PDFs for This Section

• Citing Sources
• Online Library and Citation Tools

How To Write A Research Paper

Find Sources For A Research Paper

How to Find Sources For a Research Paper | A Guide

10 min read

Published on: Mar 26, 2024

Last updated on: Mar 25, 2024

People also read

How to Write a Research Paper Step by Step

How to Write a Proposal For a Research Paper in 10 Steps

A Comprehensive Guide to Creating a Research Paper Outline

Types of Research - Methodologies and Characteristics

300+ Engaging Research Paper Topics to Get You Started

Interesting Psychology Research Topics & Ideas

Qualitative Research - Types, Methods & Examples

Understanding Quantitative Research - Definition, Types, Examples, And More

Research Paper Example - Examples for Different Formats

How To Start A Research Paper - Steps With Examples

How to Write an Abstract That Captivates Your Readers

How To Write a Literature Review for a Research Paper | Steps & Examples

Types of Qualitative Research Methods - An Overview

Understanding Qualitative vs. Quantitative Research - A Complete Guide

How to Cite a Research Paper in Different Citation Styles

Easy Sociology Research Topics for Your Next Project

200+ Outstanding History Research Paper Topics With Expert Tips

How To Write a Hypothesis in a Research Paper | Steps & Examples

How to Write an Introduction for a Research Paper - A Step-by-Step Guide

How to Write a Good Research Paper Title

How to Write a Conclusion for a Research Paper in 3 Simple Steps

How to Write an Abstract For a Research Paper with Examples

How To Write a Thesis For a Research Paper Step by Step

How to Write a Discussion For a Research Paper | Objectives, Steps & Examples

How to Write the Results Section of a Research Paper - Structure and Tips

How to Write a Problem Statement for a Research Paper in 6 Steps

How To Write The Methods Section of a Research Paper Step-by-Step

Share this article

Research papers are an essential part of academic life, but one of the most challenging aspects can be finding credible sources to support your arguments. 

With the vast amount of information available online, it's easy to feel overwhelmed. However, by following some simple steps, you can streamline the process of finding reliable sources for your research paper . 

In this guide, we'll break down the process into easy-to-follow steps to help you find the best sources for your paper.

On This Page On This Page -->

Step 1: Define Your Topic and Research Questions

Before you venture into your quest for sources, it's essential to have a clear understanding of your research topic and the specific questions you aim to address. Define the scope of your paper and identify keywords and key concepts that will guide your search for relevant sources.

Step 2: Utilize Academic Databases

Academic databases are treasure troves of scholarly articles, research papers, and academic journals covering a wide range of subjects. Institutions often provide access to these databases through their libraries. Some popular academic databases include:

• IEEE Xplore
• Google Scholar

These databases allow you to search for peer-reviewed articles and academic papers related to your topic. 

Use advanced search features to narrow down your results based on publication date, author, and keywords .

Academic Resources Classified by Discipline

Here's a breakdown of prominent databases categorized by academic discipline:

Step 3: Explore Library Catalogs

Your university or local library's catalog is another valuable resource for finding sources. Library catalogs contain books, periodicals, and other materials that may not be available online. 

Use the catalog's search function to locate relevant books, journals, and other materials that can contribute to your research.

Step 4: Consult Bibliographies and References

When you find a relevant source, take note of its bibliography or make a list of sources for the research paper. These lists often contain citations to other works that may be useful for your research. 

By exploring the references cited in a particular source, you can uncover additional resources and expand your understanding of the topic.

Step 5: Boolean Operators for Effective Searches

Boolean operators are words or symbols used to refine search queries by defining the relationships between search terms. The three primary operators include "AND," which narrows searches by requiring all terms to be present; "OR," which broadens searches by including either term or both; and "NOT," which excludes specific terms to refine results further. 

Most databases provide advanced search features for seamless application of Boolean logic.

Step 6: Consider Primary Sources 

Depending on your research topic, primary sources such as interviews, surveys, archival documents, and original data sets can provide valuable insights and support for your arguments. 

Primary sources offer firsthand accounts and original perspectives on historical events, social phenomena, and scientific discoveries.

Step 7: Evaluate the Credibility of Sources

Not all sources are created equal, and it's crucial to evaluate the credibility and reliability of the information you encounter. 

Consider the author's credentials, the publication venue, and whether the source is peer-reviewed. Look for evidence of bias or conflicts of interest that may undermine the source's credibility.

Step 8: Keep Track of Your Sources

As you gather sources for your research paper, maintain a systematic record of the materials you consult.  Keep track of bibliographic information, including author names, publication dates, titles, and page numbers . This information will be invaluable when citing your sources and creating a bibliography or works cited page.

Other Online Sources

In addition to academic databases and library catalogs, exploring popular online sources can provide valuable insights and perspectives on your research topic.  Here are some types of online sources you can consider:

Websites hosted by reputable organizations, institutions, and experts (such as the New York Times) can offer valuable information and analysis on a wide range of topics. Look for websites belonging to universities, research institutions, government agencies, and established non-profit organizations.

Crowdsourced Encyclopedias like Wikipedia

While Wikipedia can provide a broad overview of a topic and lead you to other sources, it's essential to verify the information found there with more authoritative sources. 

Use Wikipedia as a starting point for your research, but rely on peer-reviewed journal articles and academic sources for in-depth analysis and evidence.

Tips for Assessing the Credibility of Online Sources

When using online sources, it's important to exercise caution and critically evaluate the credibility and reliability of the information you find. Here are some tips for assessing the credibility of online sources:

• Check the Domain Extension: Look for websites with domain extensions that indicate credibility. URLs ending in .edu are educational resources, while URLs ending in .gov are government-related resources. These sites often provide reliable and authoritative information.
• Look for DOIs (Digital Object Identifiers): DOIs are unique alphanumeric strings assigned to scholarly articles and indicate that the article has been published in a peer-reviewed, scientific journal. Finding a DOI can help you assess the scholarly rigor of the source.
• Evaluate the Authorship and Credentials: Consider the qualifications and expertise of the author or organization behind the website or blog. Look for information about the author's credentials, affiliations, and expertise in the subject matter.
• Consider the Currency and Relevance: Assess how up-to-date the information is and whether it aligns with the scope and focus of your research. Look for recent publications and timely analyses that reflect current trends and developments in the field.

Wrapping it up!

Finding sources for your research paper may seem like a challenge, but by following these steps, you can locate credible sources to support your arguments and enhance the quality of your paper. 

By approaching the research process systematically and critically evaluating the information you encounter, you can produce a well-researched and compelling research paper.

If you are struggling with finding credible sources or have time constraints, do not hesitate to seek writing help for your research papers . CollegeEssay.org has professional writers ready to assist you. 

Connect with our essay writing service now and receive expert guidance and support to elevate your research paper to the next level.

Cathy A. (Law)

For more than five years now, Cathy has been one of our most hardworking authors on the platform. With a Masters degree in mass communication, she knows the ins and outs of professional writing. Clients often leave her glowing reviews for being an amazing writer who takes her work very seriously.

Paper Due? Why Suffer? That’s our Job!

Keep reading

• Privacy Policy
• Cookies Policy
• Terms of Use
• Refunds & Cancellations
• Our Writers
• Success Stories
• Our Guarantees
• Affiliate Program
• Referral Program
• AI Essay Writer

Disclaimer: All client orders are completed by our team of highly qualified human writers. The essays and papers provided by us are not to be used for submission but rather as learning models only.

Write an annotated bibliography

• Annotations

Who is this guide for?

We think this guide will be most useful to first- and second-year students.

What is an annotated bibliography?

An annotated bibliography is a reference list, along with a short summary or evaluation (i.e. an annotation) of each source.

A few key points about annotated bibliographies:

Like other bibliographies, annotated bibliographies should be sorted alphabetically by the author's (or authors') last name.

The length of each annotation will depend on your professor's requirements. However, if they don't have a requirement, aim for somewhere between 100-200 words.

There are different types of annotations - we'll go over those in more detail on the next page. Again, your professor will probably explain what they expect in your annotations. If they do not, you may want to ask them which type they want you to write.

Why write one?

Why is my professor asking me to write an annotated bibliography.

Professors often ask you to write an annotated bibliography as one step of a larger research paper, so that you don't have to rush to get all your research done at the last minute.

Professors might ask you to write an annotated bibliography as a way to practice a particular citation style (MLA, APA, Chicago, etc.).

How will an annotated bibliography help me?

On a basic level, annotated bibliographies can help you:

Keep track of what you read, which sources you found useful (or not), and why

Keep track of the citations in case you need to find the sources again

This is probably obvious, but in order to write good annotations, you need to read and understand your sources. Once you do, that knowledge can help you:

Decide on a thesis for your research paper

Help you form the structure of your paper - here are a few ideas to think about:

What are some of the common themes you came across in your reading?

What do your sources agree about? What do they disagree about?

How did the authors test their hypotheses? Did they use different methods than other authors?

Are there things you feel the authors should have talked about, but didn't?

What do the authors think needs to be studied in the future?

• Next: Annotations >>
• Last Updated: Mar 19, 2024 10:36 AM
• URL: https://libraryguides.oswego.edu/annotated-bib

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: mm1: methods, analysis & insights from multimodal llm pre-training.

Abstract: In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons. For example, we demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state-of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published pre-training results. Further, we show that the image encoder together with image resolution and the image token count has substantial impact, while the vision-language connector design is of comparatively negligible importance. By scaling up the presented recipe, we build MM1, a family of multimodal models up to 30B parameters, including both dense models and mixture-of-experts (MoE) variants, that are SOTA in pre-training metrics and achieve competitive performance after supervised fine-tuning on a range of established multimodal benchmarks. Thanks to large-scale pre-training, MM1 enjoys appealing properties such as enhanced in-context learning, and multi-image reasoning, enabling few-shot chain-of-thought prompting.

Submission history

Access paper:.

• Download PDF
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Scribbr Citation Generator

Accurate APA, MLA, Chicago, and Harvard citations, verified by experts, trusted by millions

Scribbr for Chrome: Your shortcut to citations

Cite any page or article with a single click right from your browser. The extension does the hard work for you by automatically grabbing the title, author(s), publication date, and everything else needed to whip up the perfect citation.

Perfectly formatted references every time

Inaccurate citations can cost you points on your assignments, so our seasoned citation experts have invested countless hours in perfecting Scribbr’s citation generator algorithms. We’re proud to be recommended by teachers and universities worldwide.

Enjoy a citation generator without flashy ads

Staying focused is already difficult enough, so unlike other citation generators, Scribbr won’t slow you down with flashing banner ads and video pop-ups. That’s a promise!

Citation Generator features you'll love

Look up your source by its title, URL, ISBN, or DOI, and let Scribbr find and fill in all the relevant information automatically.

APA, MLA, Chicago, and Harvard

Generate flawless citations according to the official APA, MLA, Chicago, Harvard style, or many other rules.

Export to Word

When your reference list is complete, export it to Word. We’ll apply the official formatting guidelines automatically.

Lists and folders

Create separate reference lists for each of your assignments to stay organized. You can also group related lists into folders.

Export to Bib(La)TeX

Are you using a LaTex editor like Overleaf? If so, you can easily export your references in Bib(La)TeX format with a single click.

Custom fonts

Change the typeface used for your reference list to match the rest of your document. Options include Times New Roman, Arial, and Calibri.

Industry-standard technology

Scribbr’s Citation Generator is built using the same citation software (CSL) as Mendeley and Zotero, but with an added layer for improved accuracy.

Annotations

Describe or evaluate your sources in annotations, and Scribbr will generate a perfectly formatted annotated bibliography .

Citation guides

Scribbr’s popular guides and videos will help you understand everything related to finding, evaluating, and citing sources.

Secure backup

Your work is saved automatically after every change and stored securely in your Scribbr account.

• Introduction
• Finding sources

Evaluating sources

• Integrating sources

Citing sources

Tools and resources, a quick guide to working with sources.

Working with sources is an important skill that you’ll need throughout your academic career.

It includes knowing how to find relevant sources, assessing their authority and credibility, and understanding how to integrate sources into your work with proper referencing.

This quick guide will help you get started!

Finding relevant sources

Sources commonly used in academic writing include academic journals, scholarly books, websites, newspapers, and encyclopedias. There are three main places to look for such sources:

• Research databases: Databases can be general or subject-specific. To get started, check out this list of databases by academic discipline . Another good starting point is Google Scholar .
• Your institution’s library: Use your library’s database to narrow down your search using keywords to find relevant articles, books, and newspapers matching your topic.
• Other online resources: Consult popular online sources like websites, blogs, or Wikipedia to find background information. Be sure to carefully evaluate the credibility of those online sources.

When using academic databases or search engines, you can use Boolean operators to refine your results.

Generate APA, MLA, Chicago, and Harvard citations in seconds

Get started

In academic writing, your sources should be credible, up to date, and relevant to your research topic. Useful approaches to evaluating sources include the CRAAP test and lateral reading.

CRAAP is an abbreviation that reminds you of a set of questions to ask yourself when evaluating information.

• Currency: Does the source reflect recent research?
• Relevance: Is the source related to your research topic?
• Authority: Is it a respected publication? Is the author an expert in their field?
• Accuracy: Does the source support its arguments and conclusions with evidence?
• Purpose: What is the author’s intention?

Lateral reading

Lateral reading means comparing your source to other sources. This allows you to:

• Verify evidence
• Contextualize information
• Find potential weaknesses

If a source is using methods or drawing conclusions that are incompatible with other research in its field, it may not be reliable.

Integrating sources into your work

Once you have found information that you want to include in your paper, signal phrases can help you to introduce it. Here are a few examples:

Following the signal phrase, you can choose to quote, paraphrase or summarize the source.

• Quoting : This means including the exact words of another source in your paper. The quoted text must be enclosed in quotation marks or (for longer quotes) presented as a block quote . Quote a source when the meaning is difficult to convey in different words or when you want to analyze the language itself.
• Paraphrasing : This means putting another person’s ideas into your own words. It allows you to integrate sources more smoothly into your text, maintaining a consistent voice. It also shows that you have understood the meaning of the source.
• Summarizing : This means giving an overview of the essential points of a source. Summaries should be much shorter than the original text. You should describe the key points in your own words and not quote from the original text.

Whenever you quote, paraphrase, or summarize a source, you must include a citation crediting the original author.

Citing your sources is important because it:

• Allows you to avoid plagiarism
• Establishes the credentials of your sources
• Backs up your arguments with evidence
• Allows your reader to verify the legitimacy of your conclusions

The most common citation styles are APA, MLA, and Chicago style. Each citation style has specific rules for formatting citations.

Generate APA, MLA, Chicago,  and Harvard citations in seconds

Scribbr offers tons of tools and resources to make working with sources easier and faster. Take a look at our top picks:

• Citation Generator: Automatically generate accurate references and in-text citations using Scribbr’s APA Citation Generator, MLA Citation Generator , Harvard Referencing Generator , and Chicago Citation Generator .
• Plagiarism Checker : Detect plagiarism in your paper using the most accurate Turnitin-powered plagiarism software available to students.
• AI Proofreader: Upload and improve unlimited documents and earn higher grades on your assignments. Try it for free!
• Paraphrasing tool: Avoid accidental plagiarism and make your text sound better.
• Grammar checker : Eliminate pesky spelling and grammar mistakes.
• Summarizer: Read more in less time. Distill lengthy and complex texts down to their key points.
• AI detector: Find out if your text was written with ChatGPT or any other AI writing tool. ChatGPT 2 & ChatGPT 3 supported.
• Proofreading services : Have a human editor improve your writing.
• Citation checker: Check your work for citation errors and missing citations.
• Knowledge Base : Explore hundreds of articles, bite-sized videos, time-saving templates, and handy checklists that guide you through the process of research, writing, and citation.

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Fatal Traffic Risks With a Total Solar Eclipse in the US

• 1 Department of Medicine, University of Toronto, Toronto, Ontario, Canada
• 2 Evaluative Clinical Science Platform, Sunnybrook Research Institute, Toronto, Ontario, Canada
• 3 Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
• 4 Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
• 5 Center for Leading Injury Prevention Practice Education & Research, Toronto, Ontario, Canada
• 6 Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
• 7 Centre for Clinical Epidemiology & Evaluation, University of British Columbia, Vancouver, British Columbia, Canada

A total solar eclipse occurs when the moon temporarily obscures the sun and casts a dark shadow across the earth. This astronomical spectacle has been described for more than 3 millennia and can be predicted with high precision. Eclipse-related solar retinopathy (vision loss from staring at the sun) is an established medical complication; however, other medical outcomes have received little attention. 1

Read More About

Redelmeier DA , Staples JA. Fatal Traffic Risks With a Total Solar Eclipse in the US. JAMA Intern Med. Published online March 25, 2024. doi:10.1001/jamainternmed.2023.5234

Manage citations:

© 2024

Artificial Intelligence Resource Center

Best of JAMA Network 2022

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

This paper is in the following e-collection/theme issue:

Published on 29.3.2024 in Vol 26 (2024)

An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study

Authors of this article:

Original Paper

• Lei Wang 1, 2 * , MS   ; 
• Yinyao Ma 3 * , MS, MD   ; 
• Wenshuai Bi 1 , MS   ; 
• Hanlin Lv 1 , MD, PhD   ; 
• Yuxiang Li 1, 2 , PhD  

1 BGI Research, Wuhan, China

2 Guangdong Bigdata Engineering Technology Research Center for Life Sciences, BGI Research, Shenzhen, China

3 Department of Obstetrics, People’s Hospital of Guangxi Zhuang Autonomous Region, Nanning, China

*these authors contributed equally

Corresponding Author:

Hanlin Lv, MD, PhD

BGI Research

1-2F, Building 2, Wuhan Optics Valley International Biomedical Enterprise Accelerator Phase 3.1

No 388 Gaoxin Road 2, Donghu New Technology Development Zone

Wuhan, 430074

Phone: 86 18707190886

Email: [email protected]

Background: The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.

Objective: This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.

Methods: The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People’s Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert’s annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.

Results: The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.

Conclusions: The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.

Introduction

Clinical text data have been widely recognized in data research due to their inclusion of multisource information [ 1 , 2 ] (eg, patient subjective statements, past objective facts, doctors’ diagnostic processes, and summary records). Extracting useful information from text data could serve as a crucial supplement to the study of disease progression; it could complement objective indicators dependent on laboratory tests and examinations [ 3 ], which has consistently been a hot research topic [ 4 , 5 ].

Historically, methods for text data extraction mainly include the following:

• Manual annotation: scales are designed based on clinical and research experience, followed by manual field extraction [ 6 - 8 ].
• Rule extraction: concepts from established knowledge base, such as International Classification of Diseases, Tenth Revision [ 9 ], are used for concept term extraction. This process is typically based on similarity algorithms and manual assistance to extract terms and their attributes (eg, negations and dependency relationships) [ 10 ].
• Named entity recognition or natural language processing algorithms: supervised learning methods, such as pretrained models like T5 [ 11 ], Bidirectional Encoder Representations from Transformers (BERT) [ 12 ], and BERT’s variants [ 13 - 15 ], with manual annotation to enhance semantic comprehension capabilities [ 16 , 17 ].

The task of extracting features from vast unstructured text presents itself as a daunting, labor-intensive, and time-consuming endeavor [ 18 ], for the following reasons:

• It is challenging to determine the dimension of extracted features initially, and from another perspective, confining the feature dimension means constraining the research scope from the outset [ 19 ].
• Given the inherent subjectivity and potential biases of recording subjects, solely relying on algorithms without annotation typically results in low accuracy and recall [ 20 ].
• Achieving higher accuracy with a broader feature scope, and the required human effort involved, is typically nonlinear [ 4 ], and the difficulty becomes apparent when confronted with massive real-world data.

The advent of large language models (LLMs) has paved a new path for the dilemma in clinical text extraction [ 21 - 23 ]. In the realm of natural language understanding research, generative large models, represented notably by ChatGPT [ 24 ] since 2022, have achieved unimaginable capabilities in semantic dimensions, leveraging the emergent intelligence from vast parameter scales. However, there are numerous considerations and limitations in their application, as follows:

• High-performing LLMs, such as OpenAI ChatGPT and Google Bard [ 25 , 26 ], are currently not open source, and patient data need to be submitted to their platform for analysis, presenting security challenges [ 27 , 28 ].
• Open-source LLMs with high intelligence generally require a large number of parameters (10-100 billion), which are hard to support on consumer-level graphics processing units (GPUs) [ 29 ].
• Low-parameter (around 10 billion) LLMs, typically require multistrategy support when dealing with tasks in certain vertical segments [ 30 ] (eg, fine-tuning, knowledge base or knowledge graph support, complex Chain of Thought (CoT) [ 31 ] along with its derivatives, and even global training) and are accompanied by various anomaly issues, including feature hallucination.

Although the application of LLM faces various potential limitations and challenges, as mentioned above, the foundational entity extraction and understanding capabilities of LLMs can still be used for low-cost extraction of clinical text data through meticulous prompt design, guidance combining CoT, and standardized examples [ 30 , 32 ].

In this study, we aimed to extract valuable features from a series of given patient admission records, which include the chief complaint and the medical histories. In light of this task, we introduced a modular LLM approach, which divides the entire extraction path into several smaller steps, with each modular LLM handling these basic steps automatically. We adopted the core idea of LLM agents [ 30 ] and self-consistency with CoT [ 33 ].

To experiment with this approach, we implemented 2 low-parameter LLMs in a local environment and compared their performances within a retrospective cohort of pregnancy to provide a reference that future researchers might draw upon.

Study Preparation

Data sources.

In this study, the text corpus was compiled from two primary sources:

• Chief complaints and medical histories, exemplified in Multimedia Appendix 1 , were extracted from inpatient admission records of an established cohort at the People’s Hospital of Guangxi Zhuang Autonomous Region in China. The established cohort for the preeclampsia risk study consisted of 25,709 pregnancies that received prenatal care between the 11th and 13th weeks of gestation from April 2012 to September 2021.
• Clinical practice guidelines consisted of the 2018 guidelines from the American College of Obstetricians and Gynecologists [ 34 ] and the 2019 guidelines from the National Institute for Health and Care Excellence [ 35 ].

To ensure linguistic consistency, the entire corpus was maintained in Chinese.

Model Deployment

We deployed 2 most exemplary LLMs in the Chinese domain until September 2023 in an intranet security environment independently: Qwen-14B-Chat (QWEN) [ 36 ] and Baichuan2-13B-Chat (BAICHUAN) [ 37 ]. In the environment, the server cluster used NVIDIA DGX-A100 (2×40 G) GPU nodes. The QWEN used 29 GB of storage and 27 GB of GPU memory, while the BAICHUAN used 26 GB of storage and 28.9 GB of GPU memory. Both models operated solely on physically isolated GPUs, and access was facilitated through the OpenAI [ 38 ] format and FastChat [ 39 ]. The LLMs were built upon PyTorch 2.0, with the temperature set to 0 and max_token adjusted task by task.

Experimental Path

In this study, we have introduced an approach that autonomously extracts valuable textual features. Diverging from traditional LLM applications, we used an “external-COT” strategy, dividing the process into several controllable steps, as illustrated in Figure 1 .

The extraction approach could be divided into four parts: (1) concept preparation, that is, extracting existing concepts from the corpus and selecting concerning concepts; (2) corpus preparation, that is, deidentifying raw data and preparing the corpus in accordance with the selected concept; (3) prompt design for different LLM tasks; (4) question-and-answer (Q&A) scales, that is, transforming concepts into question templates and extracting corresponding scales by LLMs.

Prompt Design

The design of prompt templates is fundamental to efficient and accurate extraction. Prior to processing the entire data set, an initial evaluation was conducted on 100 observations to assess the effectiveness of the templates, allowing for continuous refinement of prompt strategies and orientations. An appropriate template was defined based on the following criteria: (1) absence of redundant content generation, (2) consistent and uniform efficiency, and (3) infrequent occurrence of feature hallucination.

We adopted a 4-paragraph structure, referring to the prompt engineering suggestions of QWEN and BAICHUAN, as follows:

• Context section: defines the role and task, provides a basic understanding, and establishes a behavioral baseline for the model.
• Instruction section: outlines the execution steps, uses the CoT methodology, and provides examples to ensure guided model operation.
• Input data section: manages various inputs to meet diverse information needs.
• Output indicator section: specifies the output format and standards, setting clear expectations for the output.

To avoid input bias, the prompt templates for QWEN and BAICHUAN were maintained without any modifications. In addition, we conducted experiments using 100 observations at different levels of concurrency to select the most optimal configuration.

Concept Extraction and Aggregation

We initially extracted all discernible concepts from chief complaints and medical histories using LLMs with a designed prompt 1, and concepts were retained only with a manifestation frequency exceeding 5% occurrences. To reduce potential attention bias and expand the range of identified concepts, we also included concepts from clinical practice guidelines related to preeclampsia, particularly the American College of Obstetricians and Gynecologists 2018 guidelines and the National Institute for Health and Care Excellence 2019 guidelines.

As we defined in prompt 1, the extracted concepts were formatted using the Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) vocabulary within the Clinical Findings and Observations domain [ 40 ].

To mitigate potential output errors from LLMs, such as concepts not belonging to the Clinical Findings and Observations domain, or even errors outside of the SNOMED CT vocabulary, we implemented a rule-based matching approach to filter extraction inaccuracies.

Furthermore, in this research, we aimed to extract concepts with diverse semantic expressions (including diagnoses, various medical histories, symptoms, observations, interventions, and types of examination). To accomplish this, local experts manually filtered out concepts embedded in structured text, such as dates or numbers.

Question Generation

After the extraction and aggregation of concepts, they were transformed into specific questions by LLMs as question templates for subsequent data extraction. In this section, we leveraged ChatGPT4.0 as a question generator to produce a basic set of questions, which were then refined by local experts for specificity based on its performance across 100 observations.

Q&A Scale Extraction

To avoid contextual and temporal event confusion leading to incorrect responses (eg, confusing current medical history with a past medical history or confusing the patient’s medical history with that of family members), we preextracted the corpus using two strategies: (1) based on the position of the question templates and (2) based on the sentence containing the concepts. The extracted corpus was then labeled with the corresponding question templates for the subsequent extraction of Q&A scales.

The refined corpus, combined with corresponding question templates, guided a systematic extraction process with 2 LLMs, forming Q&A scales for further application.

Each question probed the LLMs, and the extracted sentences formed the basis of the generated responses. This approach enabled a logical mapping of questions to relevant text, ultimately improving the accuracy and efficiency of feature extraction.

Given the practical constraints and the objective of minimizing manual intervention, it was unfeasible to validate all answer scales individually across a Q&A space containing 68 questions and 25,709 observations. Therefore, a 3-fold assessment strategy was developed, as explained in the sections that follow.

Accuracy and Precision

A subset of 1500 observations chosen at random was manually annotated in collaboration with local experts, serving as the gold standard. The precision of positive identifications by both LLMs was assessed against a specified benchmark.

The null ratio of both LLMs was independently measured across all 25,709 observations. Empty or meaningless outputs (symbols and gibberish) were identified as null outputs, and the null ratio was then calculated as the proportion of such responses to the total.

Time Consumption

The efficiency of the extraction process was evaluated by measuring the time taken by the 2 LLMs to respond to the questions across all 25,709 observations.

Ethical Considerations

The study was approved by the People’s Hospital of the Guangxi Zhuang Autonomous Region in China (KT-KJT-2021-67). The requirement for informed consent was waived, due to the retrospective nature of the study, and all clinical data were deidentified and anonymized.

Path Decomposition Overview

Prompt template.

Through trials with the prompt template on 100 observations, we selected the template that demonstrated optimal consistency, as shown in Figure 2 .

Merged Concepts

After merging all concepts, we filtered out those that appeared less than 5% of the time. A total of 117 concepts and terms were listed in Table S1 in Multimedia Appendix 2 .

Question Template

Then we selected and transferred 68 concepts into question formats, for further Q&A scales. The detailed questions and their corresponding concepts are listed in Table S2 in Multimedia Appendix 2 .

Scale Extraction

We identified that the optimal performance in Q&A scale extraction occurs with a concurrency of 3 requests, enhancing speed by 17.9% compared to a single request. Furthermore, we used a max_token restriction strategy, capping it at 20, to optimize inference speed.

Ultimately, within the 2D Q&A space formed by the answer scales, there were a total of 68 question columns and 25,709 observations (listed in Table 1 ).

We used accuracy and precision metrics for assessing the accuracy of LLMs across 1500 observations. Furthermore, we used 2 parameters—null ratio and time consumption—in 25,709 observations to evaluate the consistency and efficiency of the 2 LLMs, respectively.

a Not applicable.

b The mean pregnancy weight gain was 13.73 (SD 24.12) for QWEN and 13.75 (SD 31.28) for BAICHUAN.

c G6PD: glucose-6-phosphate dehydrogenase.

Evaluation Metrics

Figure 3 A and 3B and Figure S1 (parts A and B) in Multimedia Appendix 2 illustrate the Q&A space for a sample chunk extracted by QWEN and BAICHUAN with the comparison of manual annotation. The figures demonstrate the exceptional accuracy and precision of QWEN and BAICHUAN. QWEN attained an average accuracy of 95.52% and an average precision of 92.93%, whereas BAICHUAN displayed an average accuracy of 95.86% and an average precision of 90.08%. These figures clearly indicate that the 2 LLMs have more concentrated errors in specific concepts, and overall, they achieve high levels of precision in most extractions.

LLMs demonstrated consistent performance across most questions and excelled in binary, well-defined medical history questions, often reaching 100% accuracy and precision. However, the accuracy performance varied significantly when dealing with questions that involved semantic ambiguities or definitional uncertainties. This inconsistency might be tied to the LLM’s training and inference alignment. Notable disparities were observed in questions pertaining to menstrual color (QWEN: 1000/1500, 66.7%; BAICHUAN: 1097/1500, 73.1%), early pregnancy symptoms (QWEN: 909/1500, 60.7%; BAICHUAN: 1474/1500, 98.3%), vaginal bleeding (QWEN: 593/1500, 39.5%; BAICHUAN: 1455/1500, 97%), bilateral lower limb edema (QWEN: 786/1500, 52.4%; BAICHUAN: 1486/1500, 99.7%), and menstrual flow (QWEN: 1498/1500, 99.8%; BAICHUAN: 605/1500, 40.3%).

Apart from the above, the precision inconsistency performance of concepts could be attributed to their low true positive rate, like insomnia (QWEN: 3/17, 17.7%; BAICHUAN: 8/17, 47.1%), personal history—antiphospholipid syndrome (QWEN: 0/2, 0%; BAICHUAN: 1/2, 50%), and poor pregnancy history—induced abortion (QWEN: 1/2, 50%; BAICHUAN: 2/2, 100%). The exact precision is listed in Table S3 in Multimedia Appendix 2 .

As depicted in Table 1 , both LLMs demonstrated superior performance with minimal null ratios. Specifically, QWEN (Figure S1A in Multimedia Appendix 2 ) exhibited a mean null ratio of 0.02%, in contrast to BAICHUAN (Figure S1B in Multimedia Appendix 2 ), which recorded a slightly higher null ratio of 0.21%. Failure of QWEN extraction was only in pregnancy weight gain (411/25,709, 1.60%), but failures of BAICHUAN extraction were mainly in symptoms (abdominal pain: 1252/25,707, 4.87%; vaginal infection: 275/25,709, 1.07%).

We conducted a comparative analysis of the time performance between QWEN and BAICHUAN on various Q&A scales, discovering that BAICHUAN consistently exhibits higher time consumption across almost all scales, reaching up to 4 times that of QWEN, as illustrated in Figure 4 B.

Figure 4 A compares the time consumption of LLMs in extracting diverse concepts. Although there were significant differences across different concepts, overall, the LLMs demonstrated a consistent performance across these concepts. For queries with clear definitions and concise corpora, such as drug usage and previous pregnancy history, the time consumed was minimal. In the category of medical history, both models exhibited uniform and stable performances (QWEN and BAICHUAN both revealed a time consumption ratio of 1:3).

Additional Research

In clinical practice, to address scenarios of resource constraints, we used a quantified version of the LLM in our study to validate the applicability of this approach. We used an official-release INT4 version of QWEN, QWEN(INT4). The model was deployed on an NVIDIA RTX 3090 GPU (24 GB).

With the same approach listed above, the performance of QWEN(INT4) achieved even better performance. Figure 3 and Figure S1C in Multimedia Appendix 2 demonstrate that the average accuracy of QWEN(INT4) is 97.28%, accompanied by a null ratio of 0%.

Despite a notable correlation in performance extraction between QWEN(INT4) and QWEN, QWEN(INT4) demonstrated superior efficiency on limited hardware, with an average of 31 seconds per observation, compared to 47 seconds for QWEN and 312 seconds for BAICHUAN.

Principal Findings

In this study, the extracted scales incorporated not only the conventional features of interest but also less frequently mentioned dimensions in previous cohorts or guidelines. These included food and drug allergies (6.6% for food allergy and 25.2% for drug allergy), certain pregnancy symptoms (average positive ratio of 0.9% for insomnia and 2.3% for palpitations), menstrual conditions (22.8% for dysmenorrhea), medical history (1% for asthma family history and 0.27% for mental illness family history), and gestational intervention (13.93% for progestogen and 1.4% for aspirin).

As a naturally recruited cohort of pregnancy, the extracted features show comparable proportions or trends compared with similar studies, such as systemic lupus erythematosus (average positive ratio of 0.20% vs 0.03%-0.23% of similar cohorts [ 41 , 42 ]) and antiphospholipid syndrome (average positive ratio of 0.08% vs 0.02%-0.12% of similar cohorts [ 43 ]), thereby corroborating the accuracy of our approach.

Additionally, certain scale deviations were revealed compared to similar studies, notably in fetal paternal smoking history (average positive ratio of 0.04% vs approximately 28.1%-40% in similar studies [ 44 , 45 ]). Although these deviations were few, we conducted a sample retracing to the original texts and determined that the extraction approach was not at fault and accurately reflected the original data. This discrepancy highlights persistent concerns [ 46 ] regarding the data quality in inpatient documentation, originating from patient self-reports and physician documentation, and vulnerability to recall and inquiry bias. Documentation varies among patients, influenced not only by patient conditions but also by physicians’ writing habits. Thus, we regard our approach as a preexperimental data analysis. Despite the presence of biases or missing dimensions, the approach uncovers several dimensions absent in structured medical texts, and valuable insights could still be extracted from the data with appropriate statistics [ 47 ]. In clinical practice, preliminary interviews with documenting physicians are recommended prior to the selection of concepts to enhance data quality and mitigate potential biases.

In the context of the extraction process, even when deployed solely on a standard consumer-grade GPU (NVIDIA RTX 3090), the QWEN(INT4) completed the extraction of 25,709 observations and 68 features within 15 calendar days, averaging 48.9 seconds per observation. In practical applications, deploying 2 instances of QWEN(INT4) on a single graphics card, coupled with an additional deployment in CPU [ 36 ], is hypothesized to reduce the extraction to approximately 7 days. Furthermore, multi-GPU server clusters, prevalent in clinical environments, could markedly reduce processing times, potentially to the scale of hours.

In our study, we experimented with omitting the corpus extraction step, directly using the long text of each observation’s chief complaints and medical histories as raw data for Q&A scale extraction. However, the experiment yielded poor performance in accuracy, precision, and time consumption, as illustrated in Figure 5 and Figure S2 in Multimedia Appendix 2 .

These limitations appear significantly correlated with the current technological constraints of LLMs [ 32 ], which tend to generate “feature hallucinations” more frequently when processing extensive texts [ 48 ], leading to the loss of critical information. We believe that this issue will be resolved as the technology continues to evolve [ 49 ].

Limitations

In our experimental validation, we selected a limited set of concepts, comprising only 68 items, to balance the consideration of time constraints. Despite our efforts to encompass a broad scope, some dimensions inevitably remain unaddressed, which is a limitation in verifying efficiency and accuracy across all dimensions.

Furthermore, the raw data in this study was sourced exclusively from a single hospital, spanning nearly a decade. This duration, while significant, introduces limitations in the generalizability of our approach.

Additionally, the approach used only 2 LLMs. Although we anticipate that future LLMs will be compatible with the current approach, this assumption necessitates further experimental validation.

Conclusions

Our proposed approach offers a potential methodology for clinical text data analysis. It involves extracting and summarizing concepts from the comprehensive text of a defined population, thus selecting research directions of interest, and eventually generating analyzable features for the cohort. This approach demonstrates notable precision and could provide substantial data support for future research endeavors.

Acknowledgments

We are grateful to Yinyao Ma and the clinical team for their exceptional contributions to this project and we thank the technical support provided by China National GeneBank.

This work was supported by Guangxi Key Research and Development Program (AB22035056).

Data Availability

The data sets generated and analyzed during this study are not publicly available due to privacy or ethical restrictions but are available on request from the corresponding author.

Authors' Contributions

LW and YM contributed equally to this study. LW, HL, and YM participated in the study design and drafted the manuscript. YL and HL participated in data collection and outcome rule review. LW and WB performed the statistical analysis, and established machine learning models. YL helped to draft the manuscript. All authors read and approved the final manuscript.

Conflicts of Interest

None declared.

A sample of chief complaints and medical histories.

Additional statistics.

• Tayefi M, Ngo P, Chomutare T, Dalianis H, Salvi E, Budrionis A, et al. Challenges and opportunities beyond structured data in analysis of electronic health records. WIREs Computational Stats. Feb 14, 2021;13(6):1-19. [ CrossRef ]
• Sun W, Cai Z, Li Y, Liu F, Fang S, Wang G. Data processing and text mining technologies on electronic medical records: a review. J Healthc Eng. 2018;2018:4302425. [ CrossRef ] [ Medline ]
• Varshini K, Uthra R. An approach to extract meaningful dData from unstructured clinical notes. In: Inventive Systems and Control. Singapore. Springer; Jun 08, 2021;581-590.
• Spasic I, Nenadic G. Clinical text data in machine learning: systematic review. JMIR Med Inform. Mar 31, 2020;8(3):e17984. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wu P, Yu A, Tsai C, Koh J, Kuo C, Chen ALP. Keyword extraction and structuralization of medical reports. Health Inf Sci Syst. Dec 03, 2020;8(1):18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vassar M, Holzmann M. The retrospective chart review: important methodological considerations. J Educ Eval Health Prof. Nov 30, 2013;10:12. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cassidy LD, Marsh GM, Holleran MK, Ruhl LS. Methodology to improve data quality from chart review in the managed care setting. Am J Manag Care. Sep 2002;8(9):787-793. [ FREE Full text ] [ Medline ]
• Engel L, Henderson C, Fergenbaum J, Colantonio A. Medical record review conduction model for improving interrater reliability of abstracting medical-related information. Eval Health Prof. Sep 13, 2009;32(3):281-298. [ CrossRef ] [ Medline ]
• Agbavor F, Liang H. Predicting dementia from spontaneous speech using large language models. PLOS Digit Health. Dec 22, 2022;1(12):e0000168. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mykowiecka A, Marciniak M, Kupść A. Rule-based information extraction from patients' clinical data. J Biomed Inform. Oct 2009;42(5):923-936. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR. 2020;21(140):1-67.
• Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv. [ FREE Full text ]
• Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. Feb 15, 2020;36(4):1234-1240. [ CrossRef ] [ Medline ]
• Li F, Jin Y, Liu W, Rawat BPS, Cai P, Yu H. Fine-tuning bidirectional encoder representations from transformers (BERT)-based models on large-scale electronic health record notes: an empirical study. JMIR Med Inform. Sep 12, 2019;7(3):e14830. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang S, Yilahun H, Hamdulla A. Biomedical named entity recognition based on MCBERT. 2022. Presented at: 2022 International Conference on Asian Language Processing (IALP); October 27-28, 2022; Singapore. [ CrossRef ]
• Liu H, Zhang Z, Xu Y, Wang N, Huang Y, Yang Z, et al. Use of BERT (bidirectional encoder representations from transformers)-based deep learning method for extracting evidences in chinese radiology reports: development of a computer-aided liver cancer diagnosis framework. J Med Internet Res. Jan 12, 2021;23(1):e19689. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Z, Zhu L, Yu P. Multi-level representation learning for Chinese medical entity recognition: model development and validation. JMIR Med Inform. May 04, 2020;8(5):e17637. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, et al. Clinical information extraction applications: a literature review. J Biomed Inform. Jan 2018;77:34-49. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Polnaszek B, Gilmore-Bykovskyi A, Hovanes M, Roiland R, Ferguson P, Brown R, et al. Overcoming the challenges of unstructured data in multisite, electronic medical record-based abstraction. Med Care. Oct 2016;54(10):e65-e72. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tarik AM, Sorin E, Symons J, Mayer E, Yaliraki S, Toni F. Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records. arXiv. [ FREE Full text ] [ CrossRef ]
• Choi HS, Song JY, Shin KH, Chang JH, Jang B. Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer. Radiat Oncol J. Sep 2023;41(3):209-216. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Decker H, Trang K, Ramirez J, Colley A, Pierce L, Coleman M, et al. Large language model-based chatbot vs surgeon-generated informed consent documentation for common procedures. JAMA Netw Open. Oct 02, 2023;6(10):e2336997. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Johnson S, King A, Warner E, Aneja S, Kann B, Bylund C. Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information. JNCI Cancer Spectr. Mar 01, 2023;7(2):a. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst. Mar 04, 2023;47(1):33. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moons P, Van Bulck L. Using ChatGPT and Google Bard to improve the readability of written patient information: a proof of concept. Eur J Cardiovasc Nurs. Mar 12, 2024;23(2):122-126. [ CrossRef ] [ Medline ]
• Giannakopoulos K, Kavadella A, Aaqel Salim A, Stamatopoulos V, Kaklamanos EG. Evaluation of the performance of generative AI large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: comparative mixed methods study. J Med Internet Res. Dec 28, 2023;25:e51580. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thapa S, Adhikari S. ChatGPT, Bard, and large language models for biomedical research: opportunities and pitfalls. Ann Biomed Eng. Dec 16, 2023;51(12):2647-2651. [ CrossRef ] [ Medline ]
• Sezgin E, Chekeni F, Lee J, Keim S. Clinical accuracy of large language models and Google search responses to postpartum depression questions: cross-sectional study. J Med Internet Res. Sep 11, 2023;25:e49240. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang L, Liu X, Li Z, Pan X, Dong P, Fan R. Dissecting the runtime performance of the training, fine-tuning, and inference of large language models. arXiv. [ FREE Full text ]
• Wang L, Ma C, Feng X, Zhang Z, Yang H, Zhang J. A survey on large language model based autonomous agents. arXiv. ;01 [ FREE Full text ] [ CrossRef ]
• Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E. Chain-of-thought prompting elicits reasoning in large language models. arXiv. Preprint posted online on Jan 28, 2022. [ CrossRef ]
• Lei D, Li Y, Hu M, Wang M, Yun V, Ching E. Chain of natural language inference for reducing large language model ungrounded hallucinations. arXiv. [ FREE Full text ] [ CrossRef ]
• Wang X, Wei J, Schuurmans D, Le Q, Chi E, Narang S. Self-consistency improves chain of thought reasoning in language models. arXiv. [ FREE Full text ]
• ACOG committee opinion no. 743: low-dose aspirin use during pregnancy. Obstet Gynecol. Jul 2018;132(1):e44-e52. [ CrossRef ] [ Medline ]
• National IFHEG. National Institute for Health and Care Excellence (NICE). In: The Grants Register 2019. London, UK. Palgrave Macmillan; 2019;540.
• Bai J, Bai S, Chu Y, Cui Z, Dang K, Deng X, et al. Qwen Technical Report. arXiv. ;01 [ FREE Full text ] [ CrossRef ]
• Yang A, Xiao B, Wang B, Zhang B, Bian C, Yin C. Baichuan 2: open large-scale language models. arXiv. [ FREE Full text ]
• OpenAI. URL: https://openai.com/blog/openai-api [accessed 2024-03-12]
• Zheng L, Chiang W, Sheng Y, Zhuang S, Wu Z, Zhuang Y. Judging LLM-as-a-judge with MT-bench and chatbot arena. arXiv. ;01 [ FREE Full text ] [ CrossRef ]
• SNOMED International. URL: http://www.ihtsdo.org/snomed-ct/ [accessed 2024-03-13]
• Clowse ME, Jamison M, Myers E, James AH. A national study of the complications of lupus in pregnancy. Am J Obstet Gynecol. Aug 2008;199(2):127.e1-127.e6. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rees F, Doherty M, Grainge M, Lanyon P, Zhang W. The worldwide incidence and prevalence of systemic lupus erythematosus: a systematic review of epidemiological studies. Rheumatology (Oxford). Nov 01, 2017;56(11):1945-1961. [ CrossRef ] [ Medline ]
• Hwang J, Shin S, Kim Y, Oh Y, Lee S, Kim YH, et al. Epidemiology of antiphospholipid syndrome in Korea: a nationwide population-based study. J Korean Med Sci. Feb 10, 2020;35(5):e35. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xu X, Rao Y, Wang L, Liu S, Guo JJ, Sharma M, et al. Smoking in pregnancy: a cross-sectional study in China. Tob Induc Dis. Jul 24, 2017;15(1):35. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yang Y, Liu F, Wang L, Li Q, Wang X, Chen JC, et al. Association of husband smoking with wife's hypertension status in over 5 million Chinese females aged 20 to 49 years. J Am Heart Assoc. Mar 20, 2017;6(3):e004924. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leon N, Balakrishna Y, Hohlfeld A, Odendaal W, Schmidt B, Zweigenthal V, et al. Routine Health Information System (RHIS) improvements for strengthened health system management. Cochrane Database Syst Rev. Aug 13, 2020;8(8):CD012012. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miñarro-Giménez JA, Cornet R, Jaulent M, Dewenter H, Thun S, Gøeg KR, et al. Quantitative analysis of manual annotation of clinical text samples. Int J Med Inform. Mar 2019;123:37-48. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rawte V, Chakraborty S, Pathak A, Sarkar A, Towhidul ITS, Chadha A. The troubling emergence of hallucination in large language models -- an extensive definition, quantification, and prescriptive remediations. arXiv. ;01 [ FREE Full text ] [ CrossRef ]
• Jones E, Palangi H, Simões C, Chandrasekaran V, Mukherjee S, Mitra A. Teaching language models to hallucinate less with synthetic tasks. arXiv. ;01 [ FREE Full text ] [ CrossRef ]

Abbreviations

Edited by Q Jin; submitted 15.11.23; peer-reviewed by X Tannier, M Torii; comments to author 12.01.24; revised version received 23.01.24; accepted 14.02.24; published 29.03.24.

©Lei Wang, Yinyao Ma, Wenshuai Bi, Hanlin Lv, Yuxiang Li. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.