Graduate Research: Guide to the Literature Review

There are several criteria that can aid you when you are trying to find the best resources for your literature review.

• Type of Publication: For college-level term papers, information should be obtained mostly from scholarly journals or conference proceedings. Scholarly journals contain articles describing high quality research that has been reviewed by experts in the field prior to publication.Trade magazines may be useful for topics in business or where economic data is needed. There are also good for learning what the current "hot topics" are in an area. Popular magazines, such as Time and Newsweek, should be used sparingly, or not at all
• Organization and Content: Is the material organized and focused? Is the argument or presentation understandable? Is this original research, a review of previous research, or an informative piece?
• Audience: For what type of reader is the author writing? This ties in with the type of journal, as popular magazine are geared to the general reader, while trade magazines are for the specialist and scholarly journals are directed at researchers, scholars or experts in the field. Is the article for:
  • general readers,
  • students (high school, college, graduate),
  • specialists or professionals,
  • researchers or scholars?
• Publisher: Who published the article?
  • Most sources for college papers should come from scholarly journals. Scholarly journals are journals that are peer-reviewed before publication. This means that experts in the field read and approve the material in the article before it is published.
• Authors: Who wrote the article? 
  • Good sources of information come from experts in the field. This is called authority. Many times these individuals will be employed at research institutions such as universities, labs, or founded associations. 
  • What else have the author/authors written?
    • Many times, authors with authority have written more than one article about a topic in their field. Not only does this add support to their reputation, but can also be a great source for more articles. 
  • Has the author been cited?
    • Citation is a great indicator to the effectiveness of the article. If other experts are citing the article, it is a good sign this source is trusted and relevant.
• Date: When was the article published?
  • The date of publication is especially relevant to those in the STEM disciplines. Research in science, technology, engineering, and mathematics moves very quickly, so have an article that was published recently will be more useful. 
• Publisher's Affiliation: Is the publisher affiliated with a reputable institution? 
  • Many universities operate their own press. Books and articles that are published by a reputable institution are generally accepted as valuable information. Many professional organizations also hold conferences as well as publish scholarly journals.
• Bibliography: Scholarly works always contain a bibliography of the resources that were consulted. The references in this list should be in sufficient quantity and be appropriate for the content. Look for:
  • if the bibliography cites a variety of other authors,
  • if a bibliography exists,
  • if the bibliography is short or long,
  • if the bibliography is selective or comprehensive,
  • if the references are primary sources (ex. journal articles) or only secondary sources (ex. encyclopedias),
  • if the references are contemporary to the article or much older, and
  • if the citation style is clear and consistent.

Scholarly peer review is a process to assure the quality of articles in a particular discipline or field of study. Work, activities, decision-making, and problem-solving need to be based on high quality evidence.  

In scholarly peer-review, articles are evaluated by other scholars/specialists who are experts in the specialty/topic of the article.

• The three or four reviewers of each article may recommend that the article be published as is, published after certain revisions, or not published.
• The "highest" level of peer review is a "double-blind" review in which the authors don't know who has evaluated their papers and the reviewers don't know whose papers they are reviewing. That ensures a fairer process of review.

Journals that use a peer-review process to select which articles they publish are called "peer-reviewed journals" or "refereed journals."

• In some scholarly journals, every article (except the editorial or introductory essay by an editor) has gone through peer review.
• Some scholarly journals may have news items, editorials, and other features that are not peer-reviewed and should not be confused with the peer-reviewed content.

Using peer-reviewed scholarly journals helps you base your work and decisions on credible evidence.

How can I find out if an article is peer-reviewed?

FIRST. Determine if the article is scholarly. Only scholarly articles are likely to be peer-reviewed, so you can automatically eliminate non-scholarly articles from consideration.

SECOND. Although most articles that publish original research and that are found using professional research databases are likely to be peer-reviewed, here are two ways to determine if a journal is peer-reviewed:

1. Go to the journal's website and look for the process the journal uses for selecting articles. On the journal's website:
   • look for the "About" link or
   • look for "submission guidelines," "author guidelines," "information for authors" or some other similar option. The author area will typically describe how articles are selected for publication. 
   • For example, the Journal of the American Medical Association (JAMA) states, "Peer reviewer identities are kept confidential, but author identities are made known to reviewers." Author identities remain anonymous to reviewers for the International Journal of Research and Method in Education: "Authors should prepare and upload two versions of their manuscript. One should be a complete text, while in the second all document information identifying the author should be removed from files to allow them to be sent anonymously to referees." 
      
2. Search Cabell's Directories of Publishing Opportunities.

Magazines and journals take distinct forms based on their purpose and audience. In the case of scholarly journals and technical/trade journals, they have the same audience -- a particular profession or group of people working in the same discipline. Scholarly journals and technical/trade journals are also alike in that the articles within both are typically written by professionals in the discipline. However, these two types of journals differ in terms of purpose. 

• "Technical journals" and "trade journals" help someone do their job better by providing current news of importance to the profession as well as job postings and ads relevant to the profession.  Examples of technical journals include Network Computing and IEEE Spectrum.

• The purpose of scholarly journals (also called "research journals") is to report original research -- that is, newly reported research carried out by the person or persons writing the article. Credible scholarly journals use a peer-review process to ensure the quality of their articles. Examples of scholarly journals include Journal of Management Information Systems, Journal of Applied Security Research, and IEEE Transactions on Computers.

Most reading is now done online, and often the articles are found separately  --not in the context of the journal or magazine in which the article was published. 

• This can add to the difficulty of distinguishing technical/trade articles from popular articles, because the intended audience may be unclear.
• Scholarly research articles are easier to identify because they usually follow recognizable formats and explicitly state that they are reporting the authors' research. 
• However, scholarly research articles may be published by disreputable publishers -- publishers that use poor or no peer-review.

If you would like a more detailed explanation, read on...

• Articles written for a general audience, even if the topic is serious, do not go through scholarly peer review. You can assume that articles published in magazines for the general public are not peer-reviewed. Such magazines range from popular ones that entertain such as People Weekly to substantial news magazines such as Time or Atlantic Monthly or MacWorld.

• Articles written for a specific professional audience (such as teachers, business managers, animators, computer scientists) may be scholarly, or they may be technical/trade articles. Technical/trade articles usually do not, but sometimes may, go through a peer review process.

• The key characteristic of a scholarly article is that it reports original research or original theory. The word "original" means that the author or authors have done the research study or have developed the theory or model that they are reporting. This content is critical for distinguishing scholarly articles from technical/trade articles. While technical/trade articles are very useful for keeping their audience up-to-date about tools, techniques, and news that help them do their jobs better, these articles do not report original research.

• Some secondary characteristics MUST be found in any scholarly article but may, also, be found in some technical/trade articles. A scholarly article will always have a bibliography, because the author(s) must demonstrate  knowledge of pre-existing relevant research by others and must show how their research contributes to existing knowledge.  However, a technical/trade article may also include a bibliography, so this characteristic (having a bibliography) cannot be used on its own to distinguish technical/trade from scholarly articles.

• Original research studies tend to have a similar format (although this can vary by discipline) that includes introduction, literature review, methods, results, and discussion. When you see this format, you can assume the article is scholarly. Also, when an author refers to "this study" or "this research," that's a clue that the author is reporting original research. Of course, a bogus article could be written in this format to appear to be scholarly when it's not, so it does matter that the source of the article is a reputable journal or journal website or that you found it using a reputable research database.

• The easiest way to know that an article is scholarly (though not necessarily peer-reviewed) is to find it using a source that only includes scholarly articles. Research databases for specific disciplines will often include only (or primarily) scholarly articles. For example, PubMed (for health and medicine) and Physical Education Index fall into this category. However, other discipline-specific research databases may contain a mix of scholarly and technical/trade articles, so you'll need to evaluate whether any given article is scholarly or not.

• Research databases that include articles that range from popular and substantial news articles to scholarly articles -- such as Academic Search Premier in EBSCOhost or Proquest Research Library -- will often include a search option to limit your search to only scholarly articles. Use this feature to eliminate popular articles and save time, but be aware that the search results may include the non-scholarly articles (editorials, opinion pieces) that are sometimes included in scholarly journals; or it may in some other way mis-identify an article as scholarly. You'll need to evaluate whether the article you select actually is scholarly.

This table describes the differences between scholarly journals, technical/trade journals, substantial news/general interest magazines, and popular magazines based on the set of criteria in the left column.

These criteria may be used to determine where in the continuum from scholarly to popular a particular periodical (magazine or journal) falls.

ENGL201 Scholarly table.

Karl E. Mundt Library/ Dakota State University.

Based on document developed by Purdue University. Undergraduate Library