There are two key aspects involved in the in-depth examination of a source:
What the source is saying--its arguments, methods, evidence, and conclusions
AND
the explicit and implicit responses to that source by other researchers, scholars, and critics.
Let's take a closer look at the first component of that process.
In a scholarly paper, a significant portion of the sources you'll be using will likely be peer-reviewed, scholarly articles, as this is considered the main arena in which the scholarly conversation takes place. These sources can make for difficult reading, even for experts.
Luckily, despite the often-dense nature of academic material, it frequently also follows structural patterns that we can use as guideposts in our reading. This video breaks down the major parts of academic articles and suggests a useful process for reading them, a process sometimes known as "pre-reading" or "purposeful skimming":
Scholarly articles won't always use standardized subsection headers, like Abstract, Introduction, Discussion, and so on, but scholars will still engage in familiar sets of analytical actions regardless of whether they're explicitly labeled as such. Learning to identify these actions takes practice; we'll get into the details in the next section.
Theoretically, peer review yields the highest quality of research. That's often indeed the case, but there is plenty of peer-reviewed research that is methodologically poor or reaches specious conclusions, so it's important to dive in and closely read the sources you've deemed relevant to your literature review. This section will give you some guidance on how to locate and comprehend the arguments, methods, evidence, and conclusions of a scholarly source.
Note: Depending on the field you're working within, there may be important sources that appear outside of peer-reviewed, scholarly journals. This is especially the case in the humanities and some social sciences. These sources may require a different approach for reading and comprehension.
* * *
The first step to understanding a source is to identify its argument. Start by taking a look at the introductory sections in an article, chapter, or book. As shown in this short guide to Formal Moves in Scholarly Work, the introductory area is where scholars will lay out where their paper fits into the bigger picture of the scholarly conversation. As part of that, they will assert their own argument, discuss their findings, and explain the crux of their analysis. Here's one example from an article about selfies:
Not every text will state its intentions and argument as clearly as this one does; still, in it we see clues on what to look out for when identifying an argument:
Descriptions of research methods, processes, and analytical approaches vary widely depending on the subject area an author is writing in. Similarly, while you'll get a hint of the method in the introduction, where in a text that fuller descriptions appear can vary. Sometimes they make it easy for you and title a section "Method" or "Methodology," but other times you'll have to do a bit of digging. Here's an example of the latter:
We don't see "Method" in the section title, but we do see "Approach," a close synonym. And as with the example in "Identifying the Argument," we also see self-referential language about what the authors did in their study. Given that certain fields of scholarship are especially allergic to the use of "I" pronouns, this language will sometimes be in passive voice, i.e., "Faculty and librarians were asked to engage..."
Descriptions of methods and processes can sometimes be very involved, so it may take some rereading to fully understand. Also, in some fields (particularly the natural sciences), methods are standardized, and knowledge of their details may be assumed. You may need to look at outside descriptions of those methods in order to get a complete picture of the procedure that an author followed.
Summarizing (and evaluating) evidence and analysis is the most substantive part of the source analysis process. It can be immensely helpful to start by zeroing in on how a source is structuring its analysis. The slideshow below illustrates a few patterns you'll commonly run into:
Once you've determined the structure, it's time to wade into the details. When you do this, you should have the goal of answering three key questions:
What are the facets of the argument?
What evidence is being used in support of the argument?
What are the limitations of the author's analysis?
Complete answers to these questions will give you a solid sense of the content of the source and have you well on your way to evaluating it for use in your literature review.
Of course, this is often easier said than done. If you're running into problems understanding the content of a source, it can help to keep a few things in mind:
This guide mainly focuses on quantitative research and ways in which to analyze and read it. Qualitative and quantitative are both broad categories that include various types of research methods. This section of the guide will focus on qualitative research and techniques to evaluate it.
Brief Definitions
Quantitative research collects numerical data and produces empirical data that can be measured and expressed numerically
Qualitative research focuses on non-numerical data and explores subjective experiences through observation and interviews
Ethnography is a qualitative research method in which the researcher, or ethnographer, aims to learn more about a community in a holistic manner. The ethnographer engages in participant observation, meaning they actively participate in the community they are studying and observes the group in natural settings. Ethnography is both a research method and a product with the researcher performing the research and writing the ethnography.
Analyzing Ethnographies and Qualitative Research
Analyzing qualitative research and ethnographies is different from how you analyze quantitative research. This is due to the use of different research methods and different types of data collected with qualitative consisting of descriptive and non-numerical data. This means qualitative research focuses on finding themes and answering "Why?" and "How?" questions in their data. This section will focus specifically on analyzing ethnographies which you can use to analyze other forms of qualitative research as well.
While reading all types of research, you should be on the look out for potential bias in the study. This is particularly important for reading and analyzing qualitative research where the researcher’s own personal beliefs, attitudes, and expectations can affect the information the observer gathers and how they interpret data.
Here are examples of positionality statements:
Roberts, S. O., Bareket-Shavit, C., Dollins, F. A., Goldie, P. D., & Mortenson, E. (2020). Racial inequality in psychological research: trends of the past and recommendations for the future. Perspectives on Psychological Science : A Journal of the Association for Psychological Science, 15(6), 1295–1309. https://doi.org/10.1177/1745691620927709
Davis, S. M. (2018). The aftermath of #BlackGirlsRock vs. #WhiteGirlsRock: considering the disRespectability of a black women’s counterpublic. Women’s Studies in Communication, 41(3), 269–290. https://doi.org/10.1080/07491409.2018.1505678
Positionality Statements in Brief, University of Michigan Medical School
In "Evaluating Ethnographies", Laurel Richardson argues that an effective way to analyze ethnographies is through the use of scientific and artistic lenses.
Richardson lays out five criteria to use to analyze ethnographies:
Substantive contribution: Does this piece contribute to our understanding of community or social life? Does this piece have a grounded human-world understanding?
Aesthetic merit: Does this piece succeed aesthetically? Is it satisfying, complex, and not boring?
Reflexivity: Are there ethical issues with how information was gathered or distributed? How has the author’s subjectivity affected this piece? Is there adequate self-awareness for the reader to make judgements about their point of view? Does the author hold themselves accountable to sharing the story of the community they studied?
Impact: Does this affect me emotionally or intellectually?
Expresses a reality: Does this piece have a clear, embodied sense as a lived experience? Does it seem “true” as a credible account?
(Richardson 254)
It is also important to note the various ways a researcher can verify the content of their ethnography. This verification is important to establish that their analysis is not baseless, random observations and provide validity for their analysis of the community.
Three common types of verification in ethnography include:
Prolonged engagement with the field: The ethnographer explains why the amount of time they spent participating and observing the group was sufficient for the study
Thick description of the group: Thick description is the use of extended quotes from interviews as well as the use of concrete information from their field notes during their participation and observation of the community. This contributes to the credibility of the ethnography as it allows space for the reader to draw their own conclusions from the data and can determine if the ethnographer’s conclusions are justified
Member checking: This is when the ethnographer provides the ethnography to the community, and they read it and respond to it. This gives confidence to the ethnographer and the group that the conclusions in the study are appropriate.
Introductions serve different purposes depending on the genre of writing. Magazine articles or personal essays often start with a "hook" that sets the scene, creates tension, or engages the reader's thoughts and emotions. Academic essays tend to rely on a different kind of hook, one that situates that author's work in relation to the sources, ideas, and scholarship that have paved the way for their own work, or that they are responding to or critiquing.
Scholars John Swales and Christine Feak have mapped out a pattern such introductions tend to follow, one that you can use to orient yourself to what an article is studying and where it fits into the bigger picture. An adaptation of Swales and Feak's map, which is based on three "moves," is shown below. Click on each tab for more detail and examples.
Adapted from Swales and Feak, Academic Writing for Graduate Students
The first move, Setting the Scene, is about laying out the broad strokes of the topic, its importance within the discipline(s) scholars are working within (psychology, biology, business, etc.), and/or key pieces of research that form the background to their particular focus. As you may expect, this often entails drawing on the Background kinds of sources outlined in the BEAM method of the Using Sources in Papers tab.
Here's an example from the article "Exploring Curation as a Core Competency in Digital and Media Literacy Education" by Paul Mihailidis and James N. Cohen:
By pointing to broad statistics about digital media use, Mihailidis and Cohen provide the broad context in which they are working and point to the importance of focusing on the topic. The final sentence situates their focus as being in the educational field, as shown by the use of phrases like "...how students learn..." and "...pedagogical approaches..." (i.e., methods used in teaching). As readers, we now have a bit more grounding and know the general areas the authors will be discussing.
Having created a background to work with, scholars can then turn to the second move of Describing the Problem, where they zero in on the whats and whys of the topic that they're focusing on.
What this looks like can vary: they may discuss gaps in research or unanswered questions they want to address in their writing, or they may point to previous work their own work is building on, or they may suggest that there are flaws in the existing research they've already outlined when Setting the Scene. They're not yet discussing precisely how they're going to address the problem they're defining, but they're preparing the ground for themselves to do so.
Expressed in BEAM terms, the sources they cite may be more specific Background sources, or they may be Argument sources that will be discussed in greater detail in the body of the article.
Mihailidis and Cohen's example is again illustrative:
So, what kind of problem are they framing here? They positively cite Hobbs's work on the sets of skills students need to thrive in a digital world, then focus specifically on the skill of navigating the personalized way the internet allows people to aggregate content into uniquely curated collections of information. In this sense, they build on Hobbs's work but also identify a potential gap to be filled by focusing on curation as an important competency teachers need to be prepared to cover. We now know what Mihailidis and Cohen will be focusing on.
Having outlined the problem they're addressing, scholars can then turn to Entering the Conversation. This final move entails explaining the particulars of the article, including (as applicable) the thesis and outline of the argument, the hypothesis and findings, and/or the general structure of the essay. Again, the standards of what is included will vary depending on the stylistic conventions of the discipline scholars are writing in.
Let's return one last time to Mihailidis and Cohen:
Mihailidis and Cohen now lay out that they will intervene in this scholarly conversation by discussing the specific example of one piece of online software and the ways it can be used as an instructional tool. In doing so, they outline the structure of the paper as well as point to the larger goals behind their choice to focus on this subject, which we would expect to find them discussing in the paper's conclusion.
This can all end up being an overwhelming process, so it's definitely worth it to have some tools available to help. Spreadsheets can be useful in this process. The one below was designed for doing literature reviews, but it's applicable to any situation where you have to read and analyze a number of sources. This spreadsheet aims for maximum flexibility and applicability, so you'll notice other columns in addition to Argument, Methodology, and Analysis columns that you may be expecting. You may find these columns helpful especially when you're trying to find trends and patterns around which you can organize your inquiries.
Download Literature Review Source Matrix Template.xls
RefWorks is free to you as a CIIS student, and can help with organizing your work and streamlining the creation of bibliographies. For more details, see our extensive tutorial here.
The research process will engage your emotions, past experiences, and biases. The trick is to reflect on that, rather than ignore it. Continual self-reflection is, therefore, a key ingredient in the research process. As you're reading a given source, as yourself:
How am I reacting to this?
What about my personal experience is shaping my reaction?
How can I consider other information or points of view?