The primary purpose of Phase 2 of the Replication and Reproducibility Project are to expose you to the academic literature surounding your research topic and/or question. A secondary purpose of this assignment is for you to develop a sense of how (un)common it is to find research articles in the top journals in the field of Criminology for which the authors have openly shared their data and code for reproducibility purposes. For more information about why and how to share data/code and other best practices for conducting reproducible research, check out, here, here, and here. :
Specifically, for this phase of the project, you will accomplish the following tasks:
Find five empirical research articles related to the topic or question you identified in Phase 1 and evaluate these five articles in terms of their adherence to basic open science practices (e.g., data availability, replication code availability, etc.).
Identify an empirical research article related to your topic or question that analyzes data from the NYS and review its characteristics.
Tentatively commit to performing a conceptual replication of an article that does not use the NYS or a direct reproduction of one that does and identify what aspects of the research article you plan to conceptually replicate or reproduce.
At this point, I assume you are familiar with RStudio and with creating R Markdown (RMD) files. If not, please review R Assignments 1 & 2.
Additionally, I assume you know how to search for articles and then access them where available; there are numerous ways to do this. If it were me, I might start by searching for articles using the search function on each journal’s webpage. Upon finding articles of interest, I might then search for the specific articles using the title, author, and/or keywords in Google Scholar; you can also do an “advanced search” for specific topics published in specific journals. If you decide to use Google Scholar, I recommend adding your university to the “library links” setting for quick authentication and access via your library. Alternatively, you might search for a specific journal directly via the university library. (See here for additional help with finding articles using our library website.)
Finally, I assume you know how to properly format an article citation (e.g., using APA or Chicago style); Google Scholar can also help you with getting the full citation.
Note: For this part of the assignment, you must search in the five specific journals listed above. In the second part of the assignment, aritlces from other journals will be possible.
[] followed immediately by parentheses () containing the link.[Brauer, Day, & Hammond 2019](https://journals.sagepub.com/doi/10.1177/0049124119826158) generates the following: Brauer, Day, & Hammond 2019In part 2 instead of finding an article and then looking for the data, we’ll reverse it and start with the NYS data and then find an article that analyzes it.
Go to the NYS Series landing page on ICPSR and click on the “Data-related Publications” tab to search for an article related to your topic that uses NYS data. - See instructions for Phase 1 of the Replication and Reproducibility project for details on how to do this.
Like with the articles in Part 1 above, provide the full citation for the article and a link to the article DOI.
Answer the following questions about the article: - Does the author(s) provide code or script for reproducing results in the paper?
Finally, tentatively commit to a “conceptual replication” or a “direct reproduction” of one of the articles you identified above by answering or doing the following:
Note: If the article is from Part 1 and does not use the NYS data, you will be conducting a conceptual replication. This will ultimately require that you can identify similar survey items measured in the NYS for at least some of the key variables in the article (you will identify these items in a future assignment).
Note: If the article is from Part 2 and thus does use the NYS data, it is likely you will be completing a direct reproduction
Write a brief paragraph about why you are interested in replicating or reproducing this study and any issues or problems you anticipate in trying to replicate or reproduce it.
Does the article include simple table(s) and/or figure(s) that presents basic descriptive statistics? If so, what is the title and page numbers of the tables and/or figures?
Create a "Conclusion section where you write about what you learned in this assignment and any problems or issues you had in completing it.
Upon completing the tasks in the previous sections, “knit” your final RMD file and save the final knitted html document to your “Assignments” folder in your LastName_CRM495_work folder as: LastName_CRM495_RR-Project-Phase2_YEAR_MO_DY.
Inside the “LastName_CRM495_commit” folder in our shared folder, create another folder named: RR_Project_Phase2.
To submit your assignment for grading, save copies of both your (1) “RR-Project-Phase2” html file and (2) your “RR-Project_Phase2” RMD file into the LastName_CRM495_commit > RR_Project_Phase2 folder. Remember, be sure to save copies of both files - do not just drag the files over from your “work” folder, or you may lose those original copies from your “work” folder.
Notice that in the above assignment and in the “Replication and Reproducibility Project” as a whole, we are drawing a distinction between “reproducibility” and “replicability” and, likewise, between reproduction and replication research. In a reproduction, the goal is to verify or repeat exactly some or all of the findings reported in a previous study using identical data and methods as the original study. Unfortunately, the terminology surrounding reproduction and replication is inconsistent and confusing. For example, some use the term “pure replication” to refer to what we call reproduction research (e.g., see Freese and Peterson 2017, pp.152-3). We note that our distinctions are consistent with those used in Ritchie’s (2020) book (which you are currently reading) and with others’ recent attempts to clarify terminology in this space (e.g., Patil, Peng, & Leek 2019).
In addition to distinguishing between reproducibility and replicability, we might also draw distinctions between different types of replications. Perhaps the most common is the distinction between a direct replication and a conceptual replication (cf. Crandall and Sherman 2016; Pridemore, Makel, & Plucker 2018, p.21). In a direct replication, one assesses the same theoretical or observational claim of a study using new data and measures that are collected or designed in such a way as to match the prior study’s design as exactly as possible, though perhaps with some notable exceptions (e.g., a larger sample size to improve statistical inferences). In contrast, a conceptual replication assesses the same theoretical or observational claim as a previous study using new data and/or new measurement procedures that are conceptually similar but not identical to those used in the previous study. we recommend reading Crandall and Sherman’s (2016) detailed discussion of these distinctions and their case for the relative utility of conceptual replications in advancing scientific progress; see also Nosek and Errington’s (2020) critique of these distinctions.
Underlying many of these terminological distinctions are differences in research procedures and research aims. For instance, drawing on Freese and Peterson’s (2017) typology of the different aims involved in replication and reproducibility research, reproduction research often aims to assess verifiability by attempting to reproduce or verify an original study’s findings using the same data and methods (e.g., code). Direct replications typically assess repeatability by testing whether the same findings emerge or repeat when applying the same methods to a new sample. Conceptual replications often assess repeatability, robustness, and/or generalizability of a theoretical or observational claim by, for instance, testing the original claim’s robustness to different measurement specifications using the same data or testing the claim’s generality to new samples (e.g., different groups or contexts). We recommend reading Freese and Peterson’s in-depth discussion of these aims; for convenience, we include their definitions (see 2017, p.152) of these four aims below.