https://www.edreplication.org/projects daily 0.75 2020-10-28 https://www.edreplication.org/announcements daily 0.75 2020-11-16 https://www.edreplication.org/announcements/blog-post-title-one-pwbjx monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fadf6ea4cb7230541423e07/1605236568787/kylie+angling.jpg Announcements - Apply for the PhD program in Education Policy or Research, Statistics, and Evaluation (UVA) - Kylie L Anglin Receives National Academy of Education/Spencer Dissertation Fellowship Read about Kylie Anglin’s dissertation work here. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fadf6701c7089551ab998a3/1605236376205/Anandita_0.jpg Announcements - Apply for the PhD program in Education Policy or Research, Statistics, and Evaluation (UVA) - Student Spotlight Anandita Krishnamachari talks about her graduate school experience at UVA. https://www.edreplication.org/announcements/blog-post-title-one-pwbjx-s8h7h monthly 0.5 2020-11-13 https://www.edreplication.org/announcements/blog-post-title-one-pwbjx-grsjw monthly 0.5 2020-11-13 https://www.edreplication.org/announcements/job-market-candidates monthly 0.5 2020-11-15 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fadf7d854d01722c182dc4b/1605236763218/kylie+angling.jpg Announcements - Job Market Candidates - Kylie L Anglin Research Interests: Causal Inference, Natural Language Processing, Effect Heterogeneity and Implementation, and Replication Kylie’s Personal Website Kylie’s Ed Policy Works Site Website Kylie receives NAed/Spencer Dissertation Fellowship. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fadf8ef162257496b3ecbf2/1605236978200/Anandita_0.jpg Announcements - Job Market Candidates - Anandita Krishnamachari Research Interests: Teacher preparation policy, Causal inference, Randomized control trials, Replication studies, Quantitative methodology, Data architecture and infrastructure. Anandita’s Ed Policy Works Site Website Anandita’s UVA Student Spotlight. https://www.edreplication.org/rep-methodology daily 0.75 2020-11-16 https://www.edreplication.org/rep-methodology/crf monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c31abb01209401474dae3/1605276722489/CRF+chart.PNG Replication Methodology - The Causal Replication Framework Excerpted from Wong, Steiner, and Anglin (2020) https://www.edreplication.org/rep-methodology/implementation monthly 0.5 2020-11-16 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c3723e5a2d17f4d14fea0/1604073271635/data%2Bmanagment.jpg Replication Methodology - Implementation of Replication Studies - Data Engineering and Management for Collecting High Quality Data. Led by Anandita Krishnamachari, Alexis Prijoles, and Brian Wright. Data Science principles and practices that promote transparency and replicability in social and behavioral science research. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c35f8b5ca6249ccf8faea/1604073003084/image.jpg Replication Methodology - Implementation of Replication Studies - Natural Language Processing for Implementation Research Led by Kylie L Anglin and Brian Wright. Low-cost, scalable methods of monitoring implementation in field settings using natural language processing https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c379592d29c2275d6545a/1604684020361/teachsim%2Bworkflow.jpg Replication Methodology - Implementation of Replication Studies - Research Workflow and Systems Organization. Led by Christina Taylor and Alexis Prijoles. Systematic organizations of resources that work interdependently to complete research activities and obtain project goals with high fidelity across sites. https://www.edreplication.org/rep-methodology/analysis monthly 0.5 2020-11-16 https://www.edreplication.org/rep-methodology/design-based-approaches monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa2e40a583fea58d55f36f5/1605230510043/direct%25252Brep.jpg Replication Methodology - Research Designs for Replications - Direct Replications In research designs for direct replication, the researcher examines whether two or more studies with the same causal estimand produce the same effect after introducing systematic variations in the research design, estimation approaches, or reporting procedures. If results do not replicate, the researcher interprets the cause of the replication failure as bias in the research design or estimation procedure or incorrect reporting. Our team has done a lot of thinking about one type of direct replication — within-study comparisons (WSC) for evaluating non-experimental method performance. This approach examines whether a quasi-experimental design is able to replicate the effect from an experimental benchmark with the same target population. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c35342f476b45978ff063/1604510768513/conceptual%2Brep.jpg Replication Methodology - Research Designs for Replications - Conceptual Replications In research designs for conceptual replications, the researcher examines whether two or more studies with potentially different causal estimands produce the same effect. In prospective approaches, the researcher may introduce systematic variations in treatment conditions, in outcomes used for assessing effects, in target populations and in settings for conducting the study. In systematic conceptual replication studies, when replication failure is observed, the researcher is able to identify the source of the effect variation. https://www.edreplication.org/rep-methodology/why monthly 0.5 2020-11-14 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fafe64969c60d415b479980/1605363279605/ Replication Methodology - Why Replication Science? https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa2f829544bd13331b0059c/1605276615345/RepSci.png Replication Methodology - Why Replication Science? Creating the methodological foundations for a replication science. https://www.edreplication.org/replication-studies daily 0.75 2020-11-16 https://www.edreplication.org/replication-studies/design-replication monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa40c464a476a64db0eb373/1605276510798/design+rep+large.png Replication Studies - Design Replication Studies for Evaluating Non-Experimental Methods - Dependent-arm Design Replication Study Example of a dependent-arm design replication study that shares the same experimental treatment group https://www.edreplication.org/replication-studies/is2rw monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa315df1f1736710dba3e82/1604523526702/Balanced-Literacy-in-an-Intermedite-Classroom-1536x1024.jpg Replication Studies - Iterative Systematic Replication of Read Well in First Grade (IS2RW) https://www.edreplication.org/replication-studies/sera monthly 0.5 2020-11-13 https://www.edreplication.org/replication-studies/teachsim monthly 0.5 2020-11-13 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa61d12d9418b491278f47d/1604721942066/teachsim.jpg Replication Studies - TeachSim for Improving the Preparation of New Teachers https://www.edreplication.org/replication-studies/category/Intervention monthly 0.5 https://www.edreplication.org/related-projects daily 0.75 2020-11-19 https://www.edreplication.org/related-projects/ed-evidence-mapping monthly 0.5 2020-11-19 https://www.edreplication.org/related-projects/sds-capstone-projects monthly 0.5 2021-01-29 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa3035c1d40d36a9bc1d412/1605537916740/datainfrastructure.jpg Related Projects - SDS Capstone Projects - Data Engineering Led by Anandita Krishnamachari, Alexis Prijoles, & Brian Wright. Software challenges arise because of the lack of cyber-infrastructure to support social and behavioral science research from end to end. The goal of this project is to address cyber-infrastructure concerns in Education Research by providing education researchers with tools and resources for developing, piloting, and testing data infrastructure systems to conduct high quality, systematic research in school settings. Learn More. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa302d002ba0a1008d5cd72/1605537923417/ Related Projects - SDS Capstone Projects - Natural Language Processing Led by Kylie L Anglin, Todd Hall, & Brian Wright. To improve intervention implementation, researchers need some method of tracking fidelity in large-scale interventions, ideally in real-time, so that they can provide additional training to educators who are not implementing the intervention as designed. The goal of this project is to replicate, and extend or improve the document similarity measure as currently implemented in Anglin and Wong (2020) using NLP. https://www.edreplication.org/home daily 1.0 2020-11-16 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9c2e108986d1436f04074c/1604071056435/collaboration.jpg Collaboratory Replication Lab - Collaboratory Replication Lab Developing Methods for a Replication Science https://www.edreplication.org/ourteam daily 0.75 2020-12-08 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa844ab92c73f0939d2aaa8/1605279349471/peter.jpg Our Team Peter M. Steiner, Ph.D., Co-Principal Investigator Peter M Steiner is an Associate Professor in the Measurement, Statistics, and Evaluation (EDMS) program in the Department of Human Development and Quantitative Methodology at the University of Maryland. His research focuses on causal inference, replication, and factorial surveys. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fd00e6d4510f510134d1632/1607470812418/BRIAN_001_Web.jpg Our Team Brian Wright, Ph.D., Data Science Collaborator Brian Wright is an Assistant Professor and Director of Undergraduate Programs with UVA's School of Data Science. His work focuses on bridging the gap between data science and the social sciences. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa848648a3e5f40747fdfcf/1605279360151/Anglin_Headshot-e1591725701752-1016x1024.jpg Our Team Kylie L Anglin PhD candidate in Education Policy Kylie L. Anglin is a Ph.D. candidate in Education Policy at the University of Virginia. Her research focuses on developing and using data science approaches to examine variations in policy and intervention implementation, as well as the impact of intervention heterogeneity on student outcomes. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa844908a3e5f40747f327a/1605368688193/v%2525252Bwong.jpg Our Team Vivian C. Wong, Ph.D., Principal Investigator Vivian C. Wong, Ph.D., is an Associate Professor in Research, Statistics, and Evaluation at UVA. Dr. Wong’s expertise is in improving the design, implementation, and analysis of experimental and quasi-experimental approaches. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5faead05844f9e409e5717b0/1605283083225/Patrick_Sheehan%2Bheadshot.jpg Our Team https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa847628657be665e032c2a/1605279321385/headshot-may-2020_Alexis-Prijoles-e1591725531361-1024x910.jpg Our Team Alexis Prijoles, M.Ed., Data Manager Alexis Prijoles, M.Ed., is the Data Manager. She earned her M.Ed. in Education Psychology with a concentration in Research, Statistics, and Evaluation at the University of Virginia. Her work focuses on data management and data infrastructure in social and behavioral science research. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa8478c4e5ce030c7b17d2a/1605279365698/Taylor_headshot.jpg Our Team Christina Taylor, M.Ed., Replication Manager Christina Taylor, M.Ed., is the Replication Manager. She earned her M.Ed. with a concentration in research, statistics, and evaluation at the University of Virginia. Christina oversees workflow and protocol development, and manages relationships with research partners. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa846dfea796d34de1030d1/1605279327749/Krishnamachari_headshot.jpg Our Team Anandita Krishnamachari, PhD candidate in Research, Statistics, and Evaluation Anandita Krishnamachari is a doctoral candidate in Research, Statistics and Evaluation at the University of Virginia's School of Education and Human Development. Anandita's work focuses on program evaluation and quantitative analysis. https://www.edreplication.org/our-support daily 0.75 2020-11-14 https://www.edreplication.org/natural-language-processing daily 0.75 2020-11-14 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5fa18b07d0509a7c1eebfd83/1604422414684/shutterstock_381294904.png Natural Language Processing https://www.edreplication.org/our-partners daily 0.75 2020-12-07 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9b337332a27a55b4ca58f3/1605105886568/Screen+Shot+2020-10-28+at+6.53.04+PM.png Our Partners - TeachSim for Preparing the Next Generation of Teachers TeachSim uses a mixed reality platform to provide teacher candidates with practice opportunities and targeted feedback for learning high quality pedagogical practices. The Collaboratory Replication team works with TeachSim to design a series of systematic replication studies to understand systematic sources of variation in coaching effects. Read about our work here. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9b337332a27a55b4ca58f6/1605105827846/Screen+Shot+2020-10-28+at+6.50.23+PM.png Our Partners - ED-SDS Collaboratory The Ed-SDS Collaboratory was founded on the ideal that interdisciplinary collaboration can greatly enhance research outcomes. The frontiers being redefined in the field of Data Science present a unique opportunity for Education to leverage these advances to enhance research outcomes and to create a new generation of education researchers that have the ability to envision problems through a Data Science perspective. Read about our work here. Visit the website here. https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5faddcf8ff73a225b2682a26/1605229841917/Screen%2BShot%2B2020-10-28%2Bat%2B6.46.23%2BPM.jpg Our Partners - Special Education Research Accelerator (SERA) SERA is a platform for conducting research in special education with large and representative study samples across multiple research sites and researchers. The long-term vision for SERA is to develop a network of researchers along with a data collection platform for for conducting high-quality, large-scale, and open replication studies to address critical questions in the field. Read about our work here. https://www.edreplication.org/research-workflow-and-systems-org daily 0.75 2020-11-14 https://www.edreplication.org/data-engineering-and-management daily 0.75 2020-11-14 https://static1.squarespace.com/static/5f999ac992a983105893fc79/t/5f9b3d92054e8110e3c12bb9/1605230740268/SERA%25252BData%25252Blifecycle.jpg Data Engineering and Management - Researcher Supports It is important to make data management considerations at all points of the data life-cycle. An illustration of the data life-cycle is provided here. Although the diagram suggests that this is a linear process, the actual process may include stage overlap or iterations. To support researchers in making these considerations and implementing data science best practices, we are creating a Data Management Protocol. The Data Management Protocol will provide an overview of the data management principles and practices that promote transparency and replicability, and helpful templates and links. https://www.edreplication.org/paper-and-publications daily 0.75 2020-11-13 https://www.edreplication.org/events daily 0.75 2020-11-19 https://www.edreplication.org/seer daily 0.75 2020-11-18 https://www.edreplication.org/20202021-capstone-team-data-engineering daily 0.75 2021-04-26