Category: Evidence synthesis protocols

An evidence synthesis of the Agricultural Research for Development (AR4D) literature

~ Jeroen van der Heijden ~ Leave a comment

For 40 years, the Australian Centre for International Agriculture Research (ACIAR) has been encouraging and supporting research projects. To draw key-lessons from this 40-year period of research support, an impact evaluation will be carried out. It asks: What elements of the ACIAR model in practice are associated with the most successful project outputs and enduring outcomes in different contexts? To answer this question, the existing database of ACIAR research projects (n=40-100) will be evaluated using qualitative comparative analysis (QCA) logic and tools. This allows for uncovering the various (context-specific) modes of operating that have allowed ACIAR to make contributions to innovation system development.

A first step in a QCA analysis is to identify, define, and calibrate the causal conditions (”independent variables”) and outcomes of interest (“dependent variable”). This evidence synthesis of the Agricultural Research for Development (AR4D) literature aims to distill the most common conditions and outcomes from the literature to build a theoretical model that can be applied, deductively, in the ACIAR impact evaluation. Within the impact evaluation, additional conditions and outcomes may be traced in an inductive manner.

Thus, this evidence synthesis asks:

  • What are the dominant desirable outcomes of AR4D research support currently identified in the academic AR4D literature?
  • What are the dominant causal conditions currently identified in the academic AR4D literature as necessary and/or sufficient to achieve desirable outcomes of AR4D research support?

The aim of this evidence synthesis is to synthesize the evidence base of the academic AR4D literature that maps, explores, and interrogates the outcomes of AR4D research support. To this end, this evidence synthesis will answer the following questions:

  1. What desirable outcomes may be expected from AR4D research support?
  2. Under what circumstance, when, and how AR4D research support likely to result in these desirable outcomes?
  3. For questions 2, if heterogeneity is found in studies on AR4D research support: What is the role of context on the outcomes of AR4D research support?

The original protocol (10 August 2020) for the evidence synthesis is available here:

AR4D evidence synthesis protocol – v1Download

An evidence synthesis of the academic regulatory failure literature

~ Jeroen van der Heijden ~ Leave a comment

Regulatory failure is much talked about but little understood. Discussions about regulatory failure are often about different understandings of what can be expected of regulatory governance and public regulation. The rhetoric of regulatory failure (typically a blame game) easily (and often) overshadows the analytical explanation of it (Baldwin, Cave, & Lodge, 2012, Chapter 5; Breyer, 1979; Wilson, 1984). As a result, it remains unknown:

  • What types of failure can be distinguished from the evidence base of the academic literature on regulatory failure;
  • Under what circumstances regulation is likely to fail and how it fails;
  • What strategies are available and have been proven successful in the prevention of regulatory failure.

The aim of this evidence synthesis is to synthesize the evidence base of the academic regulatory failure literature and evaluate the effectiveness (and lack thereof) of strategies to prevent regulatory failure in real-world situations. To this end, this evidence synthesis will answer the following questions:

  1. Under what circumstance, when, and how is regulation likely to fail?
  2. What evidence-based typology can be distilled from the regulatory failure literature?
  3. What strategies are applied by governments and other regulators to prevent regulatory failure and with what level of success?
  4. For questions 1 and 3, if heterogeneity is found in studies on regulatory failure: What is the role of context on the occurrence and prevention of regulatory failure?

The original protocol (10 August 2020) for the evidence synthesis is available here:

Regulatory Failure – evidence synthesis protocol v1Download

An evidence synthesis of the academic responsive regulation literature

~ Jeroen van der Heijden ~ Leave a comment

In 1992, Ayres and Braithwaite published Responsive Regulation. The responsive regulation strategy introduced in the book has become one of the most discussed regulatory strategies in the academic literature (Braithwaite, 2011; Parker, 2013). Yet, it remains unknown:

  • Whether (on average) responsive regulation outperforms the (counterfactual) regulatory strategies it replaces (i.e., traditional government-led command and control regulation, or laissez-fair market competition)
  • Under what circumstances responsive regulation works best.

Synthesizing the empirical knowledge base of the responsive regulation literature may help to fill these knowledge gaps. The aim of this evidence synthesis is to evaluate the effectiveness (and lack thereof) of responsive regulation applied to real-world situations. To this end, this evidence synthesis will answer the following questions:

  1. What is the breadth, purpose and extent of research activity on responsive regulation?
  2. Compared to the (counterfactual) regulatory strategies that responsive regulation replaces (i.e., traditional government-led command and control regulation, or laissez-faire market competition), what is the (average) comparative effectiveness of responsive regulation in achieving regulatory goals?
  3. What are the advantages and limitations of responsive regulation compared to the strategies it replaces (i.e., traditional government-led command and control regulation, or laissez-faire market competition)?
  4. For questions 2 and 3, if heterogeneity is found in studies on responsive regulation: Under what circumstances, in what situations, and for whom does responsive regulation provide better outcomes than the strategies it replaces (i.e., traditional government-led command and control regulation, or laissez-faire market competition)?

The original protocol (17 March 2020) for the evidence synthesis is available here:

Responsive Regulation evidence synthesis – protocol v1Download

A slightly updated version of the protocol (15 May 2020) is available here:

Responsive Regulation Evidence Synthesis – protocol v2Download

The final version of the protocol (another minor update; 3 August 2020) is available here:

Responsive Regulation Evidence Synthesis – protocol v3Download