Methodology

A new standard for CDR evidence synthesis

ESROC’s methodology is built on the conviction that decisions about carbon dioxide removal, which will shape the climate for decades, must be grounded in the best available scientific evidence, synthesised transparently and consistently.

To achieve this, ESROC draws on a high methodological standard: the systematic review. Unlike narrative reviews, which rely on expert selection of the literature and are vulnerable to bias, systematic reviews follow a clearly defined protocol to search for, screen, extract and synthesise evidence in a way that is reproducible and comprehensive.

What makes ESROC distinctive is not just the use of systematic reviews, but the way they are organised as an interconnected ecosystem: a suite of harmonised reviews on different CDR options and cross-cutting themes, designed to be both deep and comparable (see figure on the right).

Standards and Reporting

ESROC’s review protocols conform to the ROSES reporting standards (RepOrting Standards for Systematic Evidence Syntheses) developed for environmental systematic reviews and maps, and follow the guidelines of the Collaboration for Environmental Evidence (CEE). These standards require that search strategies be transparent and replicable, and that inclusion/exclusion decisions be documented and justified.

Each review in the ecosystem has its own study protocol, developed in line with the common template agreed by the Scientific Steering Committee and published alongside the review. The protocol specifies the research questions to be addressed, the search strategy, the inclusion and exclusion criteria for studies, the data extraction themes, and the approach to evidence synthesis. This ensures that the process is independently verifiable.

Figure: A simplified representation of how an ecosystem of systematic reviews can fit into the broader academic knowledge landscape for CDR, showing how information flows between different knowledge products.

The Scientific Steering Committee

Consistency across the ecosystem is maintained by a Scientific Steering Committee (SSC), whose members are drawn from all review teams. The SSC developed the common protocol template that underpins every review in the ecosystem, and it serves as the forum for resolving methodological questions that arise during the review process.

The Committee prevents the ecosystem from fragmenting into a loosely connected set of reviews that are difficult to compare. It brings together researchers from different disciplines which helps to prevent disciplinary silos and enables interdisciplinary learning across the ecosystem

Explore ESROC’s Partner Network

From a shared literature map to individual reviews

The starting point for the ecosystem was a systematic evidence map of the CDR literature, built following the methods described by Lück et al. (2025) and covering over 112,000 studies published between 2010 and 2025.

This map was constructed by searching major bibliographic databases – Web of Science and Scopus – using carefully developed search queries for each CDR method and cross-cutting topic. Later updates also include the OpenAlex and Dimensions databases. Literature management and deduplication is handled using NACSOS2, a purpose-built review management platform developed at the Potsdam Institute for Climate Impact Research that supports the full systematic review workflow from database retrieval through to coding and synthesis.

Machine learning for literature screening

The sheer volume of the CDR literature makes manual screening of every record neither practical nor efficient. Many of the reviews in ESROC used machine-learning classifiers to automate the most labour-intensive part of the screening process.

The classifiers were trained on thousands of hand-labelled records, annotated by trained coders from different disciplinary backgrounds. For each CDR method, a separate binary classifier is trained to distinguish relevant from non-relevant studies. The performance of each classifier is evaluated using two key metrics:

  • Precision: Of the studies the classifier identifies as relevant, how many actually are?
  • Recall: Of all the relevant studies that exist, how many does the classifier capture?

These metrics are reported transparently so that users can understand the uncertainty in how much literature may have been inadvertently included or missed.

How screening works in practice

This graphic visualizes the extraction, screening and coding process.

For reviews with large literature bodies, such as BECCS, soil carbon sequestration, afforestation and reforestation, biochar and DACCS, machine-learning screening process is as follows:

  1. A team of trained coders manually screens the titles and abstracts of a subset of publications using NACSOS2, applying agreed inclusion and exclusion criteria.
  2. This manually screened subset is used to train the machine-learning classifier.
  3. The classifier then processes the remaining, unscreened records, ranking them by predicted relevance.
  4. Human reviewers focus their attention on the high-ranked records and validate the classifier’s outputs.

This hybrid approach combining the efficiency of automation with the judgement and quality control of human experts enables a much larger volume of literature to be assessed than would be feasible with a fully manual approach.

The carbon footprint of machine learning

ESROC uses machine learning for literature screening, a decision that has energy and emissions implications worth being transparent about.

Our estimate for the energy required to rank unseen records from a CDR-related search query, based on a 15-minute runtime on a 170W GPU and 65W CPU to train on 2,000 labels and predict on 35,000 records, is approximately 70 Wh. This leads to emissions of around 23 gCO₂e (using the average German electricity emissions intensity of 329 gCO₂e/kWh in 2023). For context, the average German office worker consumes around 15 kWh per working day. The machine-learning screening step is therefore both low in energy use and could save energy compared to a fully manual screening process. 

Study coding and data extraction

Once relevant studies have been identified, review teams read and manually code them according to a set of common themes agreed by the Scientific Steering Committee. These themes, covering the literature landscape, technology status, CDR potential, techno-economic assessment, side-effects, MRV and interactions with other CDR options, are standardised across the ecosystem to enable consistent data extraction and cross-review comparison.

Where multiple studies report on the same quantity, for example, the cost of carbon removal for a particular technology, data is extracted and normalised for comparability: converted to common units, adjusted for currency and year, and documented with the key assumptions behind each estimate. This helps cross-cutting reviews to generate meaningful comparisons across CDR options.

Some themes are flexible: for topics where the relevance and knowledge base vary substantially across CDR options, such as non-CO2 impacts, spatial performance differences and policy landscape, review teams decide whether to include them as specific coding themes.

Evidence synthesis and critical appraisal

After coding, each review team undertakes a critical appraisal of the evidence and synthesises key findings. Where the literature permits, this includes consolidated quantified information, for example, ranges of cost estimates for a given technology type, or estimates of technical potential under different assumptions.

ESROC does not impose a single common synthesis method across all reviews, recognising that the appropriate approach depends on the nature and volume of the available evidence. Instead, the SSC facilitates knowledge exchange between teams.

A key part of the synthesis process is the transparent characterisation of uncertainty: identifying where evidence is robust, where it is limited or contested, and where the conclusions of reviews may be sensitive to the particular assumptions or scope of the studies included.

A two-way relationship with modelling and scenario analysis

One of the aims of ESROC is to interacts with and inform the modelling and scenario analysis community. Integrated Assessment Models (IAMs) and Earth System Models are critical tools for understanding the scale and pace of CDR needed under different climate futures, but their outputs depend heavily on the quality of input assumptions about CDR technologies, costs and constraints.

ESROC systematic reviews provide updated, critically appraised techno-economic and environmental parameters that can feed directly into model development. In turn, scenario analyses are themselves sources of evidence that ESROC reviews assess and synthesise, for example, on the role of different CDR options across mitigation pathways.

ESROC aims to draw from multiple lines of evidence, including not only IAM scenarios but also from CDR portfolio frameworks that use tools such as multi-criteria analysis, cost-optimisation methods, and decision-making under deep uncertainty.