Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 303

The art in policy Delphi practice: critical decisions for its design and

application on foresight exercises

El policy Delphi en la práctica: decisiones críticas para su diseño y aplicación

en ejercicios de prospectiva

Silvia Vicente Oliva 1*

1 University of Zaragoza, Department of Business Organization and Management. Spain.

* Corresponding author: silviav@unizar.es

Received 2025-09-29. Approved 2025-12-09

DOI: https://doi.org/10.26754/ojs_ais/ais.20254712353

Abstract

The Delphi method was initially developed in the mid-20th century as a tool for exploring future

technological developments, and its variant known as the policy Delphi was developed in 1970 to

incorporate the ideas of different stakeholder groups in foresight exercises. A review of more than

one hundred exercises that employed this technique up to 2025 allows for a reflection on its practical

implementation and on the crucial decisions involved during its design, execution, and analysis phases.

Fourteen crucial decisions have been identified for the design, execution, as well as the analysis and

interpretation of foresight exercises, which concern the balance in areas such as theoretical

considerations, participation of all stakeholder groups, uncertainty management, breadth of analysis,

methodological approaches, the use of collective expertise to examine policy instruments and

decisions, case-based experiences, and decision-making informed by dominant patterns of thought.

Keywords: Foresight; Futures; Methods; Project Design.

Resumen

El método Delphi que se diseñó para indagar acerca del futuro de la tecnología a mediados del siglo

XX, tiene una variante llamada policy Delphi que se desarrolló en 1970 para incluir las ideas de

diferentes grupos de interés en ejercicios de prospectiva. Una revisión de más de cien ejercicios que

utilizaron esta técnica hasta 2025, permite una reflexión acerca de su implementación práctica y

decisiones cruciales durante su diseño, ejecución y fases de análisis.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 304

Se han identificado catorce decisiones cruciales para el diseño la ejecución, así como el análisis de los

ejercicios de prospectiva que comprenden el equilibrio en áreas como consideraciones teóricas,

participación de todos los grupos de interés, gestión de la incertidumbre, amplitud del análisis,

enfoques metodológicos, aprovechamiento de la experiencia colectiva para analizar instrumentos y

decisiones políticas, experiencias basadas en casos y la toma de decisiones atendiendo al pensamiento

dominante.

Palabras clave: Diseño de Proyecto; Futuro; Prospectiva; Metodología.

INTRODUCTION

The Delphi method was developed in the mid-20th century to “obtain the most reliable consensus of

opinion of a group of experts [...] by a series of intensive questionnaires interspersed with controlled

opinion feedback” (Linstone & Turoff, 2002, p. 10). Over time, researchers refined and adapted the

method to better align with diverse research objectives, and Turoff (1970) introduced the concept of

the policy Delphi to facilitate expert discussions and generate multiple perspectives on complex

issues, particularly in the context of foresight. This approach has been applied in settings where

information is scarce, alternative scenarios could be explored, and divergent viewpoints are essential

for understanding both consensus and dissent (Beiderbeck et al., 2021; de Loë et al., 2016; Linstone

& Turoff, 1975; Turoff, 1970; Klenk & Hickey, 2011; Meskell et al., 2014).

Academic contemporary research employs policy Delphi across fields such as sociology, education,

energy, health, and security (e.g., Loughlin & Moore, 1979; Sharples et al., 2009; Uehara et al., 2021;

Walpole et al., 2015), although there is no universally accepted framework for designing, managing,

and analysing these studies. Typically, a steering committee design the exercise and selects a panel

of participants to complete questionnaires, often delivered remotely through online platforms

supporting iterative rounds. At least two rounds are required, as the second enables participants to

review aggregated feedback—such as means, distributions, and deviations—from the previous

iteration, fostering informed and participatory responses, sometimes in real time (Gordon & Pease,

2006). Policy Delphi is frequently combined with other foresight methods to enhance precision and

scope, either before or after the Delphi exercise. Building on Turoff’s seminal work, a myriad of

creative adaptations has emerged to engage diverse stakeholders around complex problems.

This study addresses the following research question: How can policy Delphi exercises be more

effectively designed, conducted, and interpreted? The aim is to provide foresight practitioners a

structured reflection on key decisions, trade-offs, and implications at each stage of the process. Given

the challenges of preparing for the future (exploratory approach) or shaping it (normative approach),

choices made by organizers can significantly influence outcomes. Policy-makers, researchers, and

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 305

decision-makers across domains—such as healthcare, education, environmental policy, and strategic

planning—can leverage participants judgment to inform interventions and programs. Consequently,

the findings of this study offer practical guidance for those planning, analysing, or implementing policy

Delphi exercises.

The article presents a theoretical framework and a systematic review of policy Delphi studies,

culminating in fourteen practical decisions for organizers to consider before implementation. The

conclusions highlight key insights and suggest directions for future practice.

THEORETICAL FRAMEWORK

Futures Studies and Their Vocabulary

Modern futures studies emerged from forecasting, largely enabled by the development of

computational quantitative methods in the twentieth century. Forecasting involves extrapolating

predictions from present conditions and historical trends (Armstrong, 1985; Peter & Jarratt, 2015); in

contrast, the term foresight refers to processes that incorporate interactive time horizons, open

reflection, relationship management, consultation, and dialogue aimed at shaping shared future

visions and identifying future opportunities. Foresight methodologies—such as the policy Delphi—

provide analytical systems capable of anticipating alternative futures and visualizing their

consequences through holistic and creative perspectives (Fuerth, 2009; Kadtke & Wells III, 2014), and

they can also promote new ways of understanding in contemporary contexts (Andersen & Borup,

2009).

Foresight differs from traditional long-term planning by recognizing that discontinuities may arise.

Hence, methods that rely on past events and empirical data—such as econometric models or patent

analysis—can constrain open-minded responses to disruptive changes. While foresight exercises

often begin by considering recent history and current conditions, linear extrapolation in dynamic

environments paves the way to error.

Popper (2008) classifies foresight methods into four categories: evidence, creativity, interaction, and

expert judgment. Evidence-based approaches draw on historical data to predict future developments,

whereas creative methods generate images, business models, and management practices to suggest

plausible alternatives. Participatory and interactive processes enable stakeholders to anticipate,

recommend, and transform systems, contributing to shared visions and strategies for social, political,

economic, or technological change. Expert-based approaches provide access to up-to-date empirical

knowledge and leverage the capacity of specialists to extrapolate, imagine, and collaborate on future-

oriented judgments.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 306

Although the Delphi method was originally designed for expert judgment, policy Delphi introduces

variations that incorporate stakeholder interaction, creative idea generation, and integration with

other research methodologies, aligning with Popper’s multidimensional framework.

Evolution of Futures Studies and Policy Delphi: Current State

Humanity has long sought to anticipate the future to prepare for change. However, as an academic

discipline, futures studies are generally traced back to developments in France during the 1960s, led

by authors such as Gaston Berger and Bertrand de Jouvenel, whose perspectives were especially

critical and pessimistic toward technological progress (Andersen & Rasmussen, 2012; Miles et al.,

2008). By the 1980s, the field adopted a global, pragmatic focus on major social changes and emerging

trends, consolidating analytical techniques and launching national foresight exercises.

Developed at RAND in the post–World War II era, the Delphi method served as a systematic

forecasting instrument designed to predict technological and military advancements among potential

adversaries—particularly the Soviet Union—thereby informing United States of America defence

strategies throughout the Cold War. Face-to-face expert discussions often failed due to hierarchy and

group pressures, so Helmer, Dalkey, and Rescher introduced anonymous questionnaires with iterative

feedback, allowing participants to revise opinions without external influence and converge toward

collective judgments.

In 1970, Turoff proposed the policy Delphi, shifting the focus from consensus to exploring divergent

stakeholder positions (Turoff, 1970). Since then, numerous adaptations have emerged, supported by

advances in information and communication technologies. Online platforms reduced turnaround

times, expanded panel diversity, and enabled real-time Delphi with continuous feedback (Gordon &

Pease, 2006), alongside hybrid approaches integrating bibliometrics, simulation, big data analytics,

and scenario studies.

Conceptual and Methodological Approaches to policy Delphi

The distinction between exploratory and normative approaches shapes the design of foresight

exercises: exploratory studies investigate potential changes, while normative one’s outline pathways

toward desirable futures, requiring stronger commitment to outcomes. Within policy Delphi, Turoff

(1970) identified three roles: the steering group, expert panel, and end-users, allowing the method to

operate under both frameworks.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 307

Futures research often adopts transdisciplinary and interdisciplinary perspectives to address complex

social realities (Nikolova, 2014). Expert selection depends on research scope, but policy Delphi

remains flexible, enabling diverse panels and ensuring anonymity to reduce social pressures.

However, limitations common to social sciences—such as replication challenges, sample

representativeness, and variability in expert engagement—persist (Vicente & Martínez, 2024). These

issues highlight the need for transparent documentation of methodological choices and analytical

strategies to strengthen credibility.

Practical Development of Foresight Exercises

Foresight operates as a continuous, iterative cycle. Cuhls (2003) describes a process beginning with

theme identification, followed by questionnaire design, application of research techniques, and

participatory discussions that generate feedback for new themes. Within this broader cycle, a specific

foresight exercise typically comprises three phases: planning, implementation, and

monitoring/dissemination (Andersen & Rasmussen, 2012), and Horton (1999) described three stages:

gathering and synthesizing trends, interpreting their implications for the organization, and integrating

this understanding into actionable commitments.

METHODOLOGY

The bibliometric analysis was conducted in accordance with the PRISMA 2020 guidelines (Page et al.,

2021), while also considering the scope of this study and the available data through the systematic

approach proposed by Zupic and Čater (2015). This methodology was designed to address the

research question: How can Policy Delphi studies be more effectively designed, conducted, and

analysed? Figure 1 provides a schematic representation of the process.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 308

Figure 1.

PRISMA 2020 flow diagram for new systematic reviews which included searches of databases and registers only

Source: Adapted from Page et al, (2021) (CC BY 4.0)

First, studies published in the ScienceDirect and Web of Knowledge databases were retrieved using

the keywords “policy Delphi” and related terms such as “foresight” and “foresight exercise”. The

search aimed to identify foresight exercises incorporating the Policy Delphi methodology. A total of

205 references were found, covering the period from the earliest record—the first article by Turoff in

1970—up to April 1, 2025. Of these, 203 were analysed in English due to their broader international

impact, while publications in Polish and Portuguese were excluded as they targeted more restricted

audiences. The initial corpus comprised 181 research articles, 9 book chapters, 7 conference papers,

2 review articles, 2 doctoral theses, one letter to the editor, and one scientific meeting abstract.

Duplicate studies—those published in multiple formats, such as both a conference paper and a journal

article—were removed, resulting in a refined set of 180 references.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 309

The screening prioritized works by citation counts to identify key contributions and assign thematic

categories. Sixteen purely methodological studies were excluded, leaving 164 for review. For each,

data covered method combinations, expert selection, panel size, participation, involvement of

futurists or policymakers, rounds, origin, and international scope, plus free-text observations. Finally,

studies were ranked by citation count, and key information was systematically documented in Table

1 to support subsequent analysis.

Table 1. Analysis Conducted on the Policy Delphi Studies Reviewed

Fiel

Description

DOI

Unique identifier for each reference

Tittle

Publisher work title

Authors

Full list of authors

Source

Journal, book, or conference where the work was published

Publication Year

1970-2025

Total Citations

Citations recorded up to April 1, 2025

Citations per Year

Average citations per year from publication to 2024

Topic

Category assigned for the study (e.g., business administration, education,

environment, medicine, security)

Combination of foresight methods

Indicates whether other methods were used before or after policy Delphi

to gather information or conduct analysis

Type of combined methods

Specific methods combined with policy Delphi

Expert selection method

Procedure used to identify and select panel experts

Kind of experts:

Description of participant profiles included in the panel

· Researchers

· Yes/No

· Policy-makers

· Yes/No

· Futurists

· Yes/No

Number of Participants

Total number of persons invited to join the study

Number of Participants that

completing study

Number of individuals who completed all rounds

Number of rounds

Total number of discrete rounds conducted

Country

Country where the exercise originated

Multiple countries

Yes/No

Other observations

Free-text notes on specific needs or comments related to the exercise

Source: Author’s Own Work

And finally, third, the data were synthesized (Step 8 of PRISMA), organizing the available information

to reveal the “structure and dynamics of the research field” (Zupic & Čater, 2015, p. 431). This process

enabled the interpretation phase (Step 10 of PRISMA 2020 and Step 5 of Zupic & Čater), which is

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 310

presented here as fourteen key decisions that sponsors and/or organizers should consider when

planning and conducting a policy Delphi exercise.

RESULTS

The Fourteen Decisions

These decisions derive from the analysis and are grouped according to the main phases of the

exercise.

PLANNING PHASE

Expert Judgment vs. “Wisdom of the Crowd”

Since policy Delphi exercises do not rely on representative sampling (Okoli & Pawlowski, 2004),

selecting experts becomes a demanding task for the organizing committee. While Turoff originally

worked with small expert groups, the opposite approach is also possible. Surowiecki (2004)

introduced the concept of “wisdom of the crowd,” highlighting the superior capacity of groups

compared to individuals in generating innovative public opinions and demonstrating that collective

predictions often outperform those of any single participant.

Four key elements underpin this approach: diversity of opinion, participant independence, group

decentralization, and aggregation mechanisms that transform individual judgments into collective

decisions (Davis-Stober et al., 2015; Wu et al., 2015). Furthermore, Spickermann et al. (2014)

distinguished between superficial diversity (e.g., age, gender) and deep diversity (e.g., values,

knowledge, learning curves), both of which merit consideration. Some reviewed studies report very

large panels—over 1,000 participants—combining experts and stakeholders to ensure diversity and

aggregate input across Delphi rounds (Pereira et al., 2018; Sharples et al., 2009), though

independence and decentralization were rarely explained. This underscores the need for clear

diversity criteria and robust aggregation mechanisms in future exercises.

Education vs. Indoctrination

Godet (2010) claimed that those who ignore the past cannot anticipate possible futures, although in

some topics it may be necessary to provide panellists with basic assumptions, particularly when

panels are heterogeneous rather than composed exclusively of specialists (Rowe & Wright, 2001).

Educating stakeholders can be an explicit objective of policy Delphi studies (Turoff, 1970), and it could

exist contexts where the question arises whether expertise can be developed through training or

extended practice when sufficient information is lacking, as suggested by Shanteau et al. (2002).

In exploratory exercises, where all contributions are welcome, uninformed participants may add

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 311

limited value, whereas in normative exercises their involvement implies a commitment to future

actions (Nikolova, 2014). Therefore, participants should possess substantial expertise; including those

without it is counterproductive unless they receive adequate briefing or training. Yet, any educational

intervention carries the risk of bias or indoctrination (Rourke, 1984). In normative policy Delphi

exercises, this risk is particularly critical, as training may introduce political, behavioural, or other

biases. Consequently, any informational intervention—including preparatory materials for panel

members—must be carefully justified and documented.

Anonymity vs. Constructive Forums

One of the defining features of the policy Delphi method is anonymity, which helps minimize the

influence of individuals seeking to impose their views on the expert panel. Current applications of this

methodology use to employ online platforms, offering convenience for participants and facilitating

efficient data collection (Moon & Baker, 2012), thus anonymity is easily preserved across rounds by

providing only aggregated feedback rather than individual responses. Turoff himself acknowledged

that committee meetings and analytical work could not be entirely replaced, as their purpose was to

gather perspectives to address any emerging circumstances (Turoff, 1970).

Nevertheless, anonymity does not preclude the integration of complementary approaches.

Combining policy Delphi with discussion groups or workshops—whether in-person or virtual—can

create constructive forums where participants interact openly, generating richer insights and broader

options for addressing complex issues affecting diverse stakeholders. If hybrid formats are adopted,

the design must be carefully justified, ensuring that such interactions do not compromise the

independence of participants or introduce undue influence that could bias the results.

Breadth vs. Depth

The objectives of policy Delphi studies often include exploring alternative scenarios, uncovering

underlying assumptions, fostering consensus among participants, and linking diverse topics from

multiple perspectives (Turoff, 1970). Consequently, the exploration process may require

questionnaires of considerable length, which can reduce response rates across successive rounds. In

opposition, overly concise instruments should produce risk to obtain superficial data, although follow-

up workshops may help compensate for this limitation (de Loë, 1995).

Striking an appropriate balance between breadth and depth during questionnaire design is therefore

a critical challenge. Pre-testing the questionnaires can provide valuable insights, yet it requires initial

assumptions about the time and effort panel members are willing to invest. This trade-off underscores

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 312

the importance of aligning the scope of inquiry with participant capacity to maintain both rigor and

engagement throughout the exercise.

Uncertainty vs. Quantitative Methods

Each round of questionnaires—and the final stage—requires systematic analysis. Descriptive analyses

are most common, encompassing both qualitative and quantitative dimensions, often complemented

by dissent analysis. Qualitative data in policy Delphi studies can be processed using commercial

software such as NVivo or Atlas.ti, which support systematic coding and thematic exploration.

Researchers also employ open-source alternatives like R, with packages tailored for textual data

analysis, or even ad hoc designs (e.g. Martínez & Durán, 2024).

Analytical tools should enable flexible interpretation of panellists’ responses while ensuring

transparency, yet detailed methodological reporting is uncommon. Key aspects—such as confidence

levels or optimism in predictions—are often neglected, despite evidence of polarization between

pessimistic and optimistic views (Beiderbeck et al., 2021). Understanding these variations can

strengthen data robustness and scenario implications. Although uncertainty complicates the use of

reliability measures or confidence intervals, incorporating them enhances validity and should inform

question design.

Foresight vs. Trends

The temporal horizon of the exercise is critical, due to expert projections could be classified into three

categories—possible, probable, and preferable—depending on how questions are framed (Bell,

1997). Voros (2003) expanded these categories through the “futures cone,” aiming to avoid linearity

in future-oriented research. Accordingly, foresight methods based on empirical data often employ

techniques such as econometrics, bibliometrics, or quantitative analysis (Popper, 2008). This

distinction between forecasting and foresight becomes evident when using these tools, as intuitive

forecasting has well-documented limitations (Lenz, 1962; Slocum & Lundberg, 2001).

In the context of policy Delphi, a key question arises: how can panellists deliberate on future issues in

the absence of empirical data or discernible trends? The less information available on the topics under

consideration, the greater the need for careful and deliberate expert selection. In such cases, it may

even be advisable to assess whether participants have prior experience in foresight or future-oriented

research—so-called “futurists.” Interestingly, none of the reviewed studies explicitly reported

including futurists among their panellists, underscoring the importance of clearly defining the type of

foresight to be pursued.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 313

Adaptation versus Fidelity

Managing a policy Delphi exercise often requires creativity to adapt the methodology to specific

political, cultural, or institutional contexts. Consequently, variants of the original method have

emerged to support diverse consultation formats (e.g., Haynes et al., 2016), frequently integrating

mixed methods from different disciplines to enhance flexibility. Adaptations may involve

reformulating questions, designing iterative rounds that balance openness and focus, and interpreting

results to accommodate both consensus and dissent. Such practices demand methodological

expertise rarely addressed in formal guidelines and often embedded as tacit knowledge within

research teams. Therefore, documenting and justifying every methodological decision is essential.

IMPLEMENTATION PHASE

Consensus versus Dissent

Early Delphi studies conducted with small groups often achieved high levels of agreement, sometimes

within only a few rounds. Incorporating divergent viewpoints—known a priori on certain issues—

would enrich a policy Delphi; however, this requires defining the degree of agreement that makes an

argument optimal in terms of representativeness, and on this point researchers remain divided. For

instance, Loughlin and Moore (1979) suggested a 51% agreement threshold based on the mean of a

parameter, whereas other authors advocate for 75–80%. Some studies consider agreement intervals

as a score of 3 (mean, median, or higher) on a four-point Likert scale, while others employ calculations

using the Interquartile Range. Regarding the timing for setting the agreement threshold, some

recommend establishing it during the study design phase (Edwards et al., 2013; Ward et al., 2019;

Williams & Webb, 1994), although in practice it is often more convenient to do so after the first round

has been tabulated.

A key strength of policy Delphi is balancing structured objectivity with contested assumptions. Unlike

traditional Delphi, which seeks consensus, policy Delphi values divergence. Yet implicit notions of

what counts as ‘credible’ often bias results toward agreement. The legitimacy of forward-looking

judgments—whether based on expert authority or aggregated stakeholder views—raises critical

questions: Which voices are included or excluded? What epistemic frameworks shape outcomes? To

what extent do prevailing norms influence projections? Exposing these assumptions is not only

methodological but a political and ethical imperative, clarifying whether the value lies in consensus

or in the richness of contributions.

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 314

Stakeholder Groups versus Conflicts of Interest

Expert panels typically include individuals from diverse backgrounds, primarily policy-makers and

researchers, although only fifty-five published studies explicitly detail this composition. Figure 2

illustrates the number of studies incorporating stakeholder groups (22), combined with other

categories such as researchers (11) and policy managers (11). However, a complete combination of

these categories was observed in only nine studies focused on areas such as energy, education,

environment, sustainable development, animal welfare, or tourism (see, for example, Collins et al.,

2009; Guimont & Lapointe, 2016; Kattirtzi & Winskel, 2020; Maxey & Kezar, 2015; Mehnen et al.,

2013; Rikkonen & Tapio, 2009). This finding indicates that steering committees value stakeholder

inclusion but mitigate conflicts of interest through careful selection and validation against other

expert views. Diverse panels, including researchers, enrich the process; however, conflicting opinions

should be cross-checked or supplemented, with documented justification. Ultimately, triangulation

and transparency are essential to address limitations and reduce uncertainty.

Figure 2. Type of experts included in the expert panel of the policy Delphi

Source: Author’s Own Work

Lack of Theory versus Partial Omniscience

Designing the questionnaire through which information will be elicited requires openness to exploring

all possible options. This, in turn, highlights the fact that existing theories may be insufficient to

explain future alternatives. Such theoretical gaps cannot be easily compensated by imagination alone,

understood as an expanded vision of substantive knowledge projected into the future (Loveridge,

2004).

Expert selection typically prioritizes individuals with recognized experience in the topic, though

finding specialists for every area can be challenging. Panels often include experts in specific fields who

may identify additional panellists during the process. As shown in Figure 2, researchers are commonly

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 315

included, likely to anchor theory and define the state of knowledge. Additional participants may be

added as needed, making it essential to assess whether certain arguments should carry greater weight

depending on the exercise’s purpose—raising questions about differential weighting and its criteria.

Mechanical Process versus Interpretative Judgment

Although often presented as a structured and systematic process, successful implementation

depends on the steering group’s judgment and ability to manage ambiguity. In practice, policy Delphi

requires continuous interpretative decisions—such as question design, participant selection, and

feedback synthesis—making it far from mechanical. Published studies rarely provide sufficient detail

to assess reliability or transparency, and procedural guidelines cannot anticipate situations that

demand ad hoc solutions.

Case Studies versus Generalization

Case studies showed that policy Delphi exercises rarely proceed as planned, as unforeseen

dynamics—such as stakeholder shifts or political pressures—often require real-time, ad hoc

adjustments (Havers et al., 2019; Bloor et al., 2015). These experiences suggest the method is as much

an art as a science across domains like health, education, and security. While theoretical frameworks

guide design, rigid applications fail to capture the complexity and adaptability needed in practice.

Documenting these variations remains essential for improving future exercises and consolidating

lessons learned.

MONITORING AND DISSEMINATION

Europe-USA centric versus rest of the planet

Attending to Eurocentrism about future studies Sardar said that: “Eurocentrism is all too evident in

this mode of inquiry from the way time and space are perceived, masculinity and technology are

privileged, social organisation and institutional arrangements are structured, and non-western

cultures made totally invisible” (Sardar, 2010, p. 182). Perhaps for this reason, it is evident that the

countries that have conducted the most policy Delphi studies are Canada, the United States, some

European countries, and Australia. If the future vision sought by such studies is more aligned with the

philosophical trends in how the future is understood in these countries, as opposed to how future

forecasting is perceived in other parts of the world, it goes beyond the scope of this article, which only

presents evidence from studies that have been conducted and published up to this point.

Although policy Delphi is often described as methodologically neutral, its application is shaped by

cultural, institutional, and geographical contexts. Assumptions such as the value of dissent, expert

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 316

authority, or feasibility of multiple rounds may not hold universally. In collectivist societies, anonymity

may conflict with norms favoring visible consensus, while hierarchical institutions can undermine open

exchange despite confidentiality. Geographic factors—such as digital access or political sensitivities—

also influence practice. These complexities call for a contextual, adaptable approach that avoids rigid

prescriptions and remains attuned to social dynamics.

Critique versus Technocracy

Traditional descriptions of this method tend to emphasize its capacity to produce rational and

consensual outcomes, yet they often overlook the inherently subjective and contested nature of

knowledge, as well as the influence of dominant thinking. In this context, dominant thinking is not

merely a reflection of majority beliefs but rather a strategic construction that reinforces the status

quo and suppresses alternative perspectives (e.g. Balthasar, 2024; Daheim & Uerz, 2008; Simonse et

al., 2023). Recognizing and critically analysing these dominant ideologies is essential for

understanding power dynamics and fostering social change.

When a foresight exercise challenges technocratic assumptions—favouring participatory or value-

based perspectives over expert-driven, data-centred approaches—it raises implementation

questions. Will recommendations grounded in alternative epistemologies gain legitimacy among

decision-makers accustomed to conventional norms, or face resistance as less objective or politically

inconvenient? While such critique enriches the process intellectually and ethically, it complicates

translating results into policy contexts that privilege expert authority and procedural rationality.

Steering groups must navigate competing narratives, power asymmetries, and value judgments

throughout the exercise. Table 2 summarizes fourteen key decisions for policy Delphi and their

practical implications.

Table 2

Summary of the 14 Trade-offs in Policy Delphi Practice

Trade-off

Key Tension

Practical Implication

1. Expert judgment vs. wisdom of

the crowd

Small, elite panels vs. large, diverse

groups

Define criteria for diversity and

aggregation mechanisms

2. Education vs. indoctrination

Context-setting vs. biasing

Justify and disclose preparatory materials

3. Anonymity vs. forums

Confidentiality vs. dialogue

Consider hybrid formats; justify design

4. Breadth vs. depth

Wide coverage vs. response fatigue

Pretest questionnaires; consider

workshops

5. Uncertainty vs. judgment

Lack of metrics vs. subjective

opinion

Incorporate measures of confidence and

optimism

6. Foresight vs. trends

Preferable futures vs. extrapolation

Specify foresight orientation clearly

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 317

7. Adaptation vs. fidelity

Modified methods vs. legitimacy

Justify and report methodological

changes

8. Consensus vs. dissent

Agreement vs. polarization

Clarify whether the study values

consensus or divergence

9. Stakeholders vs. conflicts

Inclusivity vs. bias

Disclose and triangulate stakeholder

views

10. Theory vs. omniscience

Absence of framework vs. reliance

on experts

State theoretical anchoring and limits

11. Mechanical vs. interpretive

Step-by-step tool vs. ad hoc choices

Document interpretive decisions

12. Cases vs. generalization

Context-specific insights vs. theory

Highlight lessons for transferability

13. Eurocentrism vs. diversity

Western bias vs. global perspectives

Adapt method to cultural contexts

14. Critique vs. technocracy

Reinforcing norms vs. challenging

power

Reflect on epistemological assumptions

Source: Source: Author’s Own Work

CONCLUSIONS

The policy Delphi method, grounded in expert judgment and iterative feedback, has proven to be a

flexible and adaptable tool for complex foresight exercises, whether used alone or combined with

other approaches. Over time, it has evolved to address diverse objectives and contexts; however, its

application entails trade-offs and methodological decisions that can significantly shape outcomes.

Balancing expert judgment with broader participation, managing tensions between consensus and

dissent, and adapting to political, cultural, and institutional settings underscore the method’s nuanced

nature.

This article reviewed studies employing policy Delphi and proposed practical decisions for its design,

implementation, and dissemination. While the method fosters exploration of alternative futures and

diversity of perspectives, its execution is complex, requiring careful participant selection, stakeholder

role consideration, and sensitivity to technocratic assumptions. Critiquing these assumptions enriches

foresight intellectually but may complicate the uptake of results within governance structures

privileging expert authority and procedural rationality. Successful exercises demand methodological

awareness beyond procedural guidelines, embracing ambiguity and dissent as legitimate analytical

dimensions.

Limitations of this review stem from incomplete reporting in published studies, where internal

decisions—such as expert selection or question design—rarely appear in detail. Future research

should enhance transparency, refine questionnaire design, and explore hybrid approaches.

Developing a standardized yet adaptable guide could improve practice while preserving flexibility.

Theoretical work should continue to advance understanding of strategic decision-making under

uncertainty, integrating diverse knowledge systems. Ultimately, the effectiveness of policy Delphi

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 318

depends on reconciling scientific rigor with creative flexibility to generate actionable insights for

decision-makers facing increasingly complex and uncertain futures.

ETHICAL CONSIDERATIONS

The Declaration of Helsinki (1964; current revision 2024), which establishes safeguards for vulnerable

people, ensures transparency in clinical trials and commitments to distributive justice as equity in

research. In the case of the research that has given rise to this article, it is based exclusively on

secondary sources, as it is a review of the literature and did not involve human participants, either as

subjects of study or as members of a research team, since it is an individual scholarly work.

AVAILABILITY OF DATA AND MATERIALS

The data used in this research are publicly available, and the analysis performed can be accessed upon

reasonable request to the author.

DECLARATION OF GENERATIVE AI AND AI-ASSISTED TECHNOLOGIES IN THE WRITING PROCESS

It was not used in the process of creation nor analysis. Copilot AI was solely consulted during the

writing stage of the research in English when it was required, for the purpose of verifying the meaning

of complex sentences, as well as to assist in reducing the length of the manuscript, which exceeded

6.000 words due to the requests and clarifications prompted by the reviewers.

CONFLICT OF INTEREST

The author declares that there is no conflict of interest.

FINANCING

The compilation of bibliographic references for the design of a robust and reliable policy Delphi was

carried out during the execution of the Ministry of Science and Innovation project «R&D&I Projects»,

within the national framework for Knowledge Generation and the strengthening of the scientific and

technological R&D&I system (2020–2022). Grant (PID2019-108036GB-I00), entitled “Rethinking the

Role of the Armed Forces in the Face of New Security Challenges.” The conclusions drawn from the

experience gained in this project made it possible to develop this article based on recent practical

experience. However, the design of a specific policy Delphi methodology for the Security and Defence

environment does not form part of the project’s deliverables.

Acciones e investigaciones Sociales. Nº 47 (2026) 
 
 Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0.        319 
 
AUTHORS’ CONTRIBUTIONS 
The  main  and  only  author  oversaw  the  documental  compilation,  analysis  and  composing  of  the 
manuscript,  with  the  correct  annotations  done  by  the  reviewers  that  suggested  a  more  logical 
distribution for an easier reading experience.  
 
REFERENCES 
Andersen, P. D., & Borup, M. (2009). Foresight and strategy in national research councils and research 
programmes.  Technology  Analysis  y  Strategic  Management,  21(8),  917–932. 
https://doi.org/10.1080/09537320903262280 
Andersen,  P.  D.,  &  Rasmussen,  B.  (2012).  Fremsyn:  Metoder,  praksis  og  erfaringer.  Styrelsen  for 
Forskning  og  Innovation.  Forskning:  Analyse  og  Evaluering  No.  1/2012 
http://www.fi.dk/publikationer/2012/ny-rapport-om-fremsyn/ny-rapport-om-fremsyn/ 
Armstrong, J. S. (1985). Long-Range Forecasting (2ª ed.). John Wiley & Sons. 
Balthasar,  D.  (2024).  Futuring  Fragility:  Embracing  uncertainty,  identifying  opportunity,  unlocking 
development. Development Policy Review, 42(S1). https://doi.org/10.1111/dpr.12779 
Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M. (2021). Preparing, 
conducting, and analyzing Delphi  surveys: Cross-disciplinary practices, new  directions, and 
advances. MethodsX, 8, 101401. https://doi.org/10.1016/j.mex.2021.101.401 
Bell, W. (1997). Foundations of futures studies, 2 vol. Transaction Publishers. 
Bloor, M., Sampson, H., & Dahlgren, K. (2015). Useful but no Oracle: Reflections on the use of a Delphi 
Group  in  a  multi-methods  policy  research  study.  Qualitative  Research,  15(1),  57–70. 
https://doi.org/10.1177/1468794113504103 
Collins,  J.,  Hanlon,  A.,  More,  S.  J.,  Wall,  P.  G.,  &  Duggan,  V.  (2009).  Policy  Delphi  with  vignette 
methodology  as  a  tool  to  evaluate  the  perception  of  equine  welfare.  Veterinary  Journal, 
181(1), 63–69. https://doi.org/10.1016/j.tvjl.2009.03.012 
Cuhls,  K. (2003).  From  forecasting  to  foresight  processes—new  participative  foresight  activities in 
Germany. Journal of forecasting, 22(2‐3), 93-111. 
Daheim,  C.,  &  Uerz,  G.  (2008).  Corporate  foresight  in  Europe:  from  trend-based  logics  to  open 
foresight.  Technology  Analysis  &  Strategic  Management,  20(3),  321–336. 
https://doi.org/10.1080/09537320802000047 
Davis-Stober, C. P., Budescu, D. V., Broomell, S. B., & Dana, J. (2015). The Composition of Optimally 
Wise Crowds. Decision Analysis, 12(3), 130–143. https://doi.org/10.1287/deca.2015.0315 
 
 

Acciones e investigaciones Sociales. Nº 47 (2026) 
 
 Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0.        320 
 
de  Loë,  R.  C.  (1995).  Exploring  complex  policy  questions  using  the  policy  Delphi  A  multi-round, 
interactive survey method. In Applied Geography (Vol. 15, Issue 1). 
de Loë, R. C., Melnychuk, N., Murray, D., & Plummer, R. (2016). Advancing the State of Policy Delphi 
Practice:  A  Systematic  Review  Evaluating  Methodological  Evolution,  Innovation,  and 
Opportunities.  Technological  Forecasting  and  Social  Change,  104,  78–88. 
https://doi.org/10.1016/j.techfore.2015.12.009 
Edwards, A., Hughes, G., & Lord, N. (2013). Urban security in Europe: Translating a concept in public 
criminology.  European  Journal  of  Criminology,  10(3),  260–283. 
https://doi.org/10.1177/1477370813483386 
Fuerth,  L.  S.  (2009).  Foresight  and  anticipatory  governance.  Foresight,  11(4),  14–32. 
https://doi.org/10.1108/14636680910982412 
Godet, M. (2010). Future memories. Technological Forecasting and Social Change, 77(9), 1457–1463. 
https://doi.org/10.1016/j.techfore.2010.06.008 
Gordon, T., & Pease, A. (2006). RT Delphi: An efficient,“round-less” almost real time Delphi method. 
Technological  Forecasting  and  Social  Change,  73(4),  321-333. 
https://doi.org/10.1016/j.techfore.2005.09.005 
Guimont, D., & Lapointe, D. (2016). Empowering Local Tourism Providers to Innovate through a Living 
Lab Process: Does Scale Matter? Technology Innovation Management Review, 6(11), 18–25. 
https://doi.org/10.22215/timreview/1031 
Havers, S. M., Martin, E., Wilson, A., & Hall, L. (2019). Implementation of government-directed policy 
in the hospital setting: A modified Delphi study. Health Research Policy and Systems, 17(1). 
https://doi.org/10.1186/s12961-019-0500-8 
Haynes, E., Palermo, C., & Reidlinger, D. P. (2016). Modified Policy-Delphi study for exploring obesity 
prevention priorities. BMJ Open. https://doi.org/10.1136/bmjopen-2016 
Horton, A. (1999). A simple guide to successful foresight. Foresight, 1(1), 5-9. 
Kadtke, J., & Wells III, L. (2014). Policy Challenges of Accelerating Technological Change: Security Policy 
and Strategy Implications of Parallel Scientific Revolutions (Número de Septiembre). 
Kattirtzi, M., & Winskel, M. (2020). When experts disagree: Using the Policy Delphi method to analyse 
divergent  expert  expectations  and  preferences  on  UK  energy  futures.  Technological 
Forecasting and Social Change, 153. https://doi.org/10.1016/j.techfore.2020.119924 
Klenk, N. L., & Hickey, G. M. (2011). A virtual and anonymous, deliberative, and analytic participation 
process for planning and evaluation: The Concept Mapping Policy Delphi. International Journal 
of Forecasting, 27(1), 152–165. https://doi.org/10.1016/j.ijforecast.2010.05.002 
 

Acciones e investigaciones Sociales. Nº 47 (2026)

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0. 321

Lenz, R. C. (1962). Technological forecasting, Report ASD-TDR 62-414. Nº AD-408,085. Clearing House

for Federal Scientific and Technical Literature.

Linstone, H. A., & Turoff, M. (Eds.). (1975). The delphi method (Vol. 1975, pp. 3-12). Reading, MA:

Addison-Wesley.

Linstone, H. A., & Turoff, M. (red.) (2002) The Delphi Method - Techniques and Applications, descarga

en: https://www.foresight.pl/en/manuals.html?page=8 (20/08/2025)

Loughlin, K. G., & Moore, L. F. (1979). Using Delphi to achieve congruent objectives and activities in a

pediatrics department. Academic Medicine, 54(2), 101–106.

https://doi.org/10.1097/00001888-197902000-00006

Loveridge, D. (2004). Experts and foresight: review and experience. J. Foresight and Innovation Policy,

1(2), 33–69.

Martínez, R. y Durán, M. (2024) Repensando el papel de las Fuerzas Armadas españolas ante los

nuevos desafíos a la seguridad. Centro de Estudios Políticos y Constitucionales.

Maxey, D., & Kezar, A. (2015). Revealing opportunities and obstacles for changing non-tenure-track

faculty practices: An examination of stakeholders’ awareness of institutional contradictions.

Journal of Higher Education, 86(4), 564–594. https://doi.org/10.1353/jhe.2015.0022

Mehnen, N., Mose, I., & Strijker, D. (2013). The Delphi Method as a Useful Tool to Study Governance

and Protected Areas? Landscape Research, 38(5), 607–624.

https://doi.org/10.1080/01426397.2012.690862

Meskell, P., Murphy, K., Shaw, D. G., & Casey, D. (2014). Insights into the use and complexities of the

Policy Delphi technique.

Miles, I.; Harper, J.C., Georghiou, L.; Keenan, M.; Popper, R. (2008). “The Many Faces of Foresight”. In:

The Handbook of Foresight, ed. Luke Georghiou et al., PRIME Series, 3-21.

Moon, N. W., & Baker, P. M. A. (2012). Assessing Stakeholder Perceptions of Workplace

Accommodations Barriers: Results from a Policy Research Instrument. Journal of Disability

Policy Studies, 23(2), 94–109. https://doi.org/10.1177/1044207311425383

Nikolova, B. (2014). The rise and promise of participatory foresight. European Journal of Futures

Research, 2(1). https://doi.org/10.1007/s40309-013-0033-2

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: An example, design

considerations and applications. Information and Management, 42(1), 15–29.

https://doi.org/10.1016/j.im.2003.11.002

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... y Alonso-

Fernández, S. (2021). Declaración PRISMA 2020: una guía actualizada para la publicación de

revisiones sistemáticas. Revista española de cardiología, 74(9), 790-799.

Acciones e investigaciones Sociales. Nº 47 (2026) 
 
 Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0.        322 
 
Peter, M. K., & Jarratt, D. G. (2015). The practice of foresight in long-term planning. Technological 
Forecasting and Social Change, 101, 49–61. https://doi.org/10.1016/j.techfore.2013.12.004 
Pereira, G. I., da Silva, P. P., & Soule, D. (2018). Policy-adaptation for a smarter and more sustainable 
EU  electricity  distribution  industry:  a  foresight  analysis.  Environment,  Development  and 
Sustainability, 20, 231–267. https://doi.org/10.1007/s10668-018-0119-x 
Popper,  R.  (2008).  How  are  foresight  methods  selected?  Foresight,  10(6),  62–89. 
https://doi.org/10.1108/14636680810918586 
Rikkonen, P., & Tapio, P. (2009). Future prospects of alternative agro-based bioenergy use in Finland-
Constructing  scenarios  with  quantitative  and  qualitative  Delphi  data.  Technological 
Forecasting  and  Social  Change,  76(7),  978–990. 
https://doi.org/10.1016/j.techfore.2008.12.001 
Rourke, F. E. (1984). Bureaucracy, Politics, and Public Policy. In The ANNALS of the American Academy 
of  Political  and  Social  Science  (Issue  1).  Little  Brown. 
https://doi.org/10.1177/000271626938500151 
Rowe, G., & Wright, G. (2001). Expert Opinions in Forecasting: The Role of the Delphi Technique. In 
Principles  of  forecasting:  A  handbook  for  researchers  and  practitioners  (pp.  125–144). 
https://doi.org/10.1007/978-0-306-47630-3_7 
Sardar, Z. (2010). The Namesake: Futures; futures studies; futurology; futuristic; foresight-What’s in a 
name? Futures, 42(3), 177–184. https://doi.org/10.1016/j.futures.2009.11.001 
Shanteau, J., Weiss, D. J., Thomas, R. P., Pounds, J., & Hall, B. (2002). How can you tell if someone is 
an  expert?  Empirical  assessment  of  expertise.  Emerging  Perspectives  on  Judgment  and 
Decision Research, 620–641. 
Sharples, M., Graber, R., Harrison, C., & Logan, K. (2009). E-safety and web 2.0 for children aged 11-
16.  Journal  of  Computer  Assisted  Learning,  25(1),  70–84.  https://doi.org/10.1111/j.1365-
2729.2008.00304.x 
Simonse,  L.,  Simons,  D.,  &  Skalska,  Z.  (2023).  Designing  foresights  by  communities:  A  new 
groundbreaker  role  for  strategic  design.  Design  Science,  9. 
https://doi.org/10.1017/dsj.2023.16 
Slocum,  M.  S.,  &  Lundberg,  C.  O.  (2001).  Technology  Forecasting:  From  Emotional  to  Empirical. 
Creativity and Innovation Management, 10(2), 140–154. 
Spickermann, A., Zimmermann, M., & von der Gracht, H. A. (2014). Surface- and deep-level diversity 
in  panel  selection  -  Exploring  diversity  effects  on  response  behaviour  in  foresight. 
Technological  Forecasting  and  Social  Change,  85,  105–120. 
https://doi.org/10.1016/j.techfore.2013.04.009 

Acciones e investigaciones Sociales. Nº 47 (2026) 
 
 Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial 4.0.        323 
 
Surowiecki, J. (2004). The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How 
Collective Wisdom Shapes Business, Economies, Societies and Nations. Doubleday. 
Turoff, M. (1970). The design of a policy Delphi. Technological Forecasting and Social Change, 2(2), 
149–171. https://doi.org/10.1016/0040-1625(70)90161-7 
Uehara, T., Sono, M., Tsuge, T., & Onuma, A. (2021). Can prior informed consent create virtuous cycle 
between  biodiversity  conservation  and  genetic  resources  utilization?.  Journal  of 
Environmental Management, 300. https://doi.org/10.1016/j.jenvman.2021.113767 
Vicente  Oliva,  S.,  y  Martínez,  R.  (2024).  Policy  Delphi  como  herramienta  de  prospectiva.  En 
Repensando el papel de las Fuerzas Armadas españolas ante los nuevos desafíos a la seguridad 
(Cap. 2, pp. 71–100). Centro de Estudios Políticos y Constitucionales. 
Voros,  J.  (2003).  A  generic  foresight  process  framework.  Foresight,  5(3),  10–21. 
https://doi.org/10.1108/14636680310698379 
Wallin, J. A. (2005). Bibliometric methods: Pitfalls and possibilities. Basic and Clinical Pharmacology 
and Toxicology, 97(5), 261–275. https://doi.org/10.1111/j.1742-7843.2005.pto_139.x 
Walpole, S. C., Mortimer, F., Inman, A., Braithwaite, I., & Thompson, T. (2015). Exploring emerging 
learning needs: a UK-wide consultation on environmental sustainability learning objectives for 
medical  education.  International  Journal  of  Medical  Education,  6,  191–200. 
https://doi.org/10.5116/ijme.5643.62cd 
Ward, S., Borden, D. S., Kabo-Bah, A., Fatawu, A. N., & Mwinkom, X. F. (2019). Water resources data, 
models  and  decisions:  International  expert  opinion  on  knowledge  management  for  an 
uncertain  but  resilient  future.  Journal  of  Hydroinformatics,  21,  32–44. 
https://doi.org/10.2166/hydro.2018.104 
Williams, P. L., & Webb,  C.  (1994).  The  Delphi technique: a  methodological discussion.  Journal of 
Advanced Nursing, 19(1), 180–186. 
Wu, T., Chen, L., Hui, P., Zhang, C. J., & Li, W. (2015). Hear the whole story. Proceedings of the VLDB 
Endowment, 8(5), 485–496. https://doi.org/10.14778/2735479.2735482 
Zupic, I., & Čater, T. (2015). Bibliometric Methods in Management and Organization. Organizational 
Research Methods, 18(3), 429–472. https://doi.org/10.1177/1094428114562629 
 
 