Measuring Policy Influence: Causal Attribution, Research Uptake, and Public Decision-Making: An Integrated Theory of Influence Generation, Observation, Partial Identification, Policy Feedback, and Public Value

Abstract

Policy influence is generated through research, political authority, economic resources, institutional rules, organizational capacities, and previous public decisions. Yet the evidence through which influence is assessed is produced by a different set of processes: recordkeeping requirements, selective documentation, sampling designs, data classifications, and institutions' responses to measurement itself. This paper develops an integrated theory of policy influence that links these two processes without conflating documentary visibility, substantive knowledge use, actual causal participation, net policy effects, and public value. Building on primary research in policy processes, power, knowledge utilization, causal inference, measurement validity, and decision theory, it defines five policy-related relations and seven primary targets, then specifies how causal structures determine which observations could distinguish rival accounts. The analysis develops a claim-specific, multi-axis assessment; formalizes partial identification under incomplete and dependent evidence; and explains why source-specific pathways can be strongly supported while their net counterfactual effects remain unidentified. Studies of municipal research delivery, organizational adoption, source credibility, administrative measurement, and policy feedback anchor the argument in observed institutional behavior. A final decision-theoretic account separates sampling uncertainty, effect magnitude, minimum policy-relevant thresholds, normative evaluation, and lawful feasibility. Under some conditions, an identified interval is sufficient for a robust action even when no point estimate is available; under others, additional evidence should be obtained because it could change a consequential choice. The paper distinguishes propositions derived from established causal and measurement theory from empirical predictions about trace visibility, dependence-sensitive attribution, diffusion and policy feedback. It specifies observations capable of weakening the latter while reserving claims about the comparative performance of any operational measurement protocol for independent evaluation. Its practical aim is credible policy impact measurement that helps institutions decide what evidence to trust, what remains unknown, and what to do next.

Keywords: policy influence; policy impact measurement; causal inference; research uptake; knowledge utilization; partial identification; evidence-based policymaking; process tracing; policy feedback; measurement validity; public value; decision-making under uncertainty.

1. The Policy Influence Problem: Why Apparently Conflicting Assessments Can Both Be Correct

1.1 Different outcomes are often described by the same word

On 26 June 2019, the United Kingdom amended its statutory climate target for 2050, replacing an 80% emissions reduction requirement with 100% relative to the 1990 baseline. The legal text establishes a consequential change in the country's binding target; it supplies no contemporaneous observation showing that national emissions had already reached net zero [1]. That distinction is elementary once the variables are named. It is regularly obscured when a change in a policy commitment, the implementation of that commitment, and the achievement of its intended environmental result are described collectively as impact.

A more revealing case comes from the United States Hospital Readmissions Reduction Program (HRRP). Studying the apparent reduction in risk-adjusted hospital readmissions, Ody and colleagues found that accounting for a change in electronic claims standards, which permitted more diagnoses to be coded per claim, reduced the previously estimated decline by 48%. After additional adjustment for baseline differences between comparison groups, the relative decline for targeted conditions could not be statistically distinguished from declines in their comparison samples [2]. The original claims were measuring risk-adjusted recorded readmissions, an outcome affected by documentation rules. The study did not establish that hospitals had produced no beneficial behavioral changes under any possible specification. It established that an important interpretation of the recorded decline had depended on how risk and comparison groups were constructed.

These cases expose a general problem in policy influence measurement. The object assessed may be a decision, a rule, its implementation, a person's behavior, a recorded statistic, or a social outcome. A policy organization may also report the number of times its research was cited. Such a count concerns the visibility or documentary reception of knowledge. Establishing that a particular study changed the decision, or that the decision improved welfare, requires different evidence. The distinction has long been present in the literature on research utilization. Weiss showed that policy research can inform decisions directly, reshape conceptual understanding over time, or perform other functions that a linear research-to-decision model does not adequately describe [3].

The same ambiguity arises when policy apparently does not change. A proposal may have been blocked, a previously expected amendment may have been abandoned, or an organization may have adjusted its behavior in anticipation of a rule that never entered into force. Conversely, the absence of a proposal may simply mean no one was considering one. Dahl's relational account of power, Bachrach and Baratz's analysis of nondecisions, and Lukes's examination of preference formation and latent conflict establish why a study limited to recorded votes captures only part of the policy domain [4][5][6]. Their arguments also make careful empirical specification indispensable. Crenson's comparative study of air-pollution politics in Gary and East Chicago examined contrasting municipal responses and used differences in local political and economic structure to investigate nondecisionmaking [7]. It offers a historical empirical approach to agenda exclusion, not a universally identified causal effect. An unobserved decision cannot be assigned to a hidden actor merely because nondecision is theoretically possible.

1.2 Four distinctions that organize the research question

A useful account begins by separating four questions that frequently appear inside one sentence. What happened? concerns events, operative rules, and changes in the relevant population. What could have made a difference? concerns actors' resources, authority, access, and the alternatives realistically available at the time. What is observable and identifiable? concerns the construction of records, exposure to data, and what can be inferred under explicit assumptions. What is worth doing? concerns the expected and observed effects of feasible actions on different groups, as judged under lawfully selected public-value criteria. The questions are connected in a particular policy episode, but answering one does not settle the others.

Two further empirical puzzles make the distinction important. First, suppose a government committee uses a technical report to frame three viable options yet subsequently preserves the existing rule. There may be a verifiable change in the options considered or the grounds for rejecting a reform without a change in the enacted policy. Weiss's account of conceptual research utilization provides an established theoretical explanation for that possibility [3]. A valid empirical study would still need minutes, working drafts, or other sufficiently diagnostic evidence to determine whether use actually occurred. Second, research ideas may circulate through a professional network until their original source is no longer named. A citation database may then register low visibility at precisely the time when an idea has become ordinary policy language. This is a testable source-tracing problem, not proof that any unnamed study supplied the decisive idea. Research on agenda exclusion and latent conflict likewise shows why unchanged legal text is compatible with several causal histories [5][6].

The four puzzles are therefore analytically different: a decision can change before its intended outcome changes; a recorded indicator can move without an equivalent real change; a policy can stay unchanged while deliberation or an actor's feasible choice set changes; and knowledge can spread while individual-source attribution becomes difficult. A satisfactory explanation should say what additional evidence would discriminate between these accounts. For a verified amendment, the legal text answers the constitutive question. For a changed indicator, one must study the observation mechanism. For a formally maintained policy, the review decision and feasible alternatives must be documented; for a prevention claim, an otherwise credible alternative path and the alleged blocking mechanism are indispensable. For diffuse conceptual uptake, provenance and reception evidence matter at least as much as citation frequency.

Policy influence also runs in several directions. Knowledge and organized interests can influence policy formation. An enacted policy may influence implementation, actor incentives, and social outcomes. A performance indicator can both represent activity and change it through rewards or scrutiny. Earlier policies can reshape the resources and institutions that later make new choices. Established theories explain important portions of this system; the present contribution is to join their mechanisms to explicitly typed influence claims and defensible observation and identification rules.

The central question is: How can policy influence be explained, observed, and identified under heterogeneous generating mechanisms and incomplete, dependent evidence, and under what conditions can the resulting measurements improve feasible research and policy decisions? The integration is meant to yield more accurate questions, stronger permissible inferences, and more useful action under uncertainty. Its distinct contribution lies particularly in the transitions from generating mechanism to observable trace, from trace to conditional identification, and from identified result to task-specific decision sufficiency.

2. Research Design: Theory Integration, Counterexamples, and Primary Evidence

2.1 The form of inquiry

This is a conceptual and theoretical investigation disciplined by primary empirical research, official policy records, and explicitly constructed counterexamples. Jaakkola describes several designs through which a conceptual article can make a scholarly contribution, including synthesis, adaptation, typology, and model development, provided that source selection and the inferential steps are transparent [8]. Here the unit of synthesis is a claim about policy influence, rather than a universal policy-process model. Theories of power and policy change supply candidate generating mechanisms; the literature on knowledge use identifies modes of reception; measurement theory specifies the interpretation of records; causal inference restricts warranted counterfactual claims; and decision theory determines what information is adequate for an authorized choice.

The selection rule is problem-driven. A theoretical source is brought in when it clarifies a distinct relation, generating mechanism, evidentiary condition, or decision rule. A primary empirical investigation enters when its actual design and measured endpoint can discriminate among propositions made here. An official legal record establishes its legal content and effective date; it is not asked to identify social effects without additional evidence. A logical counterexample is used to reject an inference that would be invalid even under favorable observation, without pretending to measure how often the counterexample occurs in practice. These four forms of support should never be aggregated as interchangeable items in a single evidence count.

To make theoretical selection auditable, each material claim can be subjected to the same sequence. Identify the exact proposition. Recover the earliest or otherwise authoritative original source relevant to it. Record the original author's definition and scope. State what is inherited, what is inferred by combining sources, and what remains a hypothesis proposed by this paper. Finally, search for a counterexample, alternative mechanism, measurement change, or different denominator that could defeat the inference. This procedure allows established work to be integrated without implying that an original author tested the combined theory.

Provenance of the main claims. The argument combines established literatures without treating the combined propositions as findings already established by those literatures. The following allocation identifies the origin and burden of proof for the principal components; "derived here" means a consequence of joining stated premises, whereas "proposed for testing" requires evidence beyond a conceptual derivation.

The measurement-validity literature provides a further discipline. Adcock and Collier distinguish background concepts, systematized concepts, operational indicators, and scores; a mismatch between these levels can invalidate an apparently precise comparison [13]. Kane's argument-based account treats validity as a property of justified interpretations and uses, with the relevant inferences open to criticism [14]. Here, a verified count of citations may be valid as a measure of citations while remaining inadequate as a measure of counterfactual policy impact. Equally, a well-estimated treatment effect may answer a question about implementation without establishing the causal share of the researchers who earlier advocated the policy. The validity judgment follows the exact claim and proposed use.

2.2 Three sampling frames and the selection problem

Evidence about policy influence commonly begins with a highly visible case: a report named in a ministerial decision, a regulation followed by a favorable trend, or a research project credited by an official. Such cases are valuable for tracing candidate mechanisms, yet their selection is partly conditional on a detectable trace. It would be a selection error to generalize from them to the frequency of research use among all publications, or to estimate a false-positive rate for influence claims without sampling the claims that were never publicized.

Three sampling frames must be kept separate. A policy-event frame starts from a defined set of decision episodes, proposed changes, maintained rules, implementation events, and, when feasible, observable nondecisions within a jurisdiction and time window. It can ask which sources appear in the record of a policy process. A source-to-document frame starts from identified research products or evidence objects and follows their documented appearance, reuse, or absence in policy material. It can ask how frequently an eligible research source generates traces under specified archive coverage. An influence-claim frame starts from explicitly asserted contributions and can examine the support, scope, and potential exaggeration of such claims. These populations have different denominators. The result of an audit of promotional influence claims cannot be reported as the share of all policy events causally influenced by research.

The difficult-case portfolio has seven diagnostic categories, without treating them as seven independently established causal episodes. Failure to adopt a proposed intervention is observed in the U.S. municipal research-adoption trials: 53 of the 73 tested interventions were not subsequently adopted [19]. Those observations identify nonadoption of the particular interventions, but do not establish whether the cities maintained all their existing policies or why each agency declined the option. Formal policy maintenance following review is illustrated by the U.S. Environmental Protection Agency's 2020 decision to retain the primary annual PM2.5 air-quality standard at 12.0 micrograms per cubic meter. In 2024, after reconsideration, the agency revised that annual standard to 9.0 micrograms per cubic meter [20]. The two decisions establish documented retention followed by revision, not an unobserved veto or the causal role of any individual research source. Research awareness without demonstrated policy influence is a related but distinct category. The withdrawal of published evidence is illustrated by the original soft-drinks article's retraction and its corrected analysis [21][22]. Outcomes contrary to a simple success narrative appear in the Kenyan scale-up study [23] and the HOPE evaluation [24]. Uncited substantive use requires evidence of actual engagement beyond the absence of a reference: interviews and document analysis in a study of 131 Australian health-policy documents address use processes without independently identifying the incremental contribution of a particular uncited source [25]. Multiple reports sharing an upstream source are represented by the logical counterexample in Section 2.3; the example is not offered as a verified historical document genealogy. These categories guide case selection and adversarial reasoning, not estimates of prevalence.

Research-tracing studies supply a useful independent observation on the gap between awareness and attributable influence. In a review of English-language, peer-reviewed studies published between 1995 and 2016 on the policy impact and use of health research, Newson and colleagues identified 144 empirical investigations and 19 reviews, distinguishing research-first forward tracing from policy-first backward tracing [26]. In a separate study of childhood-obesity research and policy in New South Wales, Australia, Newson and colleagues traced 148 selected local research projects. Twenty-three projects (approximately 16%) were reported as having policy impact, whereas for 28 projects (approximately 19%) decision-makers were aware of the research but the investigators found no evidence that it had influenced policy decisions [27]. The latter result describes awareness without demonstrated influence. It does not show that the policy itself stayed unchanged or that the research had no conceivable unrecorded role. Because these were selected local projects, neither percentage is a general rate for research uptake across policy systems.

The manuscript uses published cases analytically, not as a new systematic sample or an instrument-validation experiment. Its examples have distinct explanatory jobs: changes in legal targets and administrative coding, a formally documented decision to maintain an air-quality standard, observed research engagement that reaches beyond a reference list, experiments on research delivery and adoption, an appraisal recommendation that changes when clinical and economic evidence is reassessed, and policy feedback or valuation. Some categories are demonstrated by original observations; others remain theoretical counterexamples because the necessary historical record is unavailable. The studies therefore support particular mechanism and measurement claims without estimating the prevalence of the mechanisms across governments. Any future test of comparative accuracy, inter-rater reliability, or cost improvement requires a separately designed empirical study.

2.3 Adversarial tests and independently checkable counterexamples

Consider a stylized law requiring three simultaneous approvals. If any one approval is withheld, the law fails. An analyst who attributes 100% of the legislation's passage to each approving actor has created a set of contributions that cannot be interpreted as additive shares. Alternatively, suppose two independent actors were each sufficient to secure adoption. Removing one from a simple counterfactual may leave the outcome unchanged, although both may have performed an observable role. The examples identify why actual causal participation, counterfactual effects, and credit allocation require separate definitions. They are logical constructions, not empirical observations about an unnamed country.

Another counterexample involves a policy citation. Suppose a ministry document and two subsequent news reports repeat a sentence from an earlier committee briefing. Three documents are observed, but their relevant information may descend from one record. Counting the documents as independent evidence for a causal claim would artificially increase confidence. These examples guide the formal specification developed in Sections 4 and 5. The common-source example is deliberately hypothetical: without provenance records, repeated language alone would not prove that several historical documents descended from one ancestor.

3. Policy Influence Ontology: Relations, Targets, and Policy Anchors

3.1 The policy anchor and the minimal influence claim

A broad concept is needed to compare different policy-related processes without assuming that they are the same kind of event. We call this domain policy-related relations. Within it, realized causal policy influence denotes a narrower class: a source made a causal difference, in a specified sense, to a defined policy process, state, behavior, or outcome. Each claim is anchored to a PolicyAnchor, a concrete governance issue together with an identifiable policy stage, relevant institutions or decision authority, a time window, and a plausible connection between the proposed source and target. An anchor can be a legislative decision, an administrative rule, a public expenditure program, a court-governed obligation, or the process by which a public body decides among available measures.

The source may be a research paper, an organization, a fiscal constraint, a legal rule, a technological change, a social shock, an earlier policy, or a measurement system. It need not be an intentional person. A change in oil prices may alter a government's available fiscal options without being an actor that desired a specific policy. The policy anchor prevents every remote cause of social change from becoming an unbounded claim about policy influence: the pathway must reach a real, specified governance node and remain subject to temporal and evidentiary scrutiny.

Formally, a minimal claim is represented as

\[ C=\langle id,\; a,\; s,\; r,\; t,\; u,\; w,\; e,\; E,\; A\rangle , \]

where \(id\) is the identifier of one independently assessable claim, \(a\) the policy anchor, \(s\) the alleged source, \(r\) the asserted relation, \(t\) the target type, \(u\) the policy unit or population, \(w\) the observation or causal time window, \(e\) the relevant estimand or factual question, \(E\) the available evidence, and \(A\) the explicit assumptions used in assessment. An estimand is the precise quantity or proposition an analysis seeks to learn, such as the average change in policy adoption caused by sending an evidence briefing rather than no briefing. For purely documentary or legal claims, a causal comparator may be inapplicable; the claim remains assessable without inventing one.

3.2 Five fundamental relations and seven policy targets

We distinguish five basic relations. CAUSES asserts an actual causal connection between source and target, subject to a specified model or contrast. CONSTITUTES concerns what a valid provision, office, or institutional arrangement legally or conceptually makes the case. USES records the substantive application, consideration, or uptake of information, with the particular intensity of use stated separately. REPRESENTS denotes the relation of a document, statistic, or indicator to the object it purports to depict. CAN_INFLUENCE denotes capacity to affect an outcome under some specified range of circumstances, whether or not that capacity was exercised in the historical episode. Associations and normative assessments can be recorded alongside these relations without being silently recast as causal influence.

This separation is especially important for law and statistics. The sentence Section 1 of the amended Act requires a 100% reduction relative to the baseline describes the content of a legal obligation and is primarily constitutive [1]. A subsequent sentence claiming that this amendment reduced emissions among regulated firms is causal and needs evidence of behavioral or environmental change. A statistical publication may represent annual emissions; the rules by which firms report emissions can also causally affect the published figure. Representation, causation, and legal constitution occupy different positions in an explanatory account. The philosophical distinction between constitutive organization and causal interaction has been developed in work on mechanistic explanation; Craver and Bechtel provide a useful conceptual treatment, although the legal validity of a specific rule must still be established from legal sources [28].

The framework assigns a single primary target to each claim, while allowing several linked claims to describe one episode. The seven targets are policy process, policy decision, normative state, policy implementation, actor state, world outcome, and recorded state. They refer respectively to agenda or deliberation; an authorized choice; the duties or rights in force; administrative execution; the behavior, resources, or capacity of relevant actors; economic, social, health, or environmental results; and documents or measurements. The classification is an analytical proposal intended to preserve incompatible truth conditions, not an assertion that all policy systems have already been exhaustively coded into seven classes.

Table 1. Primary targets for policy-influence claims.

An article asserting that a research report was cited, persuaded a minister, changed a regulation, and reduced mortality contains at least four distinct truth questions. Separate claim identifiers can share the same policy anchor and source, but each receives its own evidence and assessment. The resulting record may confirm the citation, support the content's substantive use, qualify its contribution to the regulation, and remain unable to identify an effect on mortality. This is a coherent result. All-or-nothing judgment would discard valuable information.

3.3 Capacity, realized causal participation, recorded visibility, and value

An actor may have institutional authority to block a reform and yet never use that authority. A report may be heavily cited while adding no discernible new information to an already decided policy. A regulation may reduce pollution and impose important costs on a smaller group. Capacity, process participation, observation, and value therefore call for distinct arguments. We also distinguish the net effect of a source under a defined counterfactual from its participation in an observed process. A policymaker can consult a study and still choose the same policy that would have been adopted without consultation.

An absence of documented change deserves particular care. A blocked proposal and an uneventful year may produce the same final statute, while their historical causal structures differ. Bachrach and Baratz's account of nondecision and Lukes's broader conception of power provide reasons to investigate exclusion from agendas or the shaping of preferences [5][6]. The evidence must nevertheless identify the relevant actors, available alternatives, contemporaneous constraints, and events that could distinguish the explanations. Where those conditions cannot be recovered, the specific prevention claim remains unidentified even if institutional power is demonstrable.

Knowledge also has several separable attributes: its content, the identity of the organization supplying it, its delivery channel, the recipient's institutional position, and the shared stock of ideas from which it arose. The same statistical finding may be transmitted through a peer-reviewed article, a government technical note, or a research organization's briefing. The identity attached to the finding is a potential treatment variable, just as the briefing's actual content is. A subsequent change in perceptions or policy must be assessed against the exact intervention and outcome being measured. These distinctions prepare the empirical analysis of research delivery in Section 6.

4. Causal Structures, Generating Mechanisms, and the Limits of Attribution

4.1 Participation in a pathway, actual causation, and net effects

An influence account becomes substantially more informative once its causal question is specified. Consider a research briefing, a ministry decision, and a later fall in emissions. Three candidate conclusions are often compressed into one narrative: the briefing entered the decision process, the briefing was a cause of the ministry's choice, and the briefing reduced emissions by a particular amount. They concern different outcomes and need different forms of evidence. Process tracing can establish whether the briefing was delivered, discussed, and incorporated into successive policy drafts [15]. A claim of actual causation requires a defensible causal model under which that source played a specified role in bringing about the decision [16][29]. Estimating the briefing's net effect on emissions requires additional comparisons and assumptions about the decision, implementation, and the environmental system. The passage from one claim to the next cannot be supplied by the word influence alone.

For a population of decision units, let \(S=1\) indicate receipt of a specified research-delivery intervention and \(S=0\) its absence. Let \(Y_i(1)\) and \(Y_i(0)\) denote the respective potential outcomes of unit \(i\), for example whether municipality \(i\) adopts a tax-reminder practice under those two exposure conditions. A population average treatment effect is \(ATE=\mathbb{E}[Y_i(1)-Y_i(0)]\), where \(\mathbb E\) denotes an average over the target population. A well-executed experiment may identify this average for its population, treatment and horizon. It does not automatically identify how much one particular report contributed to a subsequent nationwide legislative episode, because the population effect and the historical case have different units, comparisons, and causal structures. Tian and Pearl's work on probabilities of necessary and sufficient causation formalizes the distinction: individual causal questions can sometimes be bounded or identified, but those results rest on the data and structural assumptions of the particular model [30].

In a single policy case, a path contribution is evidence that a source participated in an operating chain of events. An actual-cause claim asserts that a source satisfies an explicit criterion of causation in a specified structural model and context. A net-effect claim compares a defined outcome under the observed source with the outcome under an alternative exposure or intervention, holding the counterfactual framework fixed. The most useful target may also be a policy's timing, content, agenda status, or administrative feasibility rather than only its final binary adoption. Hall's distinction between counterfactual dependence and productive causation helps explain why these questions can come apart [29]. Halpern and Pearl offer formal actual-cause criteria within structural models, including cases of contingency, preemption, and overdetermination [16]. Neither framework eliminates the need to justify which structural model is appropriate to the political episode.

4.2 Eight causal configurations and their observable implications

Figure 1 summarizes the eight configurations as schematic causal structures. In panels (b) and (c), the conjunctive and disjunctive equations distinguish joint necessity from redundant sufficiency; the remaining panels emphasize the temporal, counterfactual, or source-dependence conditions relevant to identification.

Single-source transmission. A ministry commissions an analysis, receives its recommendation, and changes a draft rule in a way that is temporally and substantively aligned with the report. This configuration predicts a sequence of contact, consideration, draft revision, and final decision. A dated report and a uniquely matching parameter can strengthen the proposed mechanism, particularly where independent working documents corroborate the timing. The principal rivals are a pre-existing ministerial instruction, a common external source used by both parties, or a parameter independently chosen for administrative reasons. Source-specific effect size still depends on whether the ministry would otherwise have made the same change. Process evidence and a credible policy comparator contribute different parts of this argument [15][31].

Joint necessity. A decision may require political authorization, available financing, implementing capacity, and a technically acceptable design. Under a model in which all four are necessary, the absence of any one prevents implementation. The observation that a research design was approved tells us little about its standalone effect unless the financing and authority conditions were also present. Mechanism evidence should therefore establish when each condition became available and whether its removal would have blocked the target. This structure matters for evaluating organizational and expert-network participation: multiple contributors can be causally necessary in the same episode without there being an empirically observed additive partition of total social value.

Redundant sufficiency. Imagine two research teams that independently produce the same implementable recommendation, either of which would have led an authorized agency to adopt it. If both arrive before the final decision, removing one may leave the final policy unchanged. A zero removal effect on final adoption can coexist with documented use of that team's analysis. To distinguish redundant sufficiency from mere citation, an analyst needs a timeline, evidence of actual reception, and a credible account of what the remaining source could have supplied. Attribution is sensitive to the defined outcome: one source may have advanced adoption by six months even if the final two-year adoption indicator is identical under both alternatives.

Preemption and priority in time. A source may activate a decision process before an alternative source would have done so. Here the observed pathway can be real even when an eventual outcome under a later horizon is unchanged. A simple binary indicator of whether the policy exists in year five can erase a genuine effect on the year-one decision. The distinguishing evidence is temporal: when each source became available, when deliberation began, when critical parameters entered drafts, and whether the alternative source would plausibly have arrived or been accepted without the first. The structural distinction is well illustrated by philosophical and computational work on actual causes [16][29], although no generic thought experiment can substitute for policy records.

Prevention and nondecision. A regulatory proposal can be withdrawn after adverse evidence, an authorized body can choose not to introduce a rule, or an actor may deter a proposal before it is formally filed. Each can have meaningful consequences. But a final record showing only that a regulation was absent cannot distinguish successful prevention from a situation in which no proposal would ever have existed. A supported prevention claim requires some trace of the otherwise viable alternative, an actor capable of affecting it, a temporally credible intervention, and observations that discriminate between prevention and ordinary inaction. The classic work on nondecision and latent conflict clarifies why this is part of the policy domain [5][6]; it does not turn silence in an archive into evidence of a hidden successful intervention. A formal decision to retain an existing rule is a different observable event: the U.S. EPA retained its primary annual PM2.5 standard in 2020 and revised it in 2024 [20]. The recorded decisions establish review and change across two windows, without establishing that a specific source caused either decision. Crenson's historical comparison of municipal air-pollution politics illustrates the use of cross-city variation to investigate possible agenda exclusion [7], although such an interpretation must still confront rival institutional explanations.

Anticipation before implementation. Actors may change behavior after a credible announcement but before the legal start date. The appropriate time origin may therefore be the announcement, consultation, or market-expectation event. The United Kingdom announced the Soft Drinks Industry Levy in March 2016 and implemented it in April 2018. Rogers and colleagues' corrected controlled interrupted time-series analysis estimates that, one year after implementation, household purchases of sugar from soft drinks were 8.0 grams per week lower than the modeled counterfactual, approximately 2.7%, while soft-drink volume was approximately 2.6% higher [21]. These are household purchasing outcomes, and interpretation depends on the study's counterfactual specification. The earlier article was formally retracted after a data-preparation error, making source version an integral part of the evidentiary record [22]. A design that starts exposure only at implementation may misclassify some anticipatory responses as baseline trends.

Dynamic feedback. An adopted policy can alter the political and administrative conditions under which its successor is debated. Subsidies can build constituencies, rules can change the costs of future changes, and performance measures can shape which problems receive attention. At the second period, the same variable may be both an effect of earlier policy and a predictor of later treatment. The observable implication is an ordered history of policies, implementation, changes in capacities or information, and later decisions. The main identification risk is treating a post-treatment variable as though it were a fixed background confounder. Section 7 examines the feedback problem in detail.

Diffusion and co-production. Several actors may develop, revise, circulate, and institutionalize substantially overlapping knowledge. A new policy parameter can be the product of a professional community, interagency drafting, multiple experiments, and a political decision that integrates the results. Citations, textual similarity, version histories, interviews, and joint working records may reveal parts of this network. Yet a common ancestor can make several apparently different sources informationally dependent. When the origin of an idea is diffuse, evidence about its propagation may be strong while the incremental causal effect of one contributor remains bounded or unidentified. Haas's account of epistemic communities and Sabatier's treatment of policy-oriented learning provide substantive mechanisms for this configuration [10][11].

These configurations have distinct diagnostic signatures. Joint necessity calls attention to missing complementary conditions; redundancy and preemption make the time horizon and counterfactual especially consequential; prevention requires evidence of a credible forgone alternative; anticipation requires event timing before formal commencement; feedback requires sequential histories; and diffusion calls for the genealogy of ideas and sources. An evidence protocol that collects identical indicators for all configurations risks measuring the ease of documentation rather than the phenomenon under study.

4.3 Information, resources, rules, and the structure of reception

The mechanisms above can be organized into three interacting families. Information and cognition concern what actors learn, believe, perceive as a problem, or regard as technically feasible. Resources and organizational capacity concern budgets, infrastructure, staff, implementation channels, and opportunities to act. Rules and institutional constraints concern legal powers, procedural mandates, incentives, and enforceable limits on the choice set. This is a working classification intended to help select evidence, not a claim that policy processes possess exactly three natural causal categories.

Kingdon's multiple-streams approach provides mechanisms concerning problem recognition, available proposals, political circumstances, and windows for coupling them [9]. Haas's epistemic-community account concerns expert networks that coordinate interpretations under uncertainty [10]. Sabatier's advocacy-coalition framework concerns belief systems and learning over longer policy cycles [11]. Jones and colleagues' review of 311 multiple-streams applications documents variation in how central concepts have been operationalized [32]. These theories illuminate different units of analysis. To test a particular claim of research influence, one must still establish what information entered which decision, under what authority, with what alternatives and potential confounders.

The receiving structure is the configuration of decision authority, feasible options, institutional routines, legitimacy, and implementation capacity at the moment a source could have mattered. A study recommending a reform may be scientifically sound yet arrive at a unit lacking power to act. Another study may be modestly informative but appear while officials are revising precisely the relevant standard and have an established implementation channel. A source's contribution cannot be predicted from content quality alone. This point links the generation of influence to later experimental evidence on organizational bottlenecks and sender identity [33][19][34]. It also suggests concrete measurements: who had formal discretion, what programs and budgets were already approved, whether the proposed intervention was technically compatible with existing systems, and when that compatibility changed.

Potential influence deserves separate treatment. Shapley and Shubik's voting-power index measures a committee member's expected pivotal position under a specified coalition game [35]. The resulting quantity describes capacity under the modeled voting rules. It does not supply an ex post share of actual causal responsibility for a vote, much less for the policy's long-run welfare effects. A researcher may have the capacity to influence several committees without having affected a particular meeting. Conversely, actual influence may occur through an informational contribution rather than through a formally pivotal vote. Maintaining this distinction helps prevent the unexamined transfer of power indices into historical attribution.

4.4 A time-indexed generative account and its identification obligations

To express the proposed relationships over time, let \(S_t\) denote research or other source exposure at time \(t\); \(P_t\) the policy agenda, authorized decision, or rule selected then; \(N_t\) the legal rights and obligations constituted by the operative rule; \(I_{t+1}\) the subsequent implementation; \(W^*_{t+1}\) the real economic, organizational, or social conditions affected by implementation; and \(Y^*_{t+h}\) a specified underlying outcome at time \(t+h\), where \(h\) is the elapsed observation horizon. The asterisk distinguishes a real state from its recorded proxy. \(O_t\) denotes a record generated under an observation regime, which will be defined formally in Section 5. These symbols identify distinct variables and conceptual stages; they do not imply that all stages occur in a fixed linear sequence.

A candidate model may link source exposure to decision \(S_t\to P_t\), interpret the operative provision as constituting \(N_t\), and link implementation \(I_{t+1}\) to later outcomes \(Y^*_{t+h}\). Later decisions \(P_{t+1}\) may respond to prior outcomes and records, evolving resources, and legal constraints. More complex models permit \(S_t\) and \(P_t\) to share an upstream cause: for example, a government already determined to review a regulation commissions the evidence analysis. A positive association between the commissioned report and the revision then contains possible selection by government demand. This alternative must be considered before concluding that delivery independently changed the decision.

The model generates concrete identification obligations. For a content-transfer claim, recover the exact source version and the draft's modification history. For a belief-change claim, measure the recipient's relevant understanding or choices before and after exposure, preferably with a defensible comparator. For an implementation claim, verify operational delivery rather than assuming enactment equals execution. For a net-effect claim, specify the intervention, counterfactual and horizon; evaluate common causes, substitution and spillovers. For a prevention claim, recover evidence of a credible alternative path. For a feedback claim, document a sequence in which an outcome or institutional change from one period can influence a subsequent decision. Negative controls, where appropriate, may test whether the proposed mechanism appears in outcomes or periods it could not plausibly have affected.

This sequence is a substantive contribution to policy impact measurement because it translates differences between generating structures into differences between warranted measurements. It also supplies failure conditions. A postdated source cannot explain an earlier decision without a viable anticipation or communication account. A policy attributed to a unique research finding must be reconsidered if an equivalent finding was already authoritative before the alleged contact. A single selected case cannot identify population-wide adoption probabilities. The framework is useful precisely when these possibilities can change what the researcher concludes and what evidence is sought next.

5. Measurement Validity and Partial Identification Under Dependent Evidence

5.1 Influence generation and observation generation

Figure 2 depicts the distinction between substantive policy processes and the production of their observed records. The link between a policy decision and the corresponding normative state represents legal constitution rather than an ordinary causal effect. Feedback to a subsequent policy decision is indexed after the relevant outcome and observation window.

We distinguish the influence-generating process, which governs whether and how policy-related relations occur, from the observation-generating process, which governs whether relevant traces become available to researchers. Let \(Y^*_t\) denote a genuine state at time \(t\), such as whether a specific policy procedure was used or a health outcome occurred. Let \(O_t\) denote the record produced by a measurement function:

\[ O_t=M\big(Y^*_t,\,R_t,\,C_t,\,S_t^{obs},\,K_t\big). \]

Here \(R_t\) is the reporting or measurement rule, \(C_t\) the coding convention, \(S_t^{obs}\) the sample or population coverage, and \(K_t\) the documentary or extraction technology. The function \(M\) describes how a real state becomes observable. It may be deterministic in a legal registry or stochastic in surveys and incomplete archives. Its arguments are defined by the task rather than presumed to be universally known. A change in \(O_t\) can follow a change in \(Y^*_t\), a change in the observation regime, or both.

The HRRP example provides an empirical demonstration: changed electronic claims standards altered the number of diagnosis codes that could be reported, with material consequences for risk-adjusted readmission trends [2]. This does not imply that all administrative data are unreliable. It shows why the measurement rule has to be part of the empirical account. Independent sources may help, provided their observation mechanisms are sufficiently understood. Official statistics, interviews, citation databases, and policy websites often share upstream information and institutional incentives. The number of visible records is therefore a poor proxy for the number of independent observations unless their relationships are inspected.

Three kinds of validity follow. Construct validity asks whether an indicator represents the policy relationship named in the claim. Causal identification validity asks whether the design and assumptions justify a conclusion about an estimand. Transportability or predictive validity asks whether the interpretation continues to function when the policy domain, jurisdiction, archive, or future event changes. Adcock and Collier's measurement framework and Kane's argument-based validation approach supply the foundations for this distinction [13][14]. A technically accurate citation extraction can have high validity for documentary traces and still be incapable of estimating the causal contribution of a source to a policy decision.

5.2 A six-dimensional assessment of a policy-influence claim

An assessment is interpretable only when its fields correspond to distinct things that could be true or false. We organize the analysis along six dimensions. The relation and scope dimension (R) asks which relation is claimed, to which policy anchor and target, and whether the proposed object is sufficiently defined. The observation dimension (O) asks whether the alleged source and records are authentic, what they represent, how complete their coverage is, and whether the information has been superseded. The use dimension (U) concerns substantive reception of knowledge: consideration, incorporation into drafting, or application, as distinct from a visible citation alone. The causal-identification dimension (C) asks what exact estimand or causal proposition is identified, conditional on which alternatives and assumptions. The value dimension (V) describes the benefits, harms, distributional consequences, rights, and applicable valuation rule. Finally, the action dimension (A) asks what an authorized decision-maker can justifiably do given available options, costs, and uncertainty. The letters are abbreviations for different analytical questions. They are not ordinal scores, and the sequence is not a ladder of increasingly strong influence.

The distinction between scope and evidence is elementary but useful. A sentence saying that a particular think-tank report "changed climate policy" may lack a country, a decision date, a policy provision, and a source version. Its initial problem is incomplete definition on R. Retrieving a press release does not resolve that ambiguity. Conversely, an accurately specified claim can be difficult to verify because the ministry's archives are incomplete; that is an O problem. An identical press release appearing on five sites changes the count of visible records without necessarily improving independent support. A genuine file noting that the report was read may help with U, while a causal effect on enacted text or subsequent emissions remains a C question.

Consider a connected series of claims associated with the United Kingdom's 2019 climate-target amendment. The first claim, which has a legal-document target, states that the operative 2050 emissions-reduction target was amended from 80% to 100% relative to 1990. Its original legislative text provides direct evidence of the changed normative state [1]. The second claim concerns whether a named scientific assessment was substantively used in the policy process. That claim would require independently verified documentary or testimonial evidence of reception and its precise content. A hypothetical ministerial briefing might establish consideration, but the legislative change by itself would not. The third claim concerns whether that assessment caused a particular target choice. It requires a defined comparison, evidence about alternative sources and previously determined commitments, and, where appropriate, a trace of which parameter or argument influenced the decision. The fourth claim concerns the amendment's effect on actual emissions by a specified date. Its data must measure emissions and identify a plausible counterfactual for the enacted rule, not simply document the rule's existence. The fifth claim concerns public value, for which even a well-estimated environmental effect must be evaluated alongside economic costs, distribution, rights, and feasible alternatives. These are five separately assessable claims, connected by a shared policy episode. Only the first is established here by the cited statutory source; the hypothetical briefing illustrates the evidence required for the other claims and does not allege an additional historical fact.

The same separation can be written as a minimal linked record. A use claim has its own ClaimID, an identifier for one independently assessable truth question, and may have OutputKind=USE and UseStatus=USE_SUPPORTED when the evidence warrants substantive use. A distinct net-effect claim has a different ClaimID, linked to the first, with OutputKind=NET_EFFECT and IdentificationStatus=UNIDENTIFIED if the specified contrast has been evaluated and the available evidence does not identify it. The output kind names what is being evaluated; identification status says what has been learned about its estimand. A confirmed documentary statement can therefore coexist with an unidentified net causal effect without inconsistency.

Other attributes carry information orthogonal to those six dimensions. PathEvidence distinguishes a supported causal pathway from an unresolved pathway or an inapplicable assessment. ClaimSupport records whether a precisely formulated assertion is supported or contradicted under stated assumptions. ModelValidity records whether the candidate model is compatible with the observations. An incompatible causal model does not automatically falsify every statement made about the same event; one may need to revise the model and reassess the specific claims. The distinctions among NOT_ASSESSED, UNRESOLVED, and NOT_APPLICABLE are also substantive. A field may remain untouched, may have been examined without reaching a justified decision, or may have no relevance to the claim. Confusing those conditions misstates both the research work completed and the evidence still needed.

The three validity questions established in Section 5.1 apply to each linked claim separately. For the 2019 UK climate amendment, the statutory text identifies a legally constituted target; an independently documented ministerial use of a source would answer a different empirical question; and a comparison capable of estimating emissions effects would bear a third burden. Transfer to another jurisdiction would additionally require its own decision authority, archive practices, implementation timing, and outcome definitions. This application of the validity distinction is more informative than treating a citation, a causal estimate, and an institutional decision as interchangeable scores.

A six-dimensional account also exposes where effort should go next. If the policy anchor is ambiguous, better source identification may help before any statistical model is appropriate. If the event is verified but use is disputed, contemporaneous drafts and meeting records may be valuable. If use is strongly supported but the counterfactual remains open, a credible comparison or mechanism-specific process investigation becomes relevant. When the remaining causal uncertainty could not affect any feasible choice, collecting more documents may have little immediate decision value. In that sense the classification is already connected to the research-resource question developed in Section 8.

5.3 Partial identification: deriving only what the evidence permits

Let \(E=(E_1,\ldots,E_k)\) denote the available evidence items, potentially drawn from \(k\) different documents, interviews, datasets, or administrative sources. Let \(A\) denote the assumptions that specify a class of admissible influence-generating and observation-generating mechanisms. Let \(\mathcal{M}(E,A)\) be the set of candidate models compatible with the evidence and those assumptions, and let \(\Omega_m(E,A)\) be the possible causal worlds admitted by candidate model \(m\). Then define

\[ \mathcal W(E,A)=\bigcup_{m\in\mathcal M(E,A)}\Omega_m(E,A), \qquad \Theta_q(E,A)=\{q(\omega):\omega\in\mathcal W(E,A)\}. \]

The symbol \(q\) is the query or estimand of interest, for example the prevalence of substantive policy use or the average effect of a briefing. The symbol \(\omega\) denotes a possible world, meaning a complete assignment of relevant events, unobserved variables, and causal relations consistent with one admissible model. The set \(\Theta_q\), called the identified set, contains every value of \(q\) that remains compatible with the stipulated evidence and assumptions. A singleton identified set implies point identification within that model class; a non-singleton set supplies partial identification. Manski's analysis of missing data establishes the usefulness of learning what can be inferred without imposing assumptions strong enough to produce an artificial point estimate [17].

An identified set is inherently conditional. It is not automatically a statistically calibrated confidence interval, and its ability to contain the actual value depends on whether the admissible models and assumptions adequately cover the true data-generating process. Aleshin-Guendel, Sadinle, and Wakefield show in multiple-systems population estimation that distinct identifying assumptions can yield drastically different estimates despite identical fits to the observed data [36]. Imbens and Manski further distinguish confidence intervals intended to cover the true partially identified parameter from intervals designed to cover the entire identification region [37]. Thus, model compatibility, identification, sampling uncertainty, and real-world adequacy require separate checks.

An elementary misclassification example makes the practical issue concrete. Let \(T=1\) indicate that substantive research use truly occurred in a randomly sampled eligible policy instance, and let \(Z=1\) indicate that a documentary trace of use was detected. Define \(\pi=\Pr(T=1)\) as the true prevalence of substantive use, \(\alpha=\Pr(Z=1\mid T=1)\) as sensitivity or detection probability, and \(\beta=\Pr(Z=1\mid T=0)\) as the false-positive probability. If \(p=\Pr(Z=1)\) is the observed trace rate, the law of total probability gives

\[ p=\alpha\pi+\beta(1-\pi), \qquad \pi=\frac{p-\beta}{\alpha-\beta} \quad(\alpha\ne\beta). \]

Suppose a hypothetical audit finds traces in 20% of eligible records, so \(p=0.20\). If sensitivity and false-positive probability have no informative restrictions, the observed proportion alone does not identify \(\pi\). If an independently justified calibration supplies \(0.50\le\alpha\le0.90\) and \(0\le\beta\le0.10\), then the illustrative identified range is

\[ 0.125\le\pi\le0.40. \]

The lower bound follows from \((0.20-0.10)/(0.90-0.10)=0.125\); the upper bound from \(0.20/0.50=0.40\). These are hypothetical calibration assumptions, not measured detection rates for policy documents. The result applies to prevalence of a defined use event under the example's homogeneous misclassification assumptions. It supplies no automatic estimate of how much a particular research organization caused a subsequent policy change. Figure 3(a) displays the corresponding bounds across the stipulated sensitivity range.

5.4 Evidence dependence and the diagnostic value of new observations

Two different kinds of dependence affect policy-impact evidence. Documentary lineage dependence arises when several records descend from a common upstream source: a government announcement reproduced by news agencies, a briefing copied into committee minutes, or a standardized statistic cited in many reports. Bias dependence arises when formally separate sources share the same selection procedure, incentives to overreport success, administrative coding change, or unobserved confounder. Two interviews conducted by different teams may still reproduce an institutional story told after a politically salient decision. Multiple records are useful, but their count alone is an inadequate measure of the information gained.

The diagnostic consequence can be demonstrated numerically without pretending to estimate a real political event. Suppose a binary hypothesis has prior odds of 1 to 1, and a genuinely informative record has likelihood ratio 4, meaning the record would be four times as likely if the hypothesis were true as if it were false. The resulting posterior odds are 4 to 1, or 80% probability under that assumed model. Now suppose a second publication merely copies the first record. Treating the copy as independent would wrongly multiply the likelihood ratio again, yielding odds of 16 to 1, or about 94.1%. The copy supplies no equivalent second observation. Figure 3(b) contrasts the posterior probabilities obtained by counting one informative record and by incorrectly treating its copy as a second independent record. These numbers are constructed to demonstrate double-counting; the framework does not assign likelihood ratio 4 to policy evidence as a general rule.

Shared bias is harder because files can be distinct while the underlying errors remain correlated. Administrative datasets created by different offices can share an upstream registration system. Several organizations may independently interview the same small set of officials. The appropriate analysis records source ancestry, potential common errors, and the observations that would meaningfully differ across rival mechanisms. Where a joint probability model can be justified, its dependence structure should be represented. Where it cannot, a sensitivity analysis, set of admissible explanations, or transparent conditional process judgment may be the most informative output.

Lanners and colleagues' study of data fusion derives partial-identification bounds and sensitivity analyses when no-unmeasured-confounding and cross-source counterfactual exchangeability assumptions can both fail [38]. These are concrete results for specified experimental and observational data configurations. They supply a methodological lesson about model dependence, but do not furnish a ready-made formula for identifying the causal role of a single research source in a historical legislative process. A single-case analysis may instead rely on temporally ordered and diagnostically distinctive traces, as examined by Collier's process-tracing account [15] and by Befani and Mayne's integration of process tracing with contribution analysis [31]. An internal draft showing adoption of a source-specific parameter before public announcement may be more informative for a particular pathway than ten later general endorsements.

One must also distinguish information gained under a fixed model from changes in what models are accepted. When new evidence further constrains an unchanged admissible model class, the identified set can only stay the same or shrink. If independent scrutiny uncovers an omitted mechanism or invalid assumption, a corrected model class may produce a wider set. Greater honesty about uncertainty can therefore expand the reported bounds while improving the assessment. The desired result is a credible identified conclusion that responds to the decision question, not mechanically narrower intervals.

5.5 Knowledge citations, conceptual use, and diffusion

Knowledge-use measurement provides a particularly rich illustration of the two-process framework. A study may be directly cited, textually reused without attribution, considered during deliberation, transmitted through experts, incorporated into a shared conceptual vocabulary, or applied to a policy parameter. Each is in principle an empirical question. The prevalence of these events depends on the documents sampled and the operational rules for classifying them. Newson and colleagues' forward and backward tracing review shows why source-first assessments and decision-first assessments can illuminate different portions of the research-policy relationship [26].

A first-hand account of a William T. Grant Foundation project reports 6,684 instances of research-evidence use by US federal legislators and staff working on childhood-obesity policy from 2000 to 2014. Of those observed instances, 68% were classified as conceptual use, 13% as instrumental use, and 12% as tactical use; 73% of the 4,545 conceptual-use instances involved attempts to persuade other policymakers [39]. The denominator is observed instances of research-evidence use within that study. Its research-evidence category includes institutional facts and statistics as well as scholarly work, and its proportions are not population rates of use for every academic or think-tank publication. An observed persuasive function describes communicative behavior; it does not establish unobservable private intentions.

Another useful distinction concerns diffusion versus source-specific causation. A phrase or idea can persist across reports and policy documents while losing a unique textual signature of its origin. A citation network can measure this visibility and propagation in informative ways. Christensen and van den Bekerom distinguish four dimensions of policy advice's influence on subsequent advice: lasting influence, broad influence, influential followers, and brokerage across portions of a network. Their original analysis uses a network of Norwegian advisory reports [40]. These dimensions enrich the measurement of knowledge circulation. Whether any one report caused a legislative decision, or contributed a specific percentage of a welfare effect, remains a separate causal question.

Direct empirical research has already sought to move from a visible policy document toward a richer classification of research engagement and use. Williamson, Makkar, and Redman examined 131 policy products developed by six health policy or program agencies in Sydney, interviewing the staff most involved in their production and using the Staff Assessment of enGagement with Evidence (SAGE) instrument to score research engagement and forms of use [25]. The study distinguished searching for, appraising, or generating evidence from conceptual, instrumental, tactical, and imposed use. Its analysis found that access to consultants and relationships with researchers were associated with greater research engagement, without a corresponding demonstrated association with use [25]. Because participants were nominated by agencies participating in a research-use intervention, and the analysis relied on retrospective self-report, the estimates cannot be treated as a census of Australian policymaking or as independent proof of each research source's causal contribution. The design nevertheless shows how a policy-product frame and specific process interviews can ask questions a reference-count frame cannot.

A complementary large-scale approach was published by Springer Nature and Overton in 2025. Their report examined approximately 12 million policy documents dated 2015–2025 and classified over 5.5 million as linked to at least one Sustainable Development Goal. The report identified 637,000 SDG-classified documents citing scholarly publications, versus 232,000 in the non-SDG group; document types and citation practices differ across the two groups [41]. The measurable object is documentary citation and its distribution across indexed publishers and sources. Such datasets permit large-scale analysis of dissemination and source visibility. They do not independently code every document's substantive knowledge-use process, identify the incremental effect of a particular report on an enacted rule, or establish the net public benefit of that rule. Combining this coverage with policy-product interviews and causal designs is a constructive research strategy precisely because each contributes a different observation channel.

Newson and colleagues conducted a more focused audit of 86 childhood-obesity policy documents produced in New South Wales, Australia, between 2000 and 2015. They reported that 28% were not publicly available and that 41% of documents containing references used unconventional referencing formats, complicating both document retrieval and citation extraction [42]. These proportions have different denominators: all 86 eligible policy documents for the accessibility finding, and the subset containing references for the format finding. The study does not establish the frequency of hidden substantive research use, but demonstrates that an absence of retrievable conventional citations is partly a product of the observation process.

These document-based studies, the policy-product interviews, and the network analysis by Christensen and van den Bekerom [40] clarify the division of empirical labor. Citation presence, text reuse, substantive use, contribution to a decision, and consequences of the decision have different denominators, causal models, and evidentiary requirements. A positive trace should be reported confidently when verified. Its implications for stronger claims must be evaluated according to the relation and target being asserted.

5.6 Mechanism-sensitive propositions and conditions for refutation

The generating-and-observation distinction yields empirical propositions with explicit comparisons. P1, mechanism-dependent visibility: among comparable policy episodes with an adequately defined underlying event, the probability and type of surviving documentary trace will vary systematically with the mechanism through which the source operates and with institutional recording requirements. A direct citation mechanism predicts identifiable textual traces; tacit professional learning may leave records principally in interviews, repeated vocabulary, or revisions to internal option sets. The target is a conditional distribution of observations given a substantive event, rather than the unconditional number of indexed documents. A meaningful test needs a policy-event sampling frame that can discover at least some events independently of the trace under evaluation; otherwise detection bias is built into the sample. Repeated results showing no mechanism-associated variation under adequately measured archival conditions would weaken this proposition. The Newson policy-document audit [42] identifies specific obstacles arising from public access and citation format, but cannot, by itself, establish the prevalence of unrecorded substantive use.

P2, dependence-sensitive identification: when apparently separate records share an upstream source or a consequential measurement error, a correctly specified joint observation model can yield different evidentiary weight or identified sets from an analysis that multiplies those records as independent. The first part is a conditional implication of the specified dependence model. The empirical prediction concerns diagnostic improvement in a prospectively defined collection of eligible influence claims. The units are claims with ascertainable record lineage or otherwise testable shared errors; the comparisons are (a) an independence-based analysis and (b) a dependence-aware analysis using the same original information, judged against subsequently obtained independent records, calibrated error rates, or a defensible adjudicated reference outcome. Pre-specified outcomes may include false-positive causal assertions, interval coverage where a sampling design supports it, and the accuracy of decisions about which further evidence matters. Under matched information and validation conditions, repeated failure of the dependence-aware analysis to improve those outcomes, or consistent deterioration because its assumed dependence structure is wrong, would weaken the empirical prediction. A different numerical answer caused merely by imposing a correlation assumption does not itself count as improvement. Where source independence has been verified, a null change is expected and provides a negative-control comparison rather than a counterexample to the conditional model statement.

P3, diffusion without unique attribution: observed spread of a research idea across policy documents will often coexist with an unidentified or assumption-sensitive causal increment for a particular originating source. The relevant units are linked source histories and policy events; the comparison is between measures of diffusion and separately established causal contrasts. The proposition does not imply that attribution is always impossible. An experiment on source exposure, a uniquely timed change in decision authority, or other compelling diagnostic evidence could support a source-specific effect. Its proposed boundary would be weakened if, across carefully sampled cases, diffusion metrics alone repeatedly identified the same source-specific effects as credible independent counterfactual designs under stable conditions.

A fourth proposition about decision sufficiency under partial knowledge follows in Section 8. Unlike the first three, its basic logical possibility is established by decision theory: a decision may be robust even when its payoff has only an identified interval. Empirical studies would be needed to establish how frequently this occurs, or whether a specified influence-assessment procedure improves decisions in practice. Keeping logical implications separate from empirical frequencies helps prevent conceptual success from being mistaken for a measured performance advantage.

These propositions imply a minimal measurement contract. Every consequential claim needs a definable policy anchor, a relation and primary target, a source and version, an observation mechanism, the strongest relevant competing explanations, and an evidence assessment tied to its factual or causal question. A numerical estimate also requires its estimand, comparator, assumptions, uncertainty, and horizon; a public-value or action claim requires the corresponding normative rule and legitimate decision authority. Complex linked records are created only when the question warrants them. This discipline makes future empirical studies possible without forcing every simple documentary check into an elaborate causal model.

6. How Research Knowledge Influences Policy Choice, Implementation, and Social Outcomes

6.1 Knowledge has multiple possible destinations

Research enters policy through several channels. It can update a decision-maker's beliefs, supply a parameter for a regulation, legitimate a proposition in public deliberation, change the feasible design of a program, or inform the implementation of a policy whose broad objective was already settled. It can also be ignored, superseded, or used to demonstrate why an anticipated intervention should not proceed. Weiss's account of multiple meanings of utilization remains a foundation for this analysis [3]. Kirkhart extended the conceptual scope of evaluation influence across sources, intention, and time, including consequences that emerge beyond the moment at which a report is produced [43]. Mark and Henry elaborated individual, interpersonal, and collective processes through which evaluation can affect attitudes and behavior [44]. These accounts provide mechanisms to investigate rather than a license to equate every document trace with policy effectiveness.

A useful research-to-policy account therefore separates the research object from its delivery. Let \(B\) denote a body of research or a particular evidence object, such as a study, model, evidence synthesis, or technical dataset. Let \(L\) denote a knowledge-delivery intervention, such as a briefing, meeting, training, or embedded implementation tool. Let \(D\) denote a policy decision, \(I\) its implementation, and \(Y^*\) a real outcome. The pathway \(B\rightarrow L\rightarrow D\rightarrow I\rightarrow Y^*\) is possible, but none of these arrows is universal. A report might reach a decision-maker after a statutory target has already been selected, yet help define the later monitoring system; evidence may improve implementation without changing the formal decision. The source, delivery, and receiving conditions deserve their own observations and causal questions.

Knowledge utilization is also a subject of intervention research. Langer, Tripney, and Gough's synthesis examined strategies for increasing research evidence use in decision-making, incorporating 36 reviews and giving particular analytical attention to a smaller set of sufficiently relevant and credible reviews. Its findings do not establish a general law under which better access to research necessarily increases substantive use. Interventions directed at access, skills, motivation, and organizational processes differ in their observed effects and evidentiary quality [45]. The practical implication is to evaluate a knowledge service against an actual decision or behavioral endpoint, rather than to infer usefulness from dissemination volume.

6.2 Experimental evidence that research delivery can change policy

The Brazilian municipality experiments by Hjort, Moreira, Rao, and Santini provide direct evidence that research information can alter a policy decision under identifiable circumstances. Working with 2,150 Brazilian municipalities across their research program, the authors studied officials' demand for research and responses to evidence. In their policy-adoption experiment, informing mayors about research on taxpayer reminder letters increased the probability that the municipality implemented that policy by approximately 10 percentage points [33]. The 2,150 figure describes the wider municipality research program, and should not be assumed without consulting the trial details to be the number randomized in every individual component. The outcome supporting the stated estimate is implementation of the specified administrative practice. Effects on subsequent tax revenues or general public welfare require their own estimates.

This is strong positive evidence for a defined link from information delivery to policy behavior. It is also informative about boundaries: policymakers' willingness to obtain research, their belief updates, and their adoption of the recommended measure were distinct experimental subjects within the paper. A theory that treats receipt of a study as interchangeable with implementation cannot represent these findings precisely. A practical organization disseminating research should instead ask where the decision node lies, which authorized actor receives information, and whether the intervention is matched to a choice still available at that node.

The U.S. city evidence adds an organizational dimension. DellaVigna, Kim, and Linos followed 73 randomized controlled trials conducted in 30 cities with a national behavioral-insights unit. City departments adopted the specific tested communication intervention in 27% of cases, or 20 out of 73. Fourteen of 21 trials using an existing communication channel were adopted, compared with 6 of 52 trials involving new communications [19]. These fractions correspond to approximately 67% and 12%. They are an observational comparison among trial characteristics, since the studies did not randomize cities to possess one implementation channel rather than another. The authors also distinguish later adoption of the specific tested intervention from broader adoption of experimentation or related practices. The evidence supports the conclusion that organizational compatibility and existing operational channels are important predictors worth investigating; it does not identify a universal 55-percentage-point causal effect of channel type.

A subsequent Spanish field experiment provides stronger evidence on a different delivery attribute: the institutional identity attached to policy information. García-Hombrados and colleagues experimentally communicated information on a cost-effective, non-ideological policy to local politicians through organizations with different perceived ideological positions, including think tanks, newspapers, and a research institution. Their published findings report a 65% relative increase in policy adoption when the informing institution was ideologically aligned with the policymaker [34]. The change is a ratio relative to an experimental comparison, not a rise of 65 percentage points. An earlier randomized survey experiment by Doberstein showed that Canadian provincial policy analysts assigned different perceived credibility to otherwise equivalent policy research summaries depending on whether they were attributed to academics, think tanks, or advocacy organizations [46]. That experiment identified an effect on credibility assessments, while the Spanish study measured a later policy-adoption outcome. The experiment shows that source identity and recipient context can affect a concrete policy-adoption endpoint. It neither implies that ideological alignment guarantees adoption nor justifies changing the substantive content of evidence to secure agreement.

Together, these studies support a precise proposition: research information can change policy choices; the strength, direction, and likelihood of that change depend on the intervention, its source and delivery, and the receiving organization. Their designs identify different treatments. Aggregating their percentages into one generic research impact rate would be mathematically and substantively incoherent.

Table 2. Different interventions and outcomes in primary research on knowledge and policy action.

The studies address different outcomes and sampling frames. Their effect sizes cannot be pooled into a generic research-influence percentage without a new, justified common estimand.

6.3 Evidence adoption does not ensure successful implementation

An adopted policy can fail at the implementation stage despite promising pilot evidence. Bold and colleagues conducted a randomized study in 192 Kenyan public primary schools: 64 schools received an NGO-managed contract-teacher intervention, 64 a government-managed version, and 64 served as controls. The NGO-managed intervention improved test scores by roughly 0.18 standard deviations in the preferred analysis, whereas the effect under government administration was significantly smaller and not statistically distinguishable from zero [23]. The treatment concerned a comparable contract-teacher design, yet institutional implementation conditions differed, including monitoring, payroll delays, and political economy constraints. The experiment identifies effects in that intervention setting; it does not prove that government-operated programs are inherently ineffective.

The HOPE probation example reaches a related conclusion from another policy domain. In a randomized multi-site demonstration involving more than 1,500 probationers across four U.S. sites, Lattimore and colleagues assessed a model emphasizing frequent monitoring and swift sanctions. The multisite study did not find a statistically significant overall reduction in arrests relative to probation as usual, though particular measures or sites showed different results [24]. This finding is not evidence that no implementation occurred. It demonstrates why program fidelity, observed compliance, criminal justice outcomes, and downstream social value must be separately examined. An effective mechanism in one local context does not guarantee the same population effect when implemented elsewhere.

These studies also clarify research attribution. Even if an intervention is shown to improve a target outcome in a randomized trial, a different claim is needed to establish that a particular paper or research institution caused an agency to select that intervention. For the latter claim, useful observations include decision memoranda, draft revisions, treatment of competing evidence, commissioning histories, and the timing of feasible choices. The result of an intervention evaluation and the causal history of the decision to adopt it have distinct counterfactuals.

6.4 Government consideration, legal change, and source-specific causation

A government may formally record that it considered scientific advice. That is a strong observation of a policy-side knowledge process. The UK's 2019 climate target amendment is a clear example of a legally observable change in a normative target [1]. The source-specific counterfactual question, whether the amendment would have been identical without any particular advisory body's contribution, is harder. It requires analysis of alternative political commitments, international arrangements, domestic legislation, scientific advice, and government deliberation. A formal citation or acknowledgment can be decisive evidence of reception while remaining insufficient to establish exclusive causation.

Evidence can appropriately lead to rejection, suspension, or replacement. When a study proves less reliable after a correction, the organization's next action may be to stop relying on a reported effect and reassess options. The UK soft-drinks research illustrates how retaining retraction records and revised estimates is necessary for credible use [22][21]. An influence assessment that counts how frequently an invalidated claim was cited, yet never updates its validity status, measures a real historical footprint but cannot serve as a sound current policy recommendation.

An institutional example makes the distinction between research engagement, recommendation and subsequent authorization concrete. In England, the National Institute for Health and Care Excellence (NICE) issued technology appraisal TA371 on trastuzumab emtansine in December 2015; the earlier guidance did not recommend routine provision within the relevant treatment indication [47]. Reconsideration under TA458 in July 2017 produced a recommendation conditional on the manufacturer's commercial access agreement with NHS England [48]. The later appraisal committee reviewed additional follow-up evidence, clinical comparisons, cost-effectiveness models and revised access arrangements, according to its published discussion [48]. The comparison documents a change in an official recommendation and identifies evidentiary and economic considerations present in the formal appraisal. It does not establish that any single submitted analysis, considered in isolation, was the unique cause of the recommendation change. Equally, the original nonrecommendation is an observed policy decision, not evidence that research was ignored; the subject had been evaluated and the authorized choice at that stage was not to recommend routine use. This case shows why a study must preserve the distinction between substantive evaluation, the decision taken under a decision rule, and any later change in that rule or the evidence admissible under it. The historical TA371 record must remain available alongside TA458: replacing an earlier policy record with only the current guidance would erase part of the evidence needed to reconstruct what was decided and known at each date.

6.5 From knowledge dissemination to an actionable evidence service

The preceding findings suggest a practical, testable service design. Begin by identifying a concrete policy-choice node and the actor with authority to act. Determine which policy parameters remain changeable, what alternatives are legally and operationally feasible, and what evidence bears on those choices. Deliver an appropriate research product through a suitable channel; observe whether it was received, discussed, and used; and then evaluate implementation and external outcomes using designs warranted by the data. Each transition produces a separate observation and can be improved without falsely assuming that downstream policy success follows mechanically.

This sequence should not become a mandatory checklist for every document citation. A verified citation can be reported as a verified citation. A claim that a research briefing changed a municipal tax practice requires more. A claim that the briefing improved public welfare adds still another causal question. By making the tasks explicit, research organizations can use existing evidence more effectively and decide when further analysis is worth its cost.

7. Policy Feedback, Measurement Reactivity, and Institutional Change

7.1 Policy outcomes can become causes of future policy

Policy influence unfolds over time. Earlier policies change resources, organizational capacities, legal obligations, and the information available to later decision-makers. Pierson described two foundational feedback mechanisms: the resources and incentives policies provide to actors, and the information or interpretive cues they furnish about political life [12]. We retain those mechanisms and distinguish the role of institutional rules and administrative observation systems. A feedback claim has temporal content: some consequence of \(P_t\), the policy at time \(t\), contributes to the formation or selection of \(P_{t+1}\), a policy decision in a subsequent period.

This definition excludes merely redescribing the scheduled execution of a standing rule as a new discretionary decision. Automatic cost-of-living adjustments in U.S. Social Security were established under the 1972 amendments and began in 1975. The Social Security Administration's historical series reports no COLA for the benefit adjustments payable in January 2010 and January 2011 because the established index mechanism did not trigger increases [49]. Those outcomes reflect a rule being applied under particular statistical conditions. A new legal amendment to the formula would constitute a separate policy-change event; consequences of automatic payments for political participation or public finances might later form a feedback pathway if adequately documented.

The United Kingdom's temporary treatment of pension uprating during pandemic-related earnings distortions provides a contrasting case. Parliament enacted legislation in 2021 to modify the earnings element relevant to the state pension uprating process for the affected year [50]. Unlike an automatic operation of the pre-existing triple-lock rule, the temporary change required a new legal decision. Explaining why lawmakers chose it would require examination of the recorded statistical anomaly, fiscal implications, available alternatives, and legislative debate. The legal difference can be established directly; an exclusive causal interpretation of the political decision is a further inference.

7.2 Measurement can change the world it measures

The separation between real outcomes and recorded outcomes becomes dynamic when measurement itself creates incentives. Campbell's work on evaluating planned social change examined how the indicators used to judge interventions can become compromised by their decision-making roles [51]. Espeland and Sauder analyzed law-school rankings as reactive public measures: once organizations respond to evaluation, the rankings influence behavior, categories of comparison, and institutional priorities [52]. Bevan and Hood's study of English public healthcare targets likewise examined the interaction of performance criteria, reported improvement, and gaming [53].

Three effects must be investigated separately. The recording effect occurs when coding or coverage changes the statistic without necessarily changing the underlying target phenomenon. The behavioral effect occurs when the publication, sanction, or reward attached to measurement changes conduct. The policy feedback effect occurs when reported indicators, resource changes, or their political interpretation affect later rules. A ranking can exhibit all three, but evidence of one provides no automatic proof of another. In the HRRP case, the expanded diagnostic coding standard had a measurable effect on risk-adjusted records [2]. Whether a particular hospital changed treatment or access to care because of the incentives requires separate behavior and patient-outcome evidence.

Measurement may also produce systematic visibility differences. A formally published research citation is easy for a well-indexed database to register; conceptual influence circulating through meetings or shared professional vocabulary may be documented less consistently. The comparison depends on the records institutions are required to create and the languages and genres databases actually cover. If two policy areas generate very different documentary traces despite similar substantive uses of research, an unadjusted cross-sector citation count will have weak validity as a comparative measure of influence. This is a testable measurement-selection hypothesis, not a presumed fact about every jurisdiction.

7.3 Institutional changes alter both authority and observability

Measurement organizations may themselves undergo legal restructuring. Mexico provides a recent example. A reform published on 16 July 2025 reassigned social-policy evaluation and poverty-measurement functions formerly associated with CONEVAL; the national statistical institute, INEGI, stated that it assumed the new functions from 17 July 2025 [54][55]. The law, the effective institutional change, and the subsequent production of statistics are separate objects of analysis. One may document the transfer of authority without claiming that measurement quality, policy attention, or poverty itself changed in any specified direction on that date.

This example illustrates why policy influence assessment requires institutional dates, source versions, and authorized roles. A dataset produced before a legal transfer belongs to one observation and governance regime; after the transfer, responsibilities may be different. Whether statistical definitions, release schedules, operational resources, or research uptake changed requires a longitudinal comparison. The institutional restructuring is a confirmed event, while its consequences form testable causal questions.

7.4 Dynamic confounding and the conditions of feedback claims

Feedback creates identification problems because previous policy can alter variables that are both consequences of that policy and predictors of the next treatment. For example, a subsidy may change enterprise capacity, which then affects the chance of receiving an extension or additional support. Capacity is an intermediate consequence of the earlier policy and a potential confounder of the later policy's effect on outcomes. Comparing later recipients and nonrecipients without taking that structure into account may produce a biased estimate. Sequential causal designs, time-indexed structural models, and appropriate longitudinal methods can help when their assumptions are defensible. In historical cases without a credible counterfactual, a documented trajectory and mechanism evidence may remain more informative than an unsupported dynamic effect estimate.

Two propositions about policy feedback can be tested with explicit comparison structures. FB1, observability and later attention, predicts that among policy problems matched in independently assessed severity, institutional jurisdiction, and available decision capacity, problems whose outcomes are regularly and comparably measured in official statistics will receive more subsequent substantive attention than otherwise similar problems with limited official visibility. The units are problem-by-institution periods; exposure is the prior existence or introduction of an accessible official measurement series. Outcomes must include decisions, agenda placements, resource commitments, or documented deliberation obtained from sources distinct from the statistical publication itself; otherwise better recordkeeping could manufacture the apparent attention difference. Within-institution introduction of a measurement series, paired problems, and independently measured problem severity offer potential designs. Repeated comparisons showing no positive differential in substantive attention after controlling for issue importance and administrative selection, or evidence that apparent increases consist only of more reporting, would weaken FB1.

FB2, statistical anomalies and rule interventions, predicts that, among automatic adjustment regimes with real legal discretion to amend the rule, large deviations between the scheduled index and an independently documented measure of the condition the rule is intended to address will be followed by a higher incidence or shorter latency of discretionary legal intervention, conditional on fiscal capacity, the political calendar, and pre-existing reform plans. The analysis unit is a rule-by-uprating episode; an anomaly must be fixed using an ex ante measurement definition, and intervention means an enacted change or formally authorized suspension rather than routine execution of the standing formula. The 2021 UK pension case shows that such an intervention can occur [50], while U.S. zero-COLA years show that automatic formulas can produce politically salient outcomes without an amendment [49]. Neither case identifies the anomaly's causal effect by itself. If comparable prospective or historical episodes show no predictive association after the specified confounders and decision windows are accounted for, or if anticipated interventions consistently precede the alleged anomaly, FB2 would be weakened. Both hypotheses concern prediction within a stated sampling domain; neither authorizes an inference about policymakers' hidden motives.

8. Public Value and Decision Sufficiency: Turning Credible Measurement into Useful Action

8.1 From causal effects to a public-value profile

A policy may be effective on one outcome, expensive on another, and uneven in its distribution across groups. A source that helped change the policy can also have a causal contribution whose magnitude differs sharply from the policy's own total effect. These distinctions matter when measurement informs public action, because an empirically supported effect does not specify the public authority's priorities, the admissible trade-offs, or the resources available for alternative interventions. An effect profile describes measured consequences by population, outcome, and horizon. A public-value profile relates that evidence to lawful objectives, rights, resource costs, distributional consequences, and explicitly chosen criteria. Neither requires that all values be reduced to a single number.

The Scottish minimum unit pricing evaluation illustrates what an informative effect profile can establish. Wyper and colleagues used a controlled interrupted time-series design and estimated a 13.4% reduction in deaths wholly attributable to alcohol consumption associated with the policy, with a 95% confidence interval representing reductions from 8.3% to 18.4% [56]. The estimated reduction in wholly attributable hospital admissions was 4.1%, but its interval included zero [56]. The report also examined socioeconomic differences, finding more pronounced estimated benefits in several more deprived groups [56]. These are substantive outcome estimates, linked to a defined intervention and comparison design. A policy decision must additionally consider the groups bearing consumption costs, plausible behavioral responses, enforcement and implementation, public finance, and the legally authorized objectives.

The Institute for Fiscal Studies' 2025 Sure Start evaluation provides a longer-horizon accounting example. In its central modeled projections for a birth cohort, summarized in Section 11.4 and Table 11.1 of the original report, up-front program costs are approximately £2.69 billion, fiscal savings and revenue increases approximately £2.42 billion, and benefits outside government fiscal savings approximately £3.09 billion [57]. The final figure includes projected additional post-tax lifetime earnings of about £3.11 billion, benefits from reduced traumatic brain injury and fatal maltreatment, and an estimated £40 million negative component from increased victimhood associated with youth crime; several components are rounded [57]. Because the earnings and other long-horizon amounts are projections based on explicitly stated assumptions, they should be reported as modeled lifetime values, not realized cash returns. The separation between government savings and private benefits also helps prevent accidental double-counting when drawing a social valuation conclusion.

A public-value assessment should consequently preserve at least five categories: substantive outcomes, resources and opportunity costs, distribution across affected groups, legal authority and procedural rights, and consequences over time. The governing rule may weigh them in ways that cannot be represented as a universally agreed scalar objective. A rights constraint may exclude an otherwise attractive action; a fiscal rule may limit expenditure even where estimated aggregate social benefit is positive; an equity commitment may require specific attention to a group whose gains are obscured by the mean. For this reason a public institution should be able to publish a useful profile without inventing a numerical total policy influence score.

8.2 Minimum policy-relevant effects and statistical uncertainty

An identified effect and a policy-relevant effect are different objects. Let \(\theta\) denote a specified causal effect, such as avoided hospitalizations per 100,000 residents during a defined year, and let \(\delta>0\) denote the minimum effect magnitude considered meaningful for the decision at hand, expressed in the same outcome units. The value of \(\delta\) must be justified by the decision context: a statutory performance threshold, implementation cost, capacity constraint, specified public objective, or a legitimate determination about what would make action worthwhile. It is not a free parameter selected afterward to match a favorable estimate. When several outcomes matter, the relevant threshold may be a vector \(\boldsymbol\delta\), or a family of group-specific and time-specific conditions, rather than one universal cutoff.

Statistical confidence and practical importance must be interpreted separately. A study may estimate \(\theta\) with a narrow confidence interval entirely above zero yet entirely below \(\delta\). That result can be statistically informative while suggesting the effect is too small for the selected action under the stated criterion. Another study may produce a wide confidence interval that includes both zero and policy-significant benefits. It has not shown the effect to be zero; its uncertainty may justify obtaining better information if the choice is sensitive. A formal test of equivalence or a pre-specified non-inferiority margin answers another question again. The confidence interval describes sampling uncertainty under a model; \(\delta\) is a context-dependent substantive or institutional judgment. Neither substitutes for the other [37][18]. Figure 4(b) illustrates this distinction with schematic sampling intervals.

For illustration, suppose the possible improvement under a revision is \(\theta\in[4,10]\) avoided events per 10,000 eligible people, and an authorized decision rule requires at least \(\delta=3\) avoided events at acceptable costs and without violating other constraints. The entire identified range exceeds the threshold; for that particular criterion, narrowing the interval may not change whether the action qualifies. If instead \(\delta=7\), the interval crosses the threshold and the decision is sensitive to the missing information. This is a constructed example, not a published effect estimate. Figure 4(a) depicts the two thresholds relative to the same identified interval. It makes clear why a measurement system should record the threshold setter, the decision context, and the sensitivity of the choice to defensible alternative thresholds. If political or ethical authority is unsettled, the appropriate result may be a value profile and a transparent choice requiring authorization, rather than a researcher-imposed declaration of public benefit.

8.3 Jointly admissible worlds, value rules, and feasible actions

A decision can be evaluated only within an internally consistent set of empirical possibilities, value criteria, and feasible interventions. Let \(E\) denote the observed evidence, \(A\) the explicit empirical assumptions, and \(\mathcal W(E,A)\) the candidate worlds compatible with both, as defined in Section 5. Let \(\mathcal V\) be the set of public-value rules that are potentially relevant and lawfully admissible, with each rule \(v\in\mathcal V\) specifying the objectives and constraints under which consequences may be compared. Because some rules apply only in particular legal or institutional worlds, the relevant object is a set \(\Pi(E,A)\subseteq\mathcal W(E,A)\times\mathcal V\) of jointly admissible world–value-rule pairs, rather than an unrestricted combination of every conceivable factual model with every conceivable norm.

For any admissible pair \((\omega,v)\in\Pi(E,A)\), let \(\mathcal D_f(\omega,v)\) be the feasible actions that can legally and operationally be chosen in that configuration. This set includes the decision authority's resources, powers, timing, and relevant rights restrictions. An action might be feasible under one legal regime and unavailable under another. If a decision must be chosen before the true configuration is learned, a useful conservative choice set is the common feasible set

\[ \mathcal D_{\mathrm{common}}=\bigcap_{(\omega,v)\in\Pi(E,A)}\mathcal D_f(\omega,v). \]

Here \(\cap\) means the intersection of the feasible-action sets: an action in \(\mathcal D_{\mathrm{common}}\) can be committed to under every admitted configuration. When this intersection is empty, no single unconditional action can be safely promised across the entire range as modeled. The decision-maker may need to clarify legal authority, choose a state-contingent strategy with reliably observable triggers, revise the menu, or obtain information that resolves the relevant feasibility uncertainty. An empty intersection is a statement about the specified decision problem and accepted possibilities; it is not evidence that all substantive policies are equally undesirable.

Let \(U_v(d;\omega)\) denote the value assigned by rule \(v\) to the consequences of choosing action \(d\) in world \(\omega\). The function can be monetary, multidimensional with an explicit ordering, or otherwise suited to the legitimate mandate, but comparing two actions requires a meaningful common criterion. A particularly strong sufficient condition for a robust choice is that the set

\[ \mathcal D_{\mathrm{uniform}}=\bigcap_{(\omega,v)\in\Pi(E,A)} \operatorname*{arg\,max}_{d\in\mathcal D_{\mathrm{common}}} U_v(d;\omega) \]

is nonempty. The operator \(\operatorname*{arg\,max}\) returns the action or actions attaining the greatest value under a given configuration. Any action in the intersection is optimal under every admissible configuration and every retained value rule, conditional on a nonempty common feasible set and comparable utilities. This is a demanding sufficient condition for action under uncertainty, not a necessary condition for a useful decision. Its failure may reflect real normative disagreement, uncertainty about outcomes, or overly broad assumptions requiring further scrutiny.

When no uniform optimum exists, an authorized decision-maker can still use non-dominance, a legitimate partial ordering, pilot testing, constrained optimization, or minimax regret. For a fixed common feasible menu and a comparable scalar criterion, define the regret of choosing \(d\) in configuration \((\omega,v)\) as the difference between the payoff of the best available action in that configuration and the payoff of \(d\). A minimax-regret rule selects the feasible action with the smallest maximum regret across admitted configurations. Manski develops this family of decisions when the planner lacks enough knowledge to place a uniquely credible probability distribution over possible treatment effects [18]. The rule must be chosen for the decision problem; it cannot determine which social objectives or rights a government should respect.

This setup clarifies the distinction between empirical identification and decision sufficiency. A source-specific policy effect can be set-identified while the best authorized action remains stable over the entire set. Conversely, even a point estimate may leave the public decision unresolved when legitimate value criteria conflict or a necessary institutional authorization has not been granted. There is no coherent rule under which additional precision alone is guaranteed to improve action.

8.4 The value of new evidence, policy stopping, and a testable decision proposition

Consider a policy program whose proposed revision has a gain between four and ten common value units under every admissible model, while continuation has a gain of zero and both actions satisfy the applicable feasibility and rights conditions. Revision dominates continuation for that defined objective, even though the gain is only partly identified. Narrowing the interval to six through eight units increases scientific precision without changing the choice. If the gain is instead bounded between negative four and ten, no similar dominance follows. A targeted observation about a key implementation constraint or counterfactual effect may then be valuable. These examples use invented numbers solely to demonstrate decision logic.

Let \(E'\) denote a proposed additional observation, such as an independent administrative dataset, contemporaneous drafting record, or policy trial. Its decision value comes from the improvement it can produce in the selected action, the risks it helps avoid, or the quality of a relevant judgment, net of the costs of obtaining and applying it. In a probabilistic setting, computing a numerical expected value of sample information requires defensible probabilities and a common payoff metric. Without them, the analyst can report which new observations could change the choice, investigate worst-case regret, or bound plausible gains from information rather than manufacturing a precise monetary estimate. In all cases, the information-acquisition decision must account for investigation costs, delay, the sensitivity of records, and institutional authority [18].

A coherent stopping rule can therefore recommend action, further investigation, a limited pilot, revision, postponement, or cessation. A proposal may be rejected because an authorized decision-maker determines that a possible benefit is outweighed by cost or rights constraints; a deficient empirical claim may be withdrawn after its data are corrected; or an unidentifiable historical contribution percentage may no longer warrant research expenditure because no attainable evidence could affect any consequential decision. These outcomes are forms of purposeful policy and research management. Attributing the benefits of an avoided harm to a particular source remains a separate causal question that requires evidence of the forgone alternative and the source's role in preventing it.

The framework implies P4, decision sufficiency under partial knowledge: in at least some well-defined policy choice problems, the identified set for a consequential causal parameter will contain multiple values while the same feasible action is justified throughout the admissible model–value configurations. Its logical possibility follows directly from the intersection and dominance conditions above. Whether such cases are common, whether a particular institutional procedure identifies them reliably, and whether using that procedure reduces cost or improves policy outcomes are empirical questions for future assessment. A useful prospective test would record a conventional source-aware judgment, apply the integrated measurement protocol, document any changed conclusion and its cost, and examine important subsequent decisions against independent evidence. A same-analyst before-and-after comparison supplies developmental information but does not identify the protocol's comparative causal effect on decision accuracy.

8.5 A practical research and policy-action cycle

An organization assessing its own publications or advising a policy authority can implement these principles without beginning with a comprehensive technical platform. It starts from a real policy problem and an identifiable decision owner, determines which actions the owner can choose, and specifies the research product or source that might help. It then records delivery and reception, monitors the decision or implementation endpoint, and separately assesses source contribution and actual policy effects where identification is possible. Before recommending further study, it asks which remaining uncertainty is consequential for the next choice. Evidence collection should be proportional to this answer.

This cycle has two legitimate orientations. Evaluation assesses an existing source and its historical influence claims. Enablement uses what has been learned about decision nodes, organizational reception, and evidence gaps to improve the design and delivery of future research. In enablement, delivering a briefing or constructing a new monitoring tool can itself be an intervention; its uptake and consequences therefore need the same empirical discipline as any other policy-related action. A shorter decision pathway can begin with five questions: What is the precise claim? What original evidence supports it? What remains unidentified or uncertain? What feasible action depends on the answer? What additional observation would change that action, if any? Specialized models and extended records are added only when these questions require them.

The theory does not need to claim that a particular operational system already produces superior judgments. Its defensible contribution is an integrated argument linking policy anchor and relation, causal mechanism, observation process, identification assumptions, public-value criteria, and decision sufficiency. The corresponding practical method can be evaluated against strong conventional assessment, with independent policy cases, documented assumptions, adjudicated outcome quality, and measured research time. Repeated, well-designed applications would also allow explanatory and predictive tests: whether a proposed mechanism predicts the form of records in new cases; whether dependence adjustments improve judgments; and whether task-specific sufficiency rules lead to more useful investigation and action. These are substantive research commitments, directed toward measurable results rather than formal completeness for its own sake.

Conclusion

Policy influence arises from interactions among knowledge, power, resources, institutions, implementation, and the consequences of earlier public decisions. It leaves traces through documentary, administrative, statistical, and organizational systems whose own rules can change. The relation between these two processes governs what researchers can credibly observe, identify, and predict. A policy impact claim therefore needs a specified governance anchor, relation, target, source and version, time window, and, for causal questions, a defensible estimand and comparison. A citation, an actual episode of research use, a causal pathway, a net policy effect, and an assessment of public value are different empirical and normative questions that can be connected through evidence without being collapsed into one score.

The integrated framework developed here makes the dependence between generating mechanism and observation requirements explicit. Redundancy, prevention, preemption, anticipation, co-production, and feedback do not predict identical documentary histories. Incomplete records and dependent sources can still support strong conclusions where the diagnostic evidence and assumptions warrant them; partial identification gives the analyst a principled way to report the remaining possibilities. The result can be a verified fact, a supported pathway, a bounded effect, a clear contradiction under stated assumptions, or a genuinely unresolved question. Each is informative when its meaning is preserved.

The final step concerns action. Policy measurement has practical value when it helps an authorized institution make a better choice, identify a consequential uncertainty, or establish that more precise measurement is unnecessary for the decision at hand. This requires the distinction between an estimated effect and a minimum policy-relevant threshold, as well as explicit attention to costs, distributional consequences, legal feasibility, and legitimate valuation. A decision can be robust under partial knowledge; an exact numerical estimate can still be insufficient when rights or public objectives remain unsettled. The theory offers testable propositions about how influence becomes observable, how source dependence affects attribution, and when additional evidence changes decisions. Their comparative performance in institutional practice must be determined through actual assessments and independently credible follow-up. Building that evidence is the next step toward a policy impact measurement system whose principal achievement is better research and public action.

References

1. UK Government. (2019). The Climate Change Act 2008 (2050 Target Amendment) Order 2019, SI 2019/1056. https://www.legislation.gov.uk/uksi/2019/1056/made
2. Ody, C., Msall, L., Dafny, L. S., Grabowski, D. C., & Cutler, D. M. (2019). Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated. Health Affairs, 38(1), 36–43. https://doi.org/10.1377/hlthaff.2018.05178
3. Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426–431. https://doi.org/10.2307/3109916
4. Dahl, R. A. (1957). The concept of power. Behavioral Science, 2(3), 201–215. https://doi.org/10.1002/bs.3830020303
5. Bachrach, P., & Baratz, M. S. (1962). Two faces of power. American Political Science Review, 56(4), 947–952. https://doi.org/10.2307/1952796
6. Lukes, S. (2005). Power: A Radical View (2nd ed.). Palgrave Macmillan. https://books.google.com/books?id=XxpwQgAACAAJ
7. Crenson, M. A. (1971). The Un-Politics of Air Pollution: A Study of Non-Decisionmaking in the Cities. Johns Hopkins Press. https://books.google.com/books?id=1xJVAAAAMAAJ
8. Jaakkola, E. (2020). Designing conceptual articles: Four approaches. AMS Review, 10, 18–26. https://doi.org/10.1007/s13162-020-00161-0
9. Kingdon, J. W. (1984). Agendas, Alternatives, and Public Policies. Little, Brown. https://books.google.com/books?vid=ISBN9780316493915
10. Haas, P. M. (1992). Introduction: Epistemic communities and international policy coordination. International Organization, 46(1), 1–35. https://doi.org/10.1017/S0020818300001442
11. Sabatier, P. A. (1988). An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences, 21, 129–168. https://www.jstor.org/stable/4532139
12. Pierson, P. (1993). When effect becomes cause: Policy feedback and political change. World Politics, 45(4), 595–628. https://doi.org/10.2307/2950710
13. Adcock, R., & Collier, D. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3), 529–546. https://doi.org/10.1017/S0003055401003100
14. Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
15. Collier, D. (2011). Understanding process tracing. PS: Political Science & Politics, 44(4), 823–830. https://doi.org/10.1017/S1049096511001429
16. Halpern, J. Y., & Pearl, J. (2005). Causes and explanations: A structural-model approach. Part I: Causes. British Journal for the Philosophy of Science, 56(4), 843–887. https://doi.org/10.1093/bjps/axi147
17. Manski, C. F. (2005). Partial identification with missing data: Concepts and findings. International Journal of Approximate Reasoning, 39(2–3), 151–165. https://doi.org/10.1016/j.ijar.2004.10.006
18. Manski, C. F. (2011). Choosing treatment policies under ambiguity. Annual Review of Economics, 3, 25–49. https://doi.org/10.1146/annurev-economics-061109-080359
19. DellaVigna, S., Kim, W., & Linos, E. (2024). Bottlenecks for evidence adoption. Journal of Political Economy, 132(8), 2748–2789. https://doi.org/10.1086/729447
20. US Environmental Protection Agency. (2024). Timeline of Particulate Matter (PM) National Ambient Air Quality Standards (NAAQS) (official regulatory chronology documenting the 2020 retention and 2024 revision). https://www.epa.gov/pm-pollution/timeline-particulate-matter-pm-national-ambient-air-quality-standards-naaqs
21. Rogers, N. T., et al. (2023). Changes in soft drinks purchased by British households associated with the UK soft drinks industry levy: A controlled interrupted time series analysis. BMJ Open, 13, e077059. https://doi.org/10.1136/bmjopen-2023-077059
22. The BMJ. (2023). Retraction: Changes in soft drinks purchased by British households associated with the UK soft drinks industry levy. BMJ, 383, p2705. https://doi.org/10.1136/bmj.p2705
23. Bold, T., Kimenyi, M., Mwabu, G., Ng’ang’a, A., & Sandefur, J. (2018). Experimental evidence on scaling up education reforms in Kenya. Journal of Public Economics, 168, 1–20. https://doi.org/10.1016/j.jpubeco.2018.08.007
24. Lattimore, P. K., MacKenzie, D. L., Zajac, G., Dawes, D., Arsenault, E., & Tueller, S. (2016). Outcome findings from the HOPE Demonstration Field Experiment: Is swift, certain, and fair an effective supervision strategy? Criminology & Public Policy, 15(4), 1103–1141. https://doi.org/10.1111/1745-9133.12248
25. Williamson, A., Makkar, S. R., & Redman, S. (2019). How was research engaged with and used in the development of 131 policy documents? Findings and measurement implications from a mixed methods study. Implementation Science, 14, Article 44. https://doi.org/10.1186/s13012-019-0886-2
26. Newson, R., et al. (2018). Looking both ways: A review of methods for assessing research impacts on policy and the policy utilisation of research. Health Research Policy and Systems, 16, 54. https://doi.org/10.1186/s12961-018-0310-4
27. Newson, R. S., Rychetnik, L., King, L., Milat, A. J., & Bauman, A. E. (2021). Looking for evidence of research impact and use: A qualitative study of an Australian research-policy system. Research Evaluation, 30(4), 458–469. https://doi.org/10.1093/reseval/rvab017
28. Craver, C. F., & Bechtel, W. (2007). Top-down causation without top-down causes. Biology & Philosophy, 22, 547–563. https://doi.org/10.1007/s10539-006-9028-8
29. Hall, N. (2004). Two concepts of causation. In J. Collins, N. Hall, & L. A. Paul (Eds.), Causation and Counterfactuals (pp. 225–276). MIT Press. https://direct.mit.edu/books/edited-volume/3547/Causation-and-Counterfactuals
30. Tian, J., & Pearl, J. (2000). Probabilities of causation: Bounds and identification. Annals of Mathematics and Artificial Intelligence, 28, 287–313. https://ftp.cs.ucla.edu/pub/stat_ser/r271-A.pdf
31. Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17–36. https://doi.org/10.1111/1759-5436.12110
32. Jones, M. D., Peterson, H. L., Pierce, J. J., Herweg, N., Bernal, A., Raney, H. L., & Zahariadis, N. (2016). A river runs through it: A multiple streams meta-review. Policy Studies Journal, 44(1), 13–36. https://doi.org/10.1111/psj.12115
33. Hjort, J., Moreira, D., Rao, G., & Santini, J. F. (2021). How research affects policy: Experimental evidence from 2,150 Brazilian municipalities. American Economic Review, 111(5), 1442–1480. https://doi.org/10.1257/aer.20190830
34. García-Hombrados, J., Jansen, M., Martínez, Á., Özcan, B., Rey-Biel, P., & Roldán-Monés, A. (2026). Ideological alignment and evidence-based policy adoption. American Economic Review, 116(7), 2574–2603. https://doi.org/10.1257/aer.20240673
35. Shapley, L. S., & Shubik, M. (1954). A method for evaluating the distribution of power in a committee system. American Political Science Review, 48(3), 787–792. https://doi.org/10.2307/1951053
36. Aleshin-Guendel, S., Sadinle, M., & Wakefield, J. (2024). The central role of the identifying assumption in population size estimation. Biometrics, 80(1), ujad028. https://doi.org/10.1093/biomtc/ujad028
37. Imbens, G. W., & Manski, C. F. (2004). Confidence intervals for partially identified parameters. Econometrica, 72(6), 1845–1857. https://doi.org/10.1111/j.1468-0262.2004.00555.x
38. Lanners, Q., Rudin, C., Volfovsky, A., & Parikh, H. (2025). Data fusion for partial identification of causal effects. Advances in Neural Information Processing Systems, 38, 150638–150702. https://papers.nips.cc/paper_files/paper/2025/hash/dd4e79c2228c43954cc074abf2f85a07-Abstract-Conference.html
39. William T. Grant Foundation. (2020). Conceptual use of research evidence may be more common than you think (first-hand project account). https://wtgrantfoundation.org/one-more-take-on-the-conceptual-use-of-research-evidence
40. Christensen, J., & van den Bekerom, P. (2026). Unpacking the influence of policy advice using citation network analysis. Public Administration Review, 86(2), 430–444 (online first July 28, 2025). https://doi.org/10.1111/puar.70014
41. Springer Nature & Overton. (2025). SDG research in policy. Springer Nature SDG Impact Report (original analysis of policy documents, 2015–2025). https://stories.springernature.com/sdg-impact-report/sdg-research-in-policy/
42. Newson, R., Rychetnik, L., King, L., Milat, A., & Bauman, A. (2018). Does citation matter? Research citation in policy documents as an indicator of research impact – an Australian obesity policy case-study. Health Research Policy and Systems, 16, Article 55. https://doi.org/10.1186/s12961-018-0326-9
43. Kirkhart, K. E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. New Directions for Evaluation, 2000(88), 5–23. https://doi.org/10.1002/ev.1188
44. Mark, M. M., & Henry, G. T. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10(1), 35–57. https://doi.org/10.1177/1356389004042326
45. Langer, L., Tripney, J., & Gough, D. (2016). The Science of Using Science: Researching the Use of Research Evidence in Decision-Making. EPPI-Centre, UCL Institute of Education. https://eppi.ioe.ac.uk/cms/publications/systematic-reviews/science-of-using-science
46. Doberstein, C. (2017). Whom do bureaucrats believe? A randomized controlled experiment testing perceptions of credibility of policy research. Policy Studies Journal, 45(2), 384–405. https://doi.org/10.1111/psj.12166
47. National Institute for Health and Care Excellence (NICE). (2015). Trastuzumab emtansine for treating HER2-positive, unresectable locally advanced or metastatic breast cancer (TA371; superseded). https://www.nice.org.uk/guidance/ta371
48. National Institute for Health and Care Excellence (NICE). (2017). Trastuzumab emtansine for treating HER2-positive advanced breast cancer after trastuzumab and a taxane (TA458). https://www.nice.org.uk/guidance/ta458
49. US Social Security Administration. (n.d.). Cost-of-Living Adjustment (COLA) information and historical COLA series. https://www.ssa.gov/news/en/cola/index.html
50. UK Parliament. (2021). Social Security (Up-rating of Benefits) Act 2021, c. 32, 17 November 2021. https://www.legislation.gov.uk/ukpga/2021/32/contents/enacted
51. Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90. https://doi.org/10.1016/0149-7189(79)90048-X
52. Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40. https://doi.org/10.1086/517897
53. Bevan, G., & Hood, C. (2006). What’s measured is what matters: Targets and gaming in the English public health care system. Public Administration, 84(3), 517–538. https://doi.org/10.1111/j.1467-9299.2006.00600.x
54. Diario Oficial de la Federación (Mexico). (2025, July 16). Decreto reforming the Ley General de Desarrollo Social and associated finance and accounting laws. https://www.dof.gob.mx/nota_detalle.php?codigo=5763162&fecha=16/07/2025
55. Instituto Nacional de Estadística y Geografía (INEGI). (2025, July 23). El INEGI asume nuevas atribuciones de medición de pobreza y de evaluación integral de la política de desarrollo social. Press release 77/25. https://www.inegi.org.mx/contenidos/saladeprensa/boletines/2025/especiales/INEGI_Nvas_Atb.pdf
56. Wyper, G. M. A., et al. (2023). Evaluating the impact of alcohol minimum unit pricing on deaths and hospitalisations in Scotland: A controlled interrupted time series study. The Lancet, 401(10385), 1361–1370. https://doi.org/10.1016/S0140-6736(23)00497-X
57. Carneiro, P., Cattan, S., Conti, G., Crawford, C., Farquharson, C., & Ridpath, N. (2025). The short- and medium-term effects of Sure Start on children’s outcomes. Institute for Fiscal Studies research report. https://doi.org/10.1920/re.ifs.2025.0017