The Quiet Influence - Could AI shape Government decision-making in New Zealand

The Advisor You Never Hired

Every government depends on advisors. Their backgrounds, assumptions, and blind spots shape what options ministers see, which risks get flagged, and which solutions never make it onto the table at all.

New Zealand has spent decades building safeguards around this reality. Public service codes of conduct, transparency requirements, Official Information Act obligations, and Treaty partnership principles all exist, in part, to ensure that advice to government is accountable, diverse, and contestable.

Artificial intelligence is now entering that advisory space. It brings none of those safeguards with it.

This is not an argument against AI in government. The technology offers genuine and substantial benefits: faster processing of large datasets, more consistent document analysis, improved accessibility of public services, and significant productivity gains for stretched agencies.

But efficiency and influence are different things. As AI becomes embedded in how public servants research, draft, summarise, and present information, a question that has received insufficient attention in New Zealand deserves serious consideration.

Who shapes the AI that shapes government advice?

AI is not neutral - and neither is its Architecture

The idea that AI systems are objective is one of the more consequential misconceptions of the current moment.

Every AI model is the product of a long chain of human decisions: what data to train on, how to weight different sources, what outputs to reward during fine-tuning, which safety guidelines to enforce, and how to handle contested or ambiguous questions. These decisions are made by engineers, researchers, ethicists, and product managers - mostly at private technology companies, mostly offshore, operating under their own commercial incentives and cultural assumptions.

The result is not bias in the crude sense of a system that simply gets facts wrong. It is something more subtle and in some ways more consequential: a set of embedded assumptions about what questions are worth asking, what evidence is most credible, what trade-offs are reasonable, and what a good outcome looks like.

When a public servant uses AI to summarise a consultation document, analyse policy options, or draft a briefing, those assumptions travel with the output, quietly and invisibly, at scale.

This is not hypothetical. Research on large language models has consistently found that they reflect the perspectives most represented in their training data. A 2024 study published in PNAS Nexus found that responses from leading AI models systematically skewed toward culturally Western, educated, and individualistic viewpoints across a range of social and political questions. A separate analysis by researchers at the University of Washington, Carnegie Mellon University, and Xi'an Jiaotong University found measurable ideological variation between different commercial models - variation that users rarely account for, and may not even be aware of.

The AI is not making decisions. But it may be quietly narrowing the range of options that reach the people who do.

The ‘Framing Problem’

Of all the ways AI can influence policy thinking, framing may be the most important and least discussed.

Consider two versions of the same policy question:

"What is the most cost-effective approach to reducing housing unaffordability in New Zealand?"

"What is the most equitable approach to reducing housing unaffordability in New Zealand?"

Both questions are legitimate. Both will generate coherent, well-structured responses. But they will likely produce meaningfully different outputs. One will orient toward supply-side efficiency; the other toward distributional outcomes. The assumptions embedded in the question shape the space of answers the model considers relevant.

This is not a flaw in the technology. It reflects something true about policy itself: framing is constitutive. The way a problem is defined shapes the solutions that follow.

What is new is scale and speed. A human analyst brings framing assumptions to their work, but those assumptions are visible, contestable, and often stated. An AI system generates responses that can feel comprehensive and authoritative precisely because they are fluent and detailed. That fluency can make the framing invisible.

When the framing is set by whoever designed the prompt - a consultant, a vendor, a departmental template - rather than by the public servant who needs the analysis, accountability becomes murky.

Prompt design is not a technical detail. In a world where AI supports policy advice, it is a governance question.

The local knowledge gap

New Zealand has a constitutional and cultural context that is genuinely distinctive.

Te Tiriti o Waitangi is not background noise to New Zealand governance. It is a foundational document with live legal, political, and ethical implications. The relationship between the Crown and Māori shapes everything from resource management law to public health strategy to local government structure. The principles of partnership, participation, and protection that flow from Te Tiriti require ongoing, active interpretation, not algorithmic pattern-matching against historical text.

Yet the large language models now being deployed across the New Zealand public sector were developed primarily in the United States and trained predominantly on English-language content from American, British, and European sources. Their understanding of tikanga Māori, Te Ao Māori, the Waitangi Tribunal's body of jurisprudence, and the constitutional implications of the Treaty settlement process is, at best, thin.

The same applies to other aspects of New Zealand's policy environment: the specific structure of its resource management system, the role of iwi as governance entities, the demographics of Pacific communities, and the particular challenges of a small open economy at the bottom of the Pacific.

This does not make these tools unusable. It means they require users who understand the gaps, and organisations with sufficient internal expertise to catch what the AI misses or misframes. The risk is not that the technology fails visibly. It is that it fails quietly, producing outputs that are plausible but subtly miscalibrated for the New Zealand context, and that those outputs go unchallenged because the people reviewing them lack the specialist knowledge to notice.

Automation bias: The slow erosion of scrutiny

The most significant long-term risk posed by AI in government is not dramatic. It does not involve a system making a catastrophic error or seizing control of a process.

It involves the gradual erosion of critical scrutiny.

Researchers use the term automation bias to describe the well-documented human tendency to over-rely on automated systems — to accept their outputs without sufficient challenge, particularly when those outputs appear coherent and authoritative. Studies in aviation, medicine, and financial services have repeatedly found that even highly trained professionals become less likely to question automated recommendations over time, especially when the system has a track record of producing plausible results.

Government policy environments are not immune to this dynamic. If anything, they may be more vulnerable. Public servants are under significant time pressure. Ministerial deadlines are real. The temptation to accept a well-structured AI summary as a starting point and then, gradually, as a near-final product, is entirely understandable.

The concern is not that any individual official will stop thinking. It is that across an organisation, over time, the cumulative effect of AI-assisted work may be a narrowing of the questions that get asked, a compression of the range of options considered, and a reduction in the diversity of perspectives that inform advice.

Policy made within a narrower frame is not necessarily worse policy. But it is policy made with less visibility into its own assumptions, and that is a governance problem regardless of how good the AI is.

The concentration risk

New Zealand's public sector has a deep and longstanding relationship with Microsoft. That relationship has delivered genuine value: common infrastructure, familiar security frameworks, and reduced procurement complexity for agencies that are already stretched.

But as Microsoft Copilot and related AI tools become standard across government, New Zealand risks building its analytical infrastructure on a single foundation - one designed, trained, and governed by a private company in another jurisdiction, operating under that jurisdiction's legal framework and commercial incentives.

This concentration risk is not unique to Microsoft, and it is not an argument that Microsoft's products are unsuitable for government use. It is a structural observation: monocultures are fragile, and the fragility of analytical monocultures is particularly consequential in government.

When a single AI ecosystem shapes how public servants across multiple agencies research, draft, and summarise information, a shared set of assumptions about what is relevant, what is credible, and what trade-offs are reasonable can become embedded across the system simultaneously. There is no independent check, no alternative frame, no baseline against which to identify drift.

Genuine intellectual diversity requires more than multiple departments asking different questions of the same system.

What good governance looks like

The answer is not to avoid AI. The technology offers real and growing value. The answer is to govern it with the same seriousness that New Zealand brings to other forms of expert advice.

That means several things in practice.

Deliberate provider diversity. Agencies should be encouraged and, where appropriate, required to compare outputs across multiple AI systems before relying on AI-generated analysis for significant decisions. This is not about distrust of any particular vendor. It is about maintaining the intellectual diversity that good policy requires. Different systems will surface different framings, and those differences are information.

Investment in prompt governance. The way questions are put to AI systems is a policy input, not a technical afterthought. Agencies should develop internal guidelines on prompt design that reflect New Zealand's specific context, values, and Treaty obligations, and should treat prompt templates with the same scrutiny applied to terms of reference for external reviews.

Preservation of internal expertise. AI tools are most dangerous when the humans using them lack the knowledge to identify their errors. Departments should be deliberate about maintaining specialist analytical capability, particularly in areas where New Zealand's context diverges most sharply from the training data of offshore models. This means resisting the temptation to treat AI productivity gains as justification for reducing human expert capacity.

Transparency about AI use in policy processes. The public has a legitimate interest in knowing when AI-generated analysis has contributed to policy advice. New Zealand's existing transparency frameworks should be updated to address this - not to discourage AI use, but to maintain the accountability that democratic governance requires.

Culturally grounded evaluation. AI outputs that touch on Māori, Pacific, or other community-specific issues should be reviewed by people with relevant expertise before they inform policy. This is not a burden. It is the same standard of care that responsible agencies already apply to other forms of external advice.

The decisions behind the decisions

There is a version of the AI-in-government debate that focuses on the dramatic scenarios: autonomous systems making consequential decisions, algorithmic outputs replacing human judgment wholesale, the loss of democratic accountability in a single visible moment.

Those risks deserve attention. But they are not the most likely failure mode.

The more likely failure mode is quieter. It is the policy option that never made it into the briefing because the AI didn't consider it worth surfacing. It is the framing that went unchallenged because it arrived pre-packaged in fluent prose. It is the Treaty dimension that was underweighted because the model's training didn't adequately represent New Zealand's constitutional reality. It is the gradual narrowing of what government thinks is possible, driven not by any single decision, but by the accumulated effect of a thousand small ones.

New Zealand has built, over generations, a public service culture that takes seriously its obligation to provide free and frank advice - advice that is independent, diverse, and accountable to the public interest. That culture is worth protecting.

Artificial intelligence can support it. It can also, quietly and without anyone intending it, erode it.

The choice between those outcomes will not be made by the technology. It will be made by the policy choices New Zealand makes about how to govern it, and by whether those choices are made deliberately or by default.

That deliberation should start now.

References

Tao, et al. (2024). "Cultural bias and cultural alignment of large language models." PNAS Nexus, 3(9). Oxford Academic / PNAS Nexus. https://academic.oup.com/pnasnexus/article/3/9/pgae346/7756548

Feng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023). "From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models." Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pp. 11737–11762. https://par.nsf.gov/servlets/purl/10433148 (full paper PDF) 
Coverage via MIT Technology Review: https://www.technologyreview.com/2023/08/07/1077324/ai-language-models-are-rife-with-political-biases/