Sopheze Coffee Lounge in Timaru has a problem that no amount of customer service training can solve. An AI-powered tool listed the café’s phone number as the contact for a local hospital, and now staff are fielding calls from people trying to reach clinicians. It is a small, almost comic story. It is also a precise illustration of a structural failure that is about to get much worse.
The café has no clear avenue for redress. The AI tool that generated the error almost certainly disclaims liability in its terms of service, is likely operated from overseas, and has no practical incentive to correct a single listing in a small South Island town. The cost, unquantified but real, lands squarely on a business that did nothing wrong.
Cheap tools, expensive mistakes
The backdrop matters. New Zealand businesses are adopting AI at pace. The AI Forum’s third annual productivity report documents a decisive shift from experimentation to integration, with 91% of businesses reporting efficiency improvements and three-quarters reporting setup costs under $5,000. A year ago, nearly one-third were spending over $50,000. The cost barrier has collapsed, which means the governance barrier has collapsed with it.
Globally, Forrester projects that ungoverned generative AI use in commercial applications will cost B2B companies more than USD $10 billion by 2026. The firm identifies a surge in unregulated and experimental use of AI-enabled tools, combined with slow development of user skills, as the key driver. Their conclusion is blunt: the current lack of controls leaves companies exposed to substantial financial risk.
For a Timaru café, the risk is not a legal settlement. It is staff time, customer confusion, and reputational noise. Unquantified, uncompensated, invisible in every official dataset.
Most businesses using AI have no training to match
Thryv’s 2025 Business Index quantifies the gap: 57% of SMBs use AI, but only 13% provide structured AI training. Rob Hayden, Thryv’s global manager of AI innovation, is direct about what that means: “This mismatch matters for risk management and consistency. Without shared guidance, staff may adopt tools unevenly, produce variable outputs, and struggle to judge when human review is required.”
The Kordia cyber security report reinforces the point from a different angle. 24% of NZ businesses now identify staff using AI improperly as a top cyber security challenge, up from 16% the year before. Patrick Sharp, General Manager of Aura Information Security, describes the problem plainly: “Individual staff members are copying confidential data into AI systems, information they would never put into Google, without understanding the risks.”
Most of this discussion focuses on internal harm to the business deploying AI. The Timaru café represents the other failure mode: external harm to a third party caused by someone else’s AI output. That angle receives far less attention.
The accountability gap nobody has filled
Privacy Commissioner Michael Webster has begun articulating expectations for AI use in business, including senior leadership approval, privacy impact assessments, and human review before acting on AI outputs. His position on responsibility is unambiguous: “If you’re using AI in your business then it’s your responsibility to make sure you’re operating within the Privacy Act… Regardless of which staff are using AI, the size of your business… it’s your responsibility.”
That framework assigns accountability to the business deploying AI. Sensible enough. But in a directory or search AI scenario, the deploying business is not the one suffering the harm. The café did not choose to be listed. The entity that generated the error has almost certainly disclaimed liability, is domiciled overseas, and faces no regulatory consequence in New Zealand. Webster himself acknowledges the structural weakness: “Most of the public-facing AI tools now available have been developed overseas and are based on training data that may not be relevant, reliable, and ethical for use in Aotearoa New Zealand.”
The government is not blind to AI’s potential. Health Minister Simeon Brown recently announced that every NZ emergency department now has access to an AI scribe tool that enables doctors to see on average one additional patient per shift. That is genuine productivity. But clinicians have already flagged that the tool can misunderstand what is said and struggles to differentiate between patients. The failure mode is identical to the café case: AI producing incorrect outputs in real-world conditions, with humans left to catch the errors.
What business owners should do right now
Search your own business name in Google, ChatGPT, Bing Copilot, and Apple Maps. Check the phone number, address, hours, and description. Claim and verify your Google Business Profile, the primary data source many AI tools draw from. If you find errors, document them, document the operational disruption they cause, and document the time spent correcting them. There is no regulatory framework for AI accuracy yet. When one eventually arrives, documented harm will matter.
The Timaru café story is funny until you realise it is a market failure with no mechanism for correction. A tool made an error. A business bears the cost. Nobody is accountable. That is not an argument against AI. It is an argument for the most basic principle of commercial accountability: if your product causes measurable harm to someone else’s business, you should own it. Right now, in New Zealand, nobody does.
Sources
- AI Forum NZ: AI in Action – Key Findings from New Zealand’s Third AI Productivity Report (2025-08-01)
- IT Brief NZ: Forrester predicts AI errors to cost B2B firms over USD $10 billion
- NZ City News: NZ small businesses embrace AI but struggle to scale use (2025)
- Kordia NZ: Biggest AI cyber threat may be coming from inside your business (2025)
- Insurance Business Magazine NZ: Staff AI misuse now key cyber risk for NZ firms (2025)
- RNZ: AI scribe tool being used in emergency departments (2025)