Dirty data poses a significant risk for New Zealand businesses looking to capitalise on the potential of artificial intelligence (AI), the commissioned survey by tech company Datacom has shown.
The survey, aimed at determining whether New Zealand businesses are well-positioned to effectively utilise their data, has revealed that only 9% of New Zealand businesses consider 100% of their data to be “clean,” while 30% believe that half or less of their data is free of issues.
47% of the surveyed Kiwi businesses also believe that “nearly 75%” of their data is sterile and that unclean or “dirty data” faces issues like duplicates, outdated information, insecurity, incompleteness, inaccuracies, or inconsistencies.
Managing director of Datacom New Zealand, Justin Gray, states that dirty data poses significant risks for Kiwi firms as the adoption of artificial intelligence continues to accelerate.
“The quality of AI outputs is wholly dependent on the quality of data that is available—garbage in, garbage out. For example, AI can be a powerful predictive tool and can identify patterns and trends within your business that you can use to guide strategic decisions, but if the data is off, then the AI analytics will be too,” he said.
The research also uncovers the lack of basic data management hygiene practices among companies, despite having good intentions for their data. 40% confessed they do not perform regular data cleaning, while 13% were uncertain if their business did.
Almost half of businesses conduct “regular data cleansing,” but among those that do, 51% are not unsure of how often this process occurs.
Although many businesses lack data governance and hygiene practices, the research indicates that companies are using their data for a variety of mission-critical functions. These functions include streamlining business operations, strategy and decision-making, enhancing the customer experience, and optimising internal processes.
The top issues or barriers being encountered when utilising data include incomplete data (51%), duplicate data (46%), inaccurate data (43%), and low-quality data (39%). Compliance and regulation issues were also identified as a concern by 34% of respondents, while 30% cited privacy risks, and 22% mentioned security risks as additional data challenges.
The research also examined how and where businesses are storing their data. While some organisations use a combination of storage locations, 53% said the most common remains on-premise servers, a group of servers that a company privately owns and maintains. Other popular storage options include private cloud (35%), New Zealand data centres (34%), public cloud (31%), and offshore data centres (15%).
Gray emphasises that, as technologies continue to evolve, the rate at which data is produced and collected will increase rapidly. Thus, businesses need to plan ahead for how they will manage, store, and protect this growing volume of data.
“Those companies that are relying on on-premise servers, for example, need to prepare for the rapid data growth that will come with greater AI usage and look at introducing backup options for data storage so they aren’t left scrambling,” he stated.
The survey, which was commissioned by Datacom and conducted in May 2024 by Curia Market Research, involves 200 senior decision-makers, data stewards, and IT managers working in New Zealand companies with over a hundred employees.