Part 3: Using AI to see trouble coming · MODULE 17
Before you paste customer data into an AI tool, replace the names, emails and phone numbers of individuals with codes, and switch off any setting that lets the tool train on your chats. Australia's privacy regulator recommends against entering personal information into publicly available AI tools. AI doesn't need names to rank debtors or build a forecast, so anonymising costs you nothing.
6 MIN READ · UPDATED 10 OCTOBER 2026
IN THIS MODULE
What counts as personal information
Does the Privacy Act apply to you?
What the regulator recommends
How to anonymise in five minutes
Choosing and setting up AI tools
Common questions
What counts as personal information in your receivables?
Personal information is information about an identifiable individual. In accounts receivable, that usually means:
The names of homeowners and other individual customers.
Sole traders' details, especially when the business name is the owner's own name.
Contact names, email addresses and phone numbers, including for staff at business customers.
Street addresses of homes where you did work.
Notes about someone's circumstances, such as illness or hardship, which can be sensitive information.
A company name on its own, like "Bluegum Constructions Pty Ltd", generally isn't personal information. The invoice amounts and dates aren't either, once you remove what links them to a person.
Does the Privacy Act apply to your business?
The Privacy Act generally applies to businesses with annual turnover above $3 million. Some smaller businesses are covered regardless of size, including health service providers, businesses that trade in personal information, and businesses that contract to the Australian Government.
Even if your business is under the threshold, treat customer data as if the Act applies. Your bigger customers may require it in their contracts, it's what your customers expect, and the rules have been tightening. Getting the habit right now is cheaper than fixing it later.
What does the privacy regulator recommend for AI?
In October 2024, the Office of the Australian Information Commissioner (OAIC) published guidance on using commercially available AI products. The points that matter most for a small business are these:
Privacy obligations apply to any personal information you put into an AI tool, and to any personal information in what it produces.
As a matter of best practice, don't enter personal information, and especially sensitive information, into publicly available generative AI tools.
Personal information collected for one purpose, such as invoicing, generally shouldn't be used for another without consent or a reasonable expectation.
Check whether a tool might send personal information overseas, and what the provider's terms say about how inputs are used.
AI output can be wrong. Keep a person checking it, and don't record AI conclusions about someone as fact.
How to anonymise a Xero export in five minutes
Is the total overdue going up or down compared with last week?
Has anything new moved into the oldest column? Deal with it before it gets older.
Which customers have money in three or more columns? They're falling behind on more than one invoice.
Does one customer owe a large share of the total, say a quarter or more?
Are there old, small balances that are really disputes or errors? Resolve them so they stop cluttering the report.
IMPORTANT
Bank account details, card numbers, tax file numbers, driver's licence numbers, and notes about anyone's health or personal circumstances. If an AI task seems to need them, it doesn't.
Choosing and setting up AI tools
Settings to check before you use an AI tool with business data:
Training on your data: Whether your chats are used to train the provider's models, and how to switch it off. Business and team plans often exclude your data by default.
Data retention: How long chats are kept, and whether you can delete them.
Where data is stored: Whether data leaves Australia, which matters if the Privacy Act applies to you.
Who can see it: Whether shared workspaces or links could expose chats to others in your team.
Tools built into your accounting software, or accounts receivable tools that connect to Xero, are a different case. They're designed to handle customer data, and they should say in their privacy policy where it's stored and how it's used. Read that before you connect them.
If customers can talk to an AI tool on your behalf, such as a chatbot, the OAIC recommends making it clear that it's AI. Update your privacy policy to explain how you use AI with customer information.
Common questions
Can I put customer information into ChatGPT or Claude?
The OAIC recommends not entering personal information into publicly available AI tools. Replace names, emails and phone numbers of individuals with codes before pasting, keep the key on your own computer, and switch off any setting that lets the tool train on your chats.
Does the Privacy Act apply to small businesses?
The Privacy Act generally applies to businesses with annual turnover above $3 million. Some smaller businesses are covered regardless, including health service providers, businesses that trade in personal information and Commonwealth contractors. It's good practice to follow it even if you're exempt.
Is a company name personal information?
A company name on its own generally isn't, because it identifies a business rather than a person. A sole trader's name, a contact person's name or a business name that's also the owner's own name usually is, so treat those as personal information.
