The short answer
AI in small business software does a few specific jobs well: answering a phone after hours, drafting text, reading a receipt, sorting a queue. It is not judgment. Under Canadian law, performance claims must be backed by adequate and proper testing, and the AI vendor is a processor you remain accountable for.
“AI-powered” now appears on so many products that it carries roughly the information content of “cloud-based” in 2014. It tells you nothing about whether the thing works.
The useful question is narrower and answerable: which specific task is being automated, and what happens when it gets that task wrong? Every honest assessment of AI in business software comes from those two questions.
The four jobs AI genuinely does well
Turning speech into structured information. Answering a phone call, understanding what the caller wants, and producing a booking or a message. This works because the scope is narrow, the output is checkable, and the alternative — a missed call at 7pm — is a guaranteed loss rather than a possible error.
Reading documents. Extracting the vendor, date, total and tax from a photographed receipt. The output is a small set of fields a person can eyeball in two seconds. When it is wrong, it is obviously wrong.
Drafting first-pass text. A social post, a reply to a common question, a description. The value is not that the draft is good; it is that editing something is faster than starting from nothing.
Sorting and routing. Deciding which queue a ticket belongs in, flagging the urgent ones, grouping similar items. Misrouting is cheap to correct and easy to notice.
Look at what these have in common: narrow scope, checkable output, low cost of occasional error. That combination is the whole test, and it works better as a buying filter than any feature list.
What AI is consistently oversold at
The pattern inverts. AI is oversold wherever being wrong is expensive and hard to detect.
The dangerous failure mode is not the obvious one. Software that crashes gets noticed. AI that produces a fluent, confident, wrong answer does not, because it looks exactly like a right answer. If a task's output cannot be checked quickly by someone who knows better, automating it moves risk rather than removing it.
Be sceptical of three claims in particular: that a system “understands your business”, that it will replace a role rather than a task, and any specific percentage improvement offered without a described test.
Chatbot or agent? A chatbot answers. An agent acts — it books, creates, updates, sends. The risk profiles are not comparable. A wrong chatbot wastes a minute; a wrong agent changes your records or commits your business. When a vendor says “AI agent”, ask exactly which actions it can take without a human confirming, and whether that list is configurable.
What Canadian law says about the claims
This is the part buyers rarely realise gives them leverage.
The Competition Act makes it reviewable conduct to make a representation to the public that is false or misleading in a material respect, for the purpose of promoting a product or any business interest, by any means whatever [1]. More usefully for an AI purchase, it separately covers a representation in the form of a statement, warranty or guarantee of the performance, efficacy or length of life of a product that is not based on an adequate and proper test — and it states that the proof of that test lies on the person making the representation [1].
Read that again, because the burden is the point. A vendor claiming their AI resolves seventy per cent of enquiries, or cuts admin time in half, is making a performance representation, and the obligation to have tested it properly sits with them. You are entitled to ask what the test was: on whose data, over what period, measured how, compared against what.
Where a court determines a person has engaged in reviewable conduct under this Part, the remedies available include an order to stop, an order to publish a corrective notice, and administrative monetary penalties that scale with the benefit derived from the conduct [1]. The enforcement risk is a vendor problem, not yours. The practical value to you is that the question “what test supports that number?” is a reasonable, legally-grounded thing to ask, and the quality of the answer is diagnostic.
What happens to your customers' data
An AI feature almost always means information leaving your system to be processed somewhere else. That is a transfer, and PIPEDA has a settled answer about who is responsible for it.
Schedule 1, clause 4.1.3: an organization is responsible for personal information in its possession or custody, including information that has been transferred to a third party for processing, and shall use contractual or other means to provide a comparable level of protection while it is being processed [3]. Clause 4.7 requires safeguards appropriate to the sensitivity of the information [3].
Two consent tests then apply on top. Section 5(3) permits collection, use or disclosure only for purposes a reasonable person would consider appropriate in the circumstances — a standard that operates independently of whether anyone consented [2]. Section 6.1 makes consent valid only where it is reasonable to expect that an individual to whom the organization's activities are directed would understand the nature, purpose and consequences of the collection, use or disclosure [2].
The practical test is a conversation, not a document. Could you explain to the customer on the phone, in one plain sentence, what the AI is doing with what they just said? If yes, you are probably fine. If the explanation needs three qualifications, reconsider.
The geography matters too. AI processing frequently happens outside Canada, often at a provider your software vendor buys from rather than the vendor itself. The Privacy Commissioner's position is that such transfers are permitted, but that what an organization cannot do through contract — or by any other means — is override the laws of a foreign jurisdiction, and that organizations need to make plain to individuals, in clear and understandable language, that their information may be processed in a foreign country and may be accessible to that country's authorities [5].
So ask your vendor where the AI processing happens and who performs it. It is a fair question and a surprising number of resellers cannot answer it.
Automation does not suspend CASL
If AI is writing your outbound messages, every existing rule still applies, and volume makes mistakes cheaper to make and more expensive to have made.
CASL prohibits sending a commercial electronic message to an electronic address unless the recipient has consented and the message complies with the content requirements [4]. The message must set out prescribed information identifying the sender and any person on whose behalf it is sent, set out information enabling the recipient to readily contact one of them, and set out an unsubscribe mechanism [4]. The contact information must remain valid for a minimum of 60 days after the message is sent [4].
None of that changes because a machine drafted the text. If anything, the risk rises: automation makes it trivial to send ten thousand messages built on a consent assumption nobody checked.
Five questions to ask any AI vendor
- Which specific tasks does it do, and which does it not? A vendor who answers with capabilities rather than tasks is selling a category, not a product.
- What happens when it is wrong? You want a described behaviour — escalate to a human, flag for review, refuse to act — not a reassurance about accuracy.
- What test supports your performance claim? Grounded in the Competition Act obligation above.
- Where is the processing done, by whom, and does customer data leave Canada?
- Can I turn it off? If AI cannot be disabled for a workflow where you want a human, it is not a feature, it is a dependency.
A reasonable way to adopt it
Pick one task with a low cost of error and a fast feedback loop. Run it alongside your existing process for a few weeks rather than replacing anything. Look at the output yourself — not the dashboard, the actual output — and count how often you would have done something different.
If the answer is “rarely, and the differences did not matter”, expand. If it is “often”, you have learned something useful cheaply, which is the entire point of starting small.
The businesses that get value from AI are not the ones that adopted it hardest. They are the ones that automated a specific, checkable, annoying task and left everything else alone.