David Jenkins
They matter because your company, not the software, is accountable for what the assistant tells homeowners. In Moffatt v. Air Canada, 2024 BCCRT 149, a British Columbia tribunal held Air Canada liable after its website chatbot gave a customer incorrect information about bereavement fares. The tribunal rejected the idea that the chatbot was responsible for its own statements and found the company had not taken reasonable care to ensure its accuracy.
Regulators have taken the same line on AI marketing claims. When the Federal Trade Commission launched Operation AI Comply in September 2024, its chair stated there is no AI exemption from existing law. For a management company, that means an assistant that misstates an assessment deadline or invents a rule creates the same exposure as a staff member who does.
A hallucination is a confident answer that is not supported by your documents. In community management, the risky ones are specific and plausible:
Even purpose built tools make these errors. When Stanford RegLab and HAI tested legal research tools designed to ground answers in trusted sources, they still produced incorrect information on 17% to 33% of benchmark queries. Legal research is harder than answering pool questions, but the finding is a useful reminder: grounding reduces errors, it does not eliminate them.
Use these questions in your evaluation. The right column is what should make you pause.
For turning the answers into enforceable terms, see what to look for in an AI vendor contract before you sign.
Walk away if a vendor claims zero hallucinations, refuses to show full transcripts, cannot explain what happens when an answer is missing from your documents, or will not put its data training policy in the contract. Each of those means you would be carrying risk you cannot see or measure.
Build your own test before you sign, because your documents and your homeowners are what matter. A simple method:
The trap questions often tell you more than the rest combined. For how to judge the overall share an assistant can handle, see can AI really resolve 90% of homeowner questions without a human?
STAN AI starts from what it will not do. It does not guess when an answer is missing or uncertain; it escalates to a person. It does not train the AI on your data. And it does not draw from a shared pool: each community gets its own custom assistant built on the management company's own knowledge base, so one client's information structurally cannot surface in another client's answers. That isolation is part of how the system is built, not a policy layered on top.
Escalation rules are set per community during implementation, including which topics always go to a person and how after hours emergencies are triaged. Because the assistant answers in each homeowner's language and understands their accent, accuracy is part of every conversation rather than an English only feature. If a current tool has already failed you on accuracy, see what to do when your AI vendor disappoints.
What is a good hallucination rate for an HOA AI assistant?
There is no universal benchmark, and rates depend on how they are measured. Focus on whether the assistant escalates when unsure and whether you can audit every answer.
Is a management company liable for what its AI chatbot says?
This is not legal advice, but in Moffatt v. Air Canada, a British Columbia tribunal held a company liable for its chatbot's incorrect information. Assume you are accountable for your assistant's answers and consult your attorney.
How do I test an AI assistant before buying?
Run 50 real questions plus 10 whose answers are not in your documents, in every language your residents use, and have a manager grade each answer.
Should an AI vendor train its models on our homeowner data?
That is your decision to make, and it should be stated in the contract. Many management companies require a written commitment that their data is not used for training.
