OWASP is a nonprofit whose Top 10 lists are the security industry's shared shorthand for what goes wrong in software. Since 2023 it has published one for applications built on large language models, the technology behind Claude, ChatGPT, Gemini and Copilot. The current edition is from 2025.
The list is written for engineers. The firms I work with don't build software. They're brokerages, law firms, practices and advisory firms, with somewhere between ten and a hundred people who need to know how the firm does things. Count the staff, the paralegals and the independent contractors: a brokerage with a dozen staff may have a hundred agents.
That's the size where the trouble starts. Below ten, people just call each other. Past ten, the same questions get asked and answered differently, and new people learn by interrupting someone. People are already using AI assistants to fill the gap, on their own accounts. Yet these firms are nowhere near an enterprise AI plan, and there's no one on staff whose job it would be to run one.
The risks on the list apply to them all the same, so here they are in plain terms. For each one I'll say what a firm can do about it and what we do about it. If you take only the advice and never talk to us, the advice still stands.
Instructions hidden in what the assistant reads (LLM01: Prompt Injection)
An assistant can't reliably tell your instructions apart from text it happens to read. When you ask it to summarize an email, a web page or a PDF, anything written in that document can try to steer it, for example by telling it to ignore you and forward the thread somewhere.
The defenses that work are simple. Limit what the assistant reads to sources you trust, and limit what it can do afterward. An assistant that can only read can still be misled, but it can't act on the misleading.
What your firm can do: be careful with any setup where an assistant reads outside content and can also send, buy or change something because of it. Either one alone is manageable. The combination is where the risk is.
What we do: your firm's guide is written and approved by people, not gathered from inboxes or the web. Our assistant tools can look up an answer in that guide and record a question it couldn't answer. They can't send, change or delete anything in your systems.
What leaks out (LLM02: Sensitive Information Disclosure)
Whatever goes into an assistant can come back out, in an answer to someone else, in a log, or in a vendor's training data, depending on the account and its settings. For most firms the realistic risk isn't trade secrets. It's a client's name, phone number or account detail, pasted in by someone trying to be helpful.
What your firm can do: decide what the assistant should know. Your process belongs in it: how you onboard a client, who signs off on what, where the forms are. Your clients' records don't. Keep those in the systems built to protect them. Check your plan's data settings too. Business plans from the major vendors generally don't train on your conversations, but personal plans differ, and the setting is worth confirming.
What we do: the guide holds how your firm works, never its records, and we don't keep a copy of your clients' customer data at any tier. Before any guide ships, an automated scan blocks Social Security numbers, card numbers, bank account numbers and dates of birth outright. Email addresses, phone numbers and street addresses can only stay with a named person's approval. Names can't be detected reliably by software, so a person reviews for those.
More power than the job needs (LLM06: Excessive Agency)
Newer assistants don't just answer. They can send email, edit files, book meetings and connect to other systems. Each permission is useful on its own. Together they add up to a lot of power held by something that, as the first risk shows, can be talked into things.
What your firm can do: give an assistant the fewest permissions that do the job, prefer read-only access, and have a person approve anything that sends, pays or deletes. When someone leaves the firm, their access should end that day.
What we do: the tools that look things up are marked read-only, and the one that records a question is marked as unable to delete or overwrite anything. Claude and ChatGPT show those labels to the person connecting. Your owners and managers keep your firm's list of people themselves. Removing someone ends their assistant connections immediately. Our iPhone app keeps a copy of the guide so it works without signal. That copy has to check in at least once a week to keep working, and a removed person's check-in is refused.
Confident and wrong (LLM09: Misinformation)
Assistants are fluent, and fluency sounds like authority. Given a question they can't answer well, they often answer anyway. In most businesses that's an annoyance. In real estate, law and financial advice, a confident wrong answer about what the law requires is a liability.
What your firm can do: separate two kinds of question. "How does our firm do this?" is something an assistant can answer well from guidance your firm approved. "What does the law require here?" belongs to a person, such as your broker, your attorney or your compliance lead.
What we do: every answer in a guide is labeled with what's at stake (legal, compliance, financial, security or none) and whether the person asking can act on it or should take it to someone. Your firm decides who that someone is for each kind of question. When the guide has no answer, the assistant says so and names that person instead of guessing, and the question is kept so the guide can be improved.
The other six
The rest of the list is mostly about how AI systems are built and hosted. Here's where each one stands for us.
- Supply chain (LLM03). We use Claude and ChatGPT as Anthropic and OpenAI provide them, and we don't host models of our own. Every release of our software is checked against known vulnerabilities in the code it depends on before it ships.
- Data and model poisoning (LLM04). We don't train models. Changes to a guide come from your people's requests and are reviewed before they're published.
- Improper output handling (LLM05). Our tools return plain text, and anything a person types that we show on a web page is escaped so it can't run as code.
- System prompt leakage (LLM07). The instructions we give the assistant contain no secrets. Access comes from each person's own sign-in, which only ever reaches their own firm.
- Vector and embedding weaknesses (LLM08). We don't run a vector search index. A question is answered only when it clearly matches a written answer. Otherwise the assistant is shown the list of topics and can try again with the right one.
- Unbounded consumption (LLM10). Many people use AI on free plans with daily limits, so we keep what our tools add to every message small. Owners and managers can also pause the assistant for the whole firm at any time.
Five questions to ask any AI vendor, and our answers
1. Does your product read our clients' records, and where are they kept? No. The guide holds your firm's process, not its records. Your clients' records stay in your own systems. If your firm someday needs the assistant to look something up in a live system, it reads that system where it is. We don't keep a copy.
2. Can the assistant act, or only answer? Who approves an action? Our tools only answer and record questions. Your people may connect other tools to their own Claude or ChatGPT accounts, and those are worth reviewing the same way. We're glad to help with that.
3. When someone leaves, how quickly does their access end? Their assistant connections end the moment an owner or manager removes them. The copy on their phone stops working at its next check-in, and it can't go more than a week without one.
4. What happens when the assistant is asked a legal or regulatory question? It doesn't give a legal answer. It names the person your firm has chosen for that kind of question.
5. What do you keep of what our people ask, and who can read it? For answered questions, we keep only which answer was used, not the question. Questions the guide couldn't answer are kept in full, because that's how the guide improves. Your owners and managers can see them, and so can we. We can't see anything your people do in their own Claude or ChatGPT beyond their questions to our tools.
These are good questions to ask of any AI tool your firm uses, including ours. If a vendor can't answer them plainly, that tells you something too.