Why I Run AI Locally Instead of Using ChatGPT for Client Work

Let me start with a question my clients ask me a lot:

"Can't we just use ChatGPT for this?"

My answer is always the same: it depends on what "this" is.

When "this" involves client intake forms for a law firm, tax documents for an accounting practice, or patient records for a medical office — the answer is no. And once I explain why, they always get it.

This post is about that explanation, and the toolchain I actually use instead.

The Part Everyone Glosses Over

When you send a prompt to ChatGPT or Claude via the API, that data leaves your network. It travels to a third-party server, gets processed, and comes back. The companies have policies about how they handle it — and you should read them — but the fundamental truth is: you handed your client's sensitive information to someone else.

For a lot of use cases, that's totally fine. Write me a landing page? Sure, use whatever.

But when the prompt contains:

• Attorney-client communications


• Personally Identifiable Information (PII)


• Financial records subject to confidentiality agreements


• Proprietary business logic a client has spent a decade refining

...you're in a different conversation. One that involves client trust, potential legal exposure, and in some industries, real regulatory obligations. HIPAA doesn't care that the AI gave a good answer.

What I Use Instead: Ollama

or drop a comment.