In-House AI for the Support Team

Every support team knows this story. The engineer who quietly subscribes to a personal AI tool on their own card, and pastes customer tickets and log dumps into it "to get the job done." In most companies this is an open secret, quietly tolerated because the official alternative was too painful, and firing an engineer for wanting a faster answer is absurd.

For a company whose entire business is built on the promise of privacy, it is not a secret. It is a fireable offense - and the reason it still happens anyway is the interesting part. The official way does not exist. The engineer is not being careless out of malice; the engineer is doing the only workable thing available with the tools the company provided.

That is the cautionary tale of support in the AI era: the company that does not provide the tool ends up losing the data. The engineer's personal subscription becomes the de facto workflow, and customer device logs, connection logs, and network traces - some of which carry usernames and usage patterns - start flowing into third-party systems the company does not control, under privacy policies it did not write, possibly used to train models it will never see. For a privacy company that is not a policy violation. It is a brand contradiction in plain sight, being committed by well-meaning engineers who were never given a lawful alternative.

So we built the in-house version for the client, and made it the path of least resistance. The logs, the ticket, and the answer all stay on hardware the client controls. Nothing is sent to a vendor. And when the official tool is easier than the personal one, the leaks stop on their own.

1. The Compute: Two GPUs, Two Servers, Two Locations

The hardware was not even our idea. The client already owned two RTX devices that were sitting idle. We did not buy any of it - we made it work.

One thing to be precise about: these are two separate physical servers, in two separate locations. This is not a single box with two cards in it. The split is deliberate - it is the same reason we would never run a client's production workload on a single server, a single rack, or a single city. If one location loses power, or the network to it goes down, the other one is still serving the model. The agent just continues.

• Server One (RTX 5090, Location A): Serves the model with up to 262k context. This is the workhorse for long tickets, big log dumps, and reasoning-heavy jobs.
• Server Two (RTX 4090, Location B): Serves the same model at 110k context. Fast, cheaper to run, and the first failover target when Location A is unreachable.

In this deployment the model is qwen3.8:27b, and it runs exactly as well as the hardware allows. That is the whole rule of local inference, and it is a good one: any hardware that can run LLMs can run this stack - and the better the hardware, the bigger and better the model behind it. A company with a single modern GPU can run a smaller model and still get the entire workflow. A company with two high-end GPUs in two locations gets long-context, heavy-reasoning performance with genuine geographic redundancy. The stack adapts; the privacy model does not change.

Both servers expose a standard OpenAI-compatible endpoint. The Hermes Agent does not know - or care - which server is answering. It just gets tokens. There is no meter on the wire, no invoice at the end of month, and no copy of anyone's logs on a third party's disk.

The Result: Unlimited in-context work, and real redundancy. A 40,000-line log dump fits in a single prompt, retries are free, and the marginal cost of an analysis pass is a fraction of a cent in power - on hardware the client already paid for, spread across two sites so that no single outage takes the support desk offline.

2. The Stack

Nothing here is exotic. Every piece is a real, named, replaceable tool:

The point of the list is the absence in it. There is no vendor API, no hosted LLM, no SaaS chat platform. Everything that touches customer data runs on hardware the client controls.

3. Precision: A Knowledge Base That Gets Checked

Our client maintains one of the better-documented enterprise support knowledge bases we have worked with. We treat that as an asset, not a website.

Before the Hermes Agent forms an answer, it can open the relevant KB article and fact-check. A feature, a protocol, a plan limit, a server hostname - if it is going to be said to a customer, it has a reference. This closes the biggest failure mode of generic LLM support: confident wrongness. Hallucination is a confidence problem, and a well-documented KB is the antidote. The agent is allowed to be right only when it can point at the source.

4. The Nightly Crawl: Knowledge That Keeps Itself Fresh

Client websites change. New KB articles ship, prices move, app releases land. A knowledge snapshot from last month is stale by definition.

So a nightly GitHub Action does the crawl: it walks the client's site, extracts what matters, and converts it into structured .md files that the Hermes Agent loads when forming replies. Knowledge that was "out there on the web" becomes "in here, loaded, structured, fast." If the agent is ever unsure, it can still go back to the live site and verify - the local files are the fast path, the web is the fallback.

5. The Nightly Learner: Tickets That Teach

There is a second nightly job, and it points inward. It reads the day's tickets and the replies that actually shipped. Wherever the Technical Support Engineer corrected the Hermes Agent - a wrong hostname, a feature the plan does not include, a phrasing that did not land - that correction gets written into a skill file with a version number and a date.

The effect compounds. A mistake made once is never made again. A new pattern seen once becomes a standing playbook. And because the skill files are plain markdown in git, the whole learning process has an audit trail anyone can read.

6. The Workflow: Paste, Attach, Review

This is the part the support team actually uses, and it lives in the company's own Mattermost workspace - the same one the support team already uses for day-to-day communication.

That detail matters, and it is not only about privacy. Yes, self-hosting the chat tool means a conversation full of customer data never touches a third party. But the bigger point is friction. The Technical Support Engineer is not opening a second tool, learning a second interface, or switching contexts between "the chat app where we talk to each other" and "the AI app where we get help." There is only one place. Getting help from the Hermes Agent is literally the same action as messaging a colleague.

The workflow has no prompt engineering in it. The Technical Support Engineer:

• copy/pastes the ticket, with the previous replies, into the channel,
• attaches the logs the customer sent,
• hits send.

That is the whole instruction. The Hermes Agent already knows what to do, because the what to do - read the logs, map the failure to a detection layer, check the plan type, verify against the KB, draft the reply - was written down once in a skill, not improvised per conversation. Within a minute or two, a suggested reply appears, wrapped in a code block, ready to copy.

The Technical Support Engineer reads it, adjusts it, and pastes it to the customer. The human stays in the loop. The agent did the heavy lifting.

7. What It Actually Changed

• Speed. Humans skim logs. The Hermes Agent reads every line. A Wi-Fi link flap hiding inside 200 lines of healthy handshakes is invisible to a tired engineer at 5pm. It is not invisible to the agent. Manual log analysis misses things by design - it is a sampling process. The agent is an exhaustive one.
• Precision. Every claim is checked against the client's KB before it reaches a customer.
• Consistency. The 9am reply and the 11pm reply have the same quality, because there is no "tired" state to drift into.
• Satisfaction. Customers feel the difference: faster, more correct answers - even when they never see the stack. Tickets that used to take a day of back-and-forth close in one good reply.
• Zero new tooling. Because the whole thing lives in the support team's existing Mattermost workspace, there was nothing to install, no new login, no separate "AI portal" to maintain. Getting help is the same action as messaging a colleague - which is why the tool actually gets used.
• Privacy. The client's customer data never left the client's environment. That is not a feature on top of the stack. It is the design.

Conclusion: Private Is Not a Vanity Choice

Self-hosting gets framed as a hobbyist's flex. For a support operation that handles device logs, connection data, and usernames, it is something else: it is the difference between a tool and a system you can stand behind.

Two client-owned servers in two locations, a knowledge base the agent checks before it speaks, two nightly jobs that keep it current and teach it from real tickets, and a chat channel that needs no prompting. The bill at the end of the month? Just the electricity.

How It Fits Together

Figure 1: Customer data stays inside the client's environment the whole way through. The two nightly GitHub Actions keep the knowledge files fresh and teach the Hermes Agent from real tickets.

Want a support stack that keeps customer data in-house?

From two client-owned servers in two locations to a self-improving, KB-grounded Hermes Agent, we build AI support stacks your customers' data never leaves. Contact us at support@sevenseven.tech or fill out our contact form.