Resurrecting an Old Mining Rig for Night-Shift AI Work

At sevenseven.tech, we don't throw away hardware that still works.

Everyone talks about running Large Language Models (LLMs) on a $2,000 RTX 4090 or an expensive enterprise H100. But what about the hardware we already have gathering dust?

We recently set out to build a fully locally hosted AI stack, capable of privacy, document analysis, and code generation, using a setup that looks more like a 2017 crypto-mining rig than a modern AI server. Because it was: a retired mining rig, resurrected.

Here is how we deployed Ollama and OpenWebUI using Docker on a machine with four NVIDIA GTX 1060s, and why it's actually a viable tool for modern software development.

The Hardware: Quantity over Quality

The goal was to maximize VRAM on a budget. LLMs are hungry for memory; if the model doesn't fit in VRAM, it offloads to the system RAM, killing performance.

Individually, a 6 GB card struggles to run anything useful. But combined, we have 24 GB of VRAM. Thanks to Ollama's ability to split tensor layers across multiple GPUs, this stack can comfortably run 14B or even quantized 32B parameter models entirely on GPU.

The Software Stack

We are using a classic Docker stack with a reverse proxy:

1. Ollama (via OpenWebUI bundle): the backend inference engine.
2. OpenWebUI: a beautiful, ChatGPT-like interface.
3. DCGM Exporter: to keep an eye on those four GPUs.
4. Nginx: handling SSL and reverse proxying.

The Docker Compose Setup

We have stripped down our home lab configuration to show just the essentials for the AI stack. We use the official open-webui image, which handles the communication with Ollama.

Note: You must have the NVIDIA Container Toolkit installed on your host.

services:
  # The Interface + Backend
  open-webui:
    image: ghcr.io/open-webui/open-webui:ollama
    ports:
      - "3001:8080"
    volumes:
      - ollama:/root/.ollama
      - open-webui:/app/backend/data
    environment:
      - OPENAI_API_BASE_URL=/api
      - WEBUI_BASE_PATH=/
      - PUBLIC_BASE_PATH=/
    # This section unlocks the 4x GPUs
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: always

  # Optional: For function calling and external tools
  pipelines:
    image: ghcr.io/open-webui/pipelines:main
    container_name: pipelines
    ports:
      - "9099:9099"
    volumes:
      - pipelines:/app/pipelines
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: always

  # GPU Monitoring (Essential for multi-GPU setups)
  dcgm-exporter:
    image: nvcr.io/nvidia/k8s/dcgm-exporter:latest
    ports:
      - "9400:9400"
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: always

volumes:
  ollama:
    driver: local
  open-webui:
    driver: local
  pipelines:
    driver: local

Exposing to the World (Safely)

To access the UI from anywhere without using a VPN, we use Nginx with Let's Encrypt.

The most critical part of this configuration is handling WebSockets. LLMs stream tokens one by one. If your Nginx config doesn't upgrade the connection properly, you'll see the UI hang while waiting for a response.

server {
    listen 80;
    server_name your-domain.example;

    # ACME Challenge for Let's Encrypt
    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
        try_files $uri $uri/ =404;
    }

    location / {
        return 301 https://$server_name$request_uri;
    }
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name your-domain.example;

    # SSL Certs (your Let's Encrypt paths)
    ssl_certificate /etc/letsencrypt/live/your-domain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your-domain/privkey.pem;

    # Increase body size for uploading large documents (RAG)
    client_max_body_size 500M;

    # SSL Tweaks
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_session_cache shared:SSL:10m;

    location / {
        proxy_pass http://open-webui:8080;

        # Standard Headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket Headers (Critical for Token Streaming)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        # Timeouts preventing "504 Gateway Time-out" on long queries
        proxy_read_timeout 360000s;
        proxy_send_timeout 360000s;

        # Disable buffering for instant stream feedback
        proxy_buffering off;
        proxy_request_buffering off;
    }
}

Beyond the Chatbot: API & Developer Tools

The real power of this setup isn't just chatting in a browser. It's using it as a drop-in replacement for the OpenAI API in your development workflow.

Because OpenWebUI/Ollama provides an OpenAI-compatible endpoint, we can point our IDE AI plugins directly at this server. We currently use this rig to power:

• Cline & Roo Code: VS Code extensions that can edit files and refactor code autonomously.
• Mistral Vibe: for command-line coding assistance.

Instead of paying monthly subscriptions or sending proprietary code to the cloud, we simply point these tools at https://your-domain.example/api with a generated API key.

Architecture Diagram

Here is the stack as it runs in production. Note the WebSocket upgrade path that keeps token streaming alive.

Figure 1: The 4x GTX 1060 private AI cloud. Nginx handles TLS and WebSocket streaming; OpenWebUI talks to Ollama; Ollama shards the model across all four GPUs; DCGM watches the load.

The Results: 14 Tokens/Second on "Obsolete" Hardware

Does it work? Surprisingly well.

When we load a heavy model like qwen3-coder:30b with a massive 65k context window, Ollama detects the four GPUs and automatically shards the model across them.

While 14 tokens per second isn't breaking speed records compared to an H100, it is faster than human reading speed and perfectly smooth for coding assistance. The fact that the prompt processing (reading your code files) hits 36 t/s means the RAG experience feels snappy.

The Night Shift: Where This Rig Earns Its Keep

14 tokens a second is no match for a 4090 at your desk. But at 2 a.m., nobody is waiting on a keystroke. This is the rig you hand a job to at 10 p.m. and walk in to a finished result at 9 a.m.: the whole docs repo fed through RAG, a full codebase linted and refactored, overnight eval sweeps, batch embeddings, a mountain of PDFs summarized into one report. The 36 t/s prompt read means it chews through big inputs without stalling, and because it runs on its own board, it never competes with the fast rig you actually type into.

If you have a pile of older GPUs sitting around, don't throw them out. Cluster them. You might just build yourself a very capable private AI cloud.

Turn Idle Hardware into a Night-Shift Worker

Running powerful AI doesn't always require renting expensive H100s or sending your data to public API providers. A slower rig you already own is a perfect unattended worker: hand it the batch job your fast box would rather not spend tokens on, run it while you sleep, and read the output in the morning.

By hosting locally on your own bare metal, you ensure your sensitive data, proprietary code, and internal documents never leave your server room, all while eliminating unpredictable monthly subscription fees.

Whether you need a secure internal coding assistant, a private document analysis pipeline, or an overnight batch rig on aging hardware, we can build the infrastructure for you.

Want to put your idle hardware to work at night?

Contact us at support@sevenseven.tech or fill out our contact form.