Over the past few months, we've been experimenting more and more with AI tooling/technologies in an effort to find ways to extract some real value from them. I've recently announced the release of our robot-blogger tool, only to follow it up with its obituary, since its output was not very good.

In both posts I mentioned one specific technology that I've found to be super useful for running LLMs locally, which I do a lot on my journey of discovery and AI mythbusting—that's Ollama.

Ollama is like Docker for LLMs. It lets you pull LLM images locally and run them on your own machine. I compare the Ollama experience to Docker since both have similar commands that do similar things: ollama pull like docker pull pulls an image, and ollama run like docker run runs an image.

I really like this interface for running LLM's locally, but in a recent project where I wanted to run Ollama in a Docker container, I ran into an issue. In a nutshell, I wanted to run an Ollama image that had a specific model ready to use when the container started, but as far as I could tell, there was no out-of-the-box way to do this.

So, I rolled my own Dockerfile and entrypoint script that enabled the behavior I wanted, and I'll share them with you here. These files will allow you to build a custom Ollama server image that will automatically pull your specified model when the container starts. Additionally, this image will fail health-checks (by design) until the model is fully downloaded, ensuring that your model is ready to use whenever the health-checks are successful. I currently run this image in a Kubernetes cluster, and it works great.

I hope you find it useful, and stay tuned for more AI content and updates from DoltHub!

The Dockerfile

FROM ollama/ollama:latest
ENV OLLAMA_KEEP_ALIVE=24h
ENV MODEL=deepseek-r1  # or whatever model you want to use

RUN apt-get update -y && apt-get install curl -y

COPY ./entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT [ "/entrypoint.sh" ]
EXPOSE 11434

CMD ["serve"]

The Entrypoint Script

#!/bin/bash

set -e  # Exit immediately if a command fails
set -o pipefail  # Exit on command pipeline failures

# Expected output from Ollama server
EXPECTED_OUTPUT="Ollama is running"
MAX_RETRIES=10  # Number of retries
SLEEP_TIME=2    # Seconds between retries

# Ensure MODEL is set
if [[ -z "$MODEL" ]]; then
    echo "❌ ERROR: MODEL variable is not set. Please specify a model to pull."
    exit 1
fi

# Start Ollama server in the background
OLLAMA_HOST=127.0.0.1:11155 /bin/ollama serve &
serve_pid=$!

echo "⏳ Starting Ollama server (PID: $serve_pid)..."

# Wait for Ollama to be ready
for ((i=1; i<=MAX_RETRIES; i++)); do
    RESPONSE=$(curl -s http://localhost:11155 || echo "")

    if [[ "$RESPONSE" == "$EXPECTED_OUTPUT" ]]; then
        echo "✅ Ollama is running (Attempt $i)"
        break  # Exit loop and continue script execution
    fi

    echo "⏳ Attempt $i/$MAX_RETRIES failed. Retrying in $SLEEP_TIME seconds..."
    sleep $SLEEP_TIME
done

# If the loop completes without breaking, Ollama didn't start successfully
if [[ "$RESPONSE" != "$EXPECTED_OUTPUT" ]]; then
    echo "❌ Ollama did not start within $((MAX_RETRIES * SLEEP_TIME)) seconds"
    kill $serve_pid  # Ensure we clean up the background process
    exit 1
fi

# Continue with pulling the model
echo "🔄 Pulling model: $MODEL..."
if OLLAMA_HOST=127.0.0.1:11155 /bin/ollama pull "$MODEL"; then
    echo "✅ Successfully pulled model: $MODEL"
else
    echo "❌ Failed to pull model: $MODEL"
    kill $serve_pid
    exit 1
fi

# Shut down the Ollama server
echo "🔄 Stopping Ollama server (PID: $serve_pid)..."
kill $serve_pid
wait $serve_pid 2>/dev/null || true  # Ensure graceful shutdown
echo "✅ Successfully shut down pulling Ollama server"

# Start Ollama in the foreground on the correct port (so container keeps running)
export OLLAMA_HOST=0.0.0.0:11434
exec /bin/ollama serve

Conclusion

We are excited to continue working with these growing technologies and would love to chat with you if you have any suggestions or feedback, or just want to share your own experiences. Maybe you found a great use-case for RAG at your company! Just come by our Discord and give us a shout. Don't forget to check out each of our cool products below:

• Dolt—it's Git for data.
• Doltgres—it's Dolt + PostgreSQL.
• DoltHub—it's GitHub for data.
• DoltLab—it's GitLab for data.
• Hosted Dolt—it's RDS for Dolt databases.
• Dolt Workbench—it's a SQL workbench for Dolt databases.