OpenWebUI is one of the best known and quickly growing frontends for those prepared to take on the challenge of self-hosting a frontend for large language models.

While self-hosting an LLM UI isn't for everyone and comes with the usual pros and cons of self-hosting, the advantages, for those prepared to invest the time, are many.

Users can avoid the usage caps associated with SaaS offerings like ChatGPT and gain direct access to a vast array of models from both mainstream and niche providers. There's also a prompt library, a built-in feature for maintaining knowledge, enabling RAG workflows, and a growing library of tools and pipelines delivered through Open Web UI Community.

Another powerful advantage of customisable frontends like OpenWebUI, LibreChat and others, is the ability to dial in models to suit your exact preferences. Everything from the system prompt to the temperature to parameters like Top P and Top K can be fine-tuned until you experience the kind of performance you're looking for.

Open Web UI is well known for its association with Ollama - and many choose to back the front-end with entirely self-hosted infrastructure for inference. However it works perfectly well without any attachment to that project whatsoever. My own instance simply uses cloud LLMs. As the name suggests, OpenWebUI places a high value upon excellent UI, and personally I've come to much prefer it than anything else I've found on the market.

But … is Open Web UI “production” ready?

While many open source projects like OpenWebUI have an association with hobbyists and tinkerers, with AI workflows embedding themselves quickly in the technology stack of even small businesses, a growing pool of users are wondering whether these tools can take the place of mainstream AI platforms in their workloads.

While ChatGPT recently informed me that SQLite is deployed in avionics, which sounded impressive, many old-schoolers, like me, will still balk at the idea of relying upon it as a database for production use. This isn't to say that SQLite isn't suitable for countless use cases, including this one. But from a data integrity and backup perspective, I’m firmly on the side of Postgres, and use it wherever and whenever I can in tech stacks.

OpenWebUI’s “RAG Problem”

You don't need to be an internet sleuth or spend much time on GitHub to see that users are not enamoured by the out-of-the-box performance of OpenWebUI's RAG integration.The issue of lacklustre or unacceptable RAG performance has come up on countless threads on GitHub, Reddit and wherever else the project is discussed.

Like many open source projects, the pace at which documentation is generated to keep track of features doesn't always move in sync with the evolution of the codebase. Thus, many users are unaware that other RAG databases are supported and virtually anything can be configured with a pipeline.

Out of the box, OpenWebUI uses ChromaDB, which comes pre-installed with the container. However, digging through the environment variable documentation reveals that a number of other databases can be provisioned.These can be configured by setting the appropriate choice with the vector_db environment variable. Besides Chroma, Milvus, Qdrant, OpenSearch, and PGVector are currently supported.

After moving from Chroma to a standalone Qdrant database, which I deployed as part of a stack, I found that the RAG performance was significantly faster, although some quirks about file type remained. Qdrant has the advantage over other other self-hostable RAG databases of having a basic web frontend as part of its design, so it's relatively easy, by visiting that or monitoring the container logs, to verify that embeddings and retrieval are actually happening against it and not ChromaDB.

Configuring Postgres

Another facet of Open Web UI configuration that should probably be highlighted far more prominently in the documentation is the fact that Postgres is also a supported database. However, as the documentation rightly notes, simply configuring an updated environment variable won't actually set the migration.

To set OpenWebUI set the DATABASE_URL environment variable and provide a Postgres connection string.

Migrating From SQLite to Postgres

So what can you do if your instance started with SQLite and you’re ready to move over to Postgres (and swap out ChromaDB for another RAG database?) Fortunately, there are a number of useful community projects, including a server-side Postgres script, which can make fairly light work of database data migration. However, expect some imperfections. I wasn't able to import my models with the web UI feature and instead had to resort to scripting and import with PSQL.

https://ciodave.medium.com/migrating-open-webui-sqlite-database-to-postgresql-8efe7b2e4156

Sadly, RAG is not so straightforward. Given that every vector database employs a slightly different method for embedding text and providing it for retrieval, the best advice is probably to either “start again” or, better yet, avoid this quandary from the get-go by separating your front-end from the other components of your AI stack.

Why Go To The Trouble?

Much like how I spent the first few months of my journey with Linux “distro hopping” (as it's come to be called) I spent a few months trying out all manner of front-ends for AI use before deciding that I liked Open Web UI the best.

I had a landmark moment when my instance wasn't accessible for a few hours and I had to go back to using ChatGPT (like a normal person!). For the record, I think that ChatGPT is an outstanding product and has provided the gateway for many to explore more fully the world of AI. But I was surprised to reach the conclusion that an open source project can actually reach a higher level of performance and utility than a product like ChatGPT with the backing of some of the world's leading and best funded technology providers.

While it took two very long days without much sleep to get there, I decided that my long-term interests were best served by ripping up the starter instance which I provisioned without much thought to longevity and replacing it with a much more deliberate and planned instance that I hope will serve my needs for quite some time.

One of the amazing things about AI is its ability to help even with understanding and developing with AI! I find that AI tools are extremely capable at creating and editing custom Docker Compose files, possibly a reflection of the fact that there is so much data around this that presumably makes it into their training data.

My main challenges in deploying the stack were the more usual and mundane complications of database migrations and DevOps concerns. But once all the pieces were finally in place and the container networking and reverse proxying and security figured out, the components talked to one another just as they should.

The Bottom Line

While the environment parameters are buried in a very long piece of documentation, it's absolutely possible to provision OpenWebUI with Postgres and a non-default vector database including Qdrant and others. Those who want to explore even further can use tools and pipelines to bring just about any type of RAG storage and connect it to the front end.