Self Hosted AI on a VPS: What You Can Run, What It Costs, and When It Makes Sense
Self hosted AI on a VPS means running an AI model or AI application on a virtual private server you control, instead of sending every request to a company’s AI service. A regular CPU only VPS can run small quantized language models, chat interfaces, embedding models and automation tools. It cannot run every model well. Larger models, longer prompts and several users at once push you toward more RAM, a stronger processor or a GPU.
This post is meant to help you decide, not to sell you on self hosting. For some workloads a VPS is a good fit. For others, paying for an AI API is cheaper and better. We will show both sides with numbers you can check.
We sell VPS hosting, so we gain when you pick a VPS. That is why we spell out where it is the wrong tool.
What Does Self Hosted AI on a VPS Actually Mean?
It means the AI software runs on a server you rent and administer, and your prompts and documents stay on that server. You pick the model, you install the software and you decide who can reach it. A hosting company supplies the hardware, and everything above the operating system is yours.
AI models run as a service on your own virtual private server
A language model is a large file of numbers. To use it, a program called an inference engine loads that file into memory and turns your question into an answer, one word fragment at a time. On a VPS, that program runs as a background service, much like a database does.
Your VPS has a fixed share of CPU and RAM. The model has to fit in that RAM, and the engine has to compute on that CPU. That is the whole story, and it is why hardware limits matter so much.
A rough sense of scale: a small quantized model is a download of a few gigabytes. A large one can be tens of gigabytes. The file size is a good first clue about whether a plan can hold it.
Self hosted AI and cloud AI services differ in who owns the machine
With a cloud AI service, you send a request over the internet to someone else’s models running on someone else’s hardware. You pay per use, and the provider handles scaling, updates and the model itself.
With self hosting, the model runs on your server. You pay for the server whether or not anyone is chatting, and you handle updates, security and capacity.
Neither is automatically better. Cloud services usually offer stronger models. Self hosting offers more control over data and a flat monthly infrastructure bill.
Many teams end up using both for different jobs. The right choice depends on your workload, which the later sections walk through.
Three separate pieces make up a working AI stack
The first piece is the model, the file of weights, such as an open weight model you download. The second is the inference engine, such as Ollama, llama.cpp or vLLM, which loads the model and answers requests. The third is the interface, such as Open WebUI, a browser chat screen that talks to the engine.
People often blur these together. Keeping them apart helps when something breaks. A slow answer is usually an engine or hardware problem, a bad answer a model problem and a login failure an interface problem.
You can also swap pieces. The same interface can talk to a local engine today and an external API tomorrow.
You control the data, the software and the settings
Hosting AI yourself gives you real control. Prompts and uploaded documents stay on your server unless you send them elsewhere. You choose the model version and decide when to update it. You set the rules for who can log in and what logs are kept.
Control comes with duties. If you leave a service open to the internet, that is on you. If a disk fills up, nobody else will notice.
Keep one more point in mind. Control over the server does not guarantee privacy by itself. You still need authentication, HTTPS and sensible backups, covered later in this post.
Running a model is different from connecting to an AI API
Running a model means your server does the computing. Connecting to an AI API means your server, or your app, sends text to a provider and gets an answer back. The first needs a lot of RAM and CPU. The second needs almost nothing, because the heavy work happens elsewhere.
This difference trips up many first time buyers. A tiny VPS can host an app that calls an AI API all day. The same tiny VPS cannot run a large model at all.
When you read advice online, check which one it describes. A tutorial that says a 2 GB server is enough usually means the API route.
What AI Tools and Models Can You Run on a VPS?
A VPS can run small to mid sized open weight language models through Ollama, a private chat screen through Open WebUI, automation workflows through n8n, document search through RAG, and lightweight embedding models. Each has different demands, and several of them need no local model at all.
Ollama runs local language models with one command
Ollama is a popular way to download and run open weight models. It detects a CPU only server and runs on the CPU with no extra setup, and it exposes a local API on port 11434. You pull a model, start it and send prompts.
Pick a model that fits. A 1B to 3B parameter model in 4 bit form is a sensible start on a small plan. A 7B or 8B model in 4 bit form is a realistic ceiling on an 8 GB or 16 GB plan.
Be ready for modest speed. Smaller models answer faster, bigger ones slower, and the quality difference between them is real. Test on your own plan before building anything on top.
Open WebUI gives you a private chat interface
Open WebUI is a browser interface that looks like the chat tools you already know. It connects to Ollama, or to any compatible API, and adds user accounts, conversation history and document uploads. It also lets people switch between a small, fast model and a larger, slower one.
It is a good fit for a small team that wants one shared, private place to chat with a model. It runs well in Docker, and the whole stack can start from a single Compose file.
One caution from experience: the first account created on a fresh Open WebUI install becomes the admin. Create yours immediately, then switch off open sign ups so strangers cannot register.
Automation platforms such as n8n can use AI without hosting a model
n8n is a workflow tool that connects apps and can call an AI model as one step. Running n8n on your VPS does not mean running the AI model there. An n8n step can call an external AI API, or it can call a local Ollama server. You choose.
That matters for sizing. An n8n server that only calls outside APIs needs modest resources. If you want the install steps, see our guide to running self hosted n8n on a VPS with Docker.
In our experience, many people who ask about AI on a VPS really want automation, such as sorting emails or summarizing tickets. That job often needs far less hardware than a local chat model does.
Document search with retrieval augmented generation fits small teams
Retrieval augmented generation, or RAG, lets a model answer questions about your own files. A program splits documents into chunks, turns each chunk into numbers called embeddings and stores them in a vector database. When you ask a question, it finds the closest chunks and hands them to the model.
RAG on a VPS works well for a manual, a policy library or a support knowledge base of modest size. The embedding step is light. The answering step still needs a model, local or external.
Watch the context window. Stuffing many chunks into a prompt makes each answer slower on CPU, so keep retrieval tight and limit how many chunks you send.
Embedding models and other lightweight AI services run well on CPU
Not all AI is a chatbot. Embedding models, text classifiers, speech to text for short clips and small vision models are far lighter than a large language model. Many run comfortably on a few gigabytes of RAM and a couple of cores.
This is where a modest VPS shines. Tagging support tickets, finding duplicate content or powering a site search with embeddings are all jobs a CPU handles without strain.
Pair one of these small services with an external API for the heavy reasoning, and you get a setup that is cheap, fast and honest about what each part does best.
What VPS Specifications Do You Need to Run AI?
RAM decides whether a model fits. CPU speed and memory bandwidth decide how fast it answers. Storage holds the model files.
For a small quantized model, 8 GB of RAM and 4 vCPUs is a workable starting point. For comfortable use of a 7B or 8B model, think 16 GB. Anything much bigger is GPU territory.
RAM sets the ceiling for local AI models
A quick rule: a model’s file size is roughly its parameter count times the bits per weight, divided by eight. A 7 billion parameter model at about 4.5 bits per weight comes to roughly 4 GB.
Add overhead. Budget about 1 GB for the engine and buffers, plus memory for the context, which is roughly 200 MB for a 4,096 token prompt on a 7B model. That makes about 5.3 GB for the model alone, before the operating system, Open WebUI and anything else on the server.
So an 8 GB plan can hold one 7B model with little room to spare, and a 16 GB plan is more comfortable. CPU cores matter less than people expect. Storage needs are small, since a model is a few gigabytes, but keep room for several versions.
A CPU only VPS and a GPU powered VPS behave very differently
On CPU, text generation is limited mostly by memory bandwidth, because the engine reads the whole model for every token. A useful estimate is bandwidth divided by model size. With 40 GB/s available and a 4.1 GB model, the ceiling is about 10 tokens a second.
Real results land lower, and a shared VPS gets only a slice of the host’s bandwidth. Published community tests of VPS class CPUs report single digit tokens a second for 7B models. For contrast, one published test on a large dual socket AMD EPYC server reached 20 to 28 tokens a second on 4 bit models, so the class of hardware changes the answer.
A GPU has far more memory bandwidth and can serve several users at once. Our NVMe VPS plans are CPU based and do not include a GPU. If you need GPU speed, plan on a different kind of server.
Model size, quantization and context windows shape performance
Quantization shrinks a model by storing each weight in fewer bits. A 4 bit version is roughly a quarter the size of the 16 bit original, so it needs less RAM and runs faster, with some loss in quality. Most CPU setups use 4 bit versions for that reason.
The context window is how much text the model can consider at once. Longer windows use more memory and slow each answer. Ollama’s own FAQ has described a default of 4,096 tokens that you can raise, and the setting is documented there. Raise it only when you need it.
Put simply, a small model with a short context on a 4 bit file is the safe zone for a CPU VPS. Every step up in size, precision or context costs speed.
Different AI workloads need different VPS resources
The table below matches common jobs to rough memory needs. Our NVMe VPS plans currently run from 4 GB to 32 GB of RAM on AMD EPYC processors with NVMe storage, so the plan column uses those sizes. Check the page for current specs and prices.
| Workload | Rough RAM needed | Plan size to consider | Honest expectation |
| Embeddings, classifiers, API calling apps | 2 to 4 GB | 4 GB plan | Fast and cheap. No local chat model involved. |
| 1B to 3B chat model, 4 bit | 4 to 6 GB | 8 GB plan | Usable for light chat and simple tasks. Answers are basic. |
| 7B to 8B chat model, 4 bit | 6 to 8 GB | 16 GB plan for comfort | Slow typist speed for one user. Fine for batch jobs. |
| 13B model, 4 bit, or RAG with a larger index | 10 to 14 GB | 16 to 32 GB plan | Noticeably slower. Test before committing. |
| 30B or larger, or many simultaneous users | 20 GB or more | GPU server, not a VPS | A CPU VPS will feel painful at this size. |
These are planning ranges, not guarantees. Measure on the plan you intend to buy.
More users and concurrent requests need extra resources
One person chatting is easy to serve. Five people sending requests at once is a different workload. On CPU, the same memory bandwidth now serves everyone, so each user gets a share.
A rough picture: if one user gets 5 tokens a second, four users hitting the model at the same moment might each see around 1 to 1.5. Batching can improve total throughput, but on CPU the gains are limited.
This is the point where many teams discover they need a GPU or an API. Count your peak simultaneous users, not your total user list. Ten people who chat at different times are far easier than ten who chat at 9 a.m.
How Much Does It Cost to Run Self Hosted AI on a VPS?
There is no single monthly cost, and anyone quoting one is hiding assumptions. The honest approach is example budgets with their assumptions shown. Below, infrastructure is kept separate from API usage and from the time you spend maintaining it.
Monthly VPS costs depend on how big a model you run
Our NVMe VPS plans listed at the time of writing: $5.99 a month for 2 vCPU and 4 GB, $10.99 for 4 vCPU and 8 GB, $20.99 for 6 vCPU and 16 GB, and $38.99 for 8 vCPU and 32 GB. Prices change, so confirm on the page.
Mapped to work: the $5.99 plan suits embeddings or an app that calls an API. The $10.99 plan can host a small local model. The $20.99 plan is the first sensible home for a 7B or 8B model with RAG. The $38.99 plan gives room for bigger models or several services together.
Pick the smallest plan that passes your own test, then move up only if you must.
GPU resources, storage and backups add to the bill
A GPU changes the cost scale. Renting a GPU server by the hour is typically priced very differently from a VPS, and the monthly total depends heavily on how many hours it runs. The plans on our NVMe VPS page are CPU based, so any GPU figure in this post is an assumption, not our pricing.
Storage is cheap by comparison. Models take a few gigabytes each, and document indexes grow with your content. Plan extra room for several model versions.
Backups deserve a line in the budget too. Back up your configuration, your vector index and any chat history you care about, and store a copy off the server. Models can be downloaded again, so skip backing those up.
Free AI software and paid API usage are different cost models
Ollama, Open WebUI, n8n and most open weight models cost nothing to download and run, though some models carry licenses with limits on commercial use. Read the license before you build a business on a model.
API usage is billed by tokens, which are word fragments. Prices vary by provider and model, and they change.
For the examples below, we assume a blended price of $3 per million tokens. That is an illustration. Replace it with a current price from the provider you would use.
Self hosting swaps a variable bill for a fixed one. That is good when usage is high and steady, and wasteful when usage is low.
Bandwidth, maintenance and administration are the hidden costs
Bandwidth is usually small for text. Downloading models uses some, but chat traffic is light. It can matter if you serve many users or move large files, so check your plan’s allowance. Watch memory and CPU during real use with a simple tool such as htop, so you see problems before users do.
Maintenance is the real hidden cost: updating the engine, patching the server, renewing certificates, watching disk space and fixing what breaks. Put a price on your own time. At an assumed $30 an hour, three hours a month is $90, which can exceed the server bill.
If nobody on your team wants that job, that is a reason to prefer an API.
Example monthly budgets show the range for three deployments
These are labelled examples built on stated assumptions. They are not quotes. The first two use our listed plan prices. The third uses an assumed GPU rental rate of $1.00 an hour for 176 hours a month, which is about 8 hours a day over 22 working days.
| Personal assistant | Small business, hybrid | GPU based, on demand | |
| Server | VPS at $10.99 (4 vCPU, 8 GB) | VPS at $20.99 (6 vCPU, 16 GB) | Assumed $176 (176 hours at $1.00) |
| Domain and backups (assumed) | $3 | $6 | $20 |
| Infrastructure subtotal | $13.99 | $26.99 | $196 |
| API usage (assumed) | $0, fully local | $20 for hard questions | $0, fully local |
| Money total | $13.99 | $46.99 | $196 |
| Maintenance time (assumed) | About 1 hour | About 3 hours | About 5 hours |
| Total with time at $30 an hour | $43.99 | $136.99 | $346 |
Read the bottom row twice. Time often dominates the cost of the cheapest setups.
Is Self Hosted AI Better Than Using ChatGPT or Other AI APIs?
Not automatically. Self hosting wins on data control and, at high steady volume, on cost. APIs win on quality, speed, simplicity and low volume cost.
Many teams do best with a mix. The honest answer depends on your data, your volume and who will maintain the server.
Data privacy, ownership and control are the strongest reasons to self host
If your prompts contain customer records, contracts or internal plans, keeping them on a server you control is a real advantage. You decide the retention period and who can read logs. Our post on AI in WHMCS support and customer data privacy goes deeper on that angle.
Two limits. First, many providers offer business terms that restrict how they use your data, so read those before assuming an API is unsafe. Second, a self hosted server that is badly secured can leak more than a well run API.
Check your own obligations, such as contracts or regulations, with a qualified adviser. We cannot give legal advice here.
Response quality, speed and capabilities favor the large APIs
The best hosted models are larger and stronger than anything that runs well on a small VPS. They handle long documents, complex reasoning and tricky code better, and they answer quickly.
A 7B model on CPU is a different tool. It is fine for summaries, classification, rewriting and simple questions. It struggles with hard reasoning, and it may be slow.
Do a blind test. Run your own real prompts through both and compare the answers. Your actual tasks beat any benchmark.
Speed matters too. A hosted model may answer in a second or two, while a CPU VPS may take half a minute for a long reply. For a chat tool people use all day, that gap is felt.
Predictable infrastructure costs compete with usage based API pricing
Use the assumed $3 per million tokens and a typical request of 1,500 tokens. One person making 50 requests a day uses 75,000 tokens a day, or 2.25 million a month. That costs $6.75 through an API, which is less than a $10.99 VPS.
Five users at 20 requests a day each use 4.5 million tokens a month, or $13.50. Still cheaper than the $20.99 server needed for a 7B model. The VPS only wins on cost when volume rises.
Break even against a $26.99 server is about 9 million tokens a month, roughly 200 requests a day. Against a $196 GPU setup, it is about 65 million tokens a month, roughly 1,450 requests a day. Below those lines, the API is cheaper.
Updates, security and maintenance become your job
Self hosting means you own security. Ollama binds to 127.0.0.1 by default, which keeps it private to the server, and its documentation explains how to change that bind address. The catch is that Ollama has no built in authentication, so never expose port 11434 to the public internet.
Keep the model endpoint private and put an authenticated reverse proxy in front of anything people reach. Use HTTPS, allow only the firewall ports you need, store credentials outside the code and update the server and containers on a schedule.
One Docker trap catches many people. Publishing a container port can bypass your firewall rules, so bind it to loopback, as in the Compose file below.
services:
ollama:
image: ollama/ollama
volumes:
– ollama:/root/.ollama
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports:
– “127.0.0.1:3000:8080”
environment:
– OLLAMA_BASE_URL=http://ollama:11434
– ENABLE_SIGNUP=false
volumes:
ollama:
Ollama publishes no port here at all, and the chat interface listens on loopback only. A reverse proxy with HTTPS and a login then exposes the interface alone. Putting a service such as the Cloudflare CDN in front can also hide your server address and absorb junk traffic. Attackers scan for open AI endpoints, which we describe in our outlook on AI driven website attacks.
Hybrid setups often beat full self hosting
A hybrid setup runs the light, private or high volume jobs locally and sends the hard ones to an API. Embeddings and document search stay on your VPS. Complex reasoning goes to a stronger hosted model.
This gives you much of the privacy benefit for the sensitive parts and avoids the quality gap for the hard parts. It also keeps your VPS small, which keeps the monthly bill low. Keep the routing rule simple, such as sending anything under a set length to the local model.
Design for swapping. If your interface and workflows treat the model as a replaceable setting, you can move a task between local and external without rebuilding anything.
When Does Self Hosted AI on a VPS Make Sense for Your Business?
It makes sense when privacy is a hard requirement, when usage is steady and light enough for CPU, or when you want predictable infrastructure. It stops making sense when you need top quality, many simultaneous users or no administration work. The checklist at the end turns that into a decision.
Private AI infrastructure suits some business workloads
Good fits include internal document search, ticket summarizing, drafting from templates, tagging and routing, and a private chat tool for a small team. If you resell hosting, support tickets from WHMCS are a natural source for an internal knowledge search.
Agencies and resellers can also build AI features into client services. Look at the tools that come with reseller plans before adding a new layer.
The common thread: the work is repetitive, text based and does not need the strongest model on earth.
Avoid workloads where a wrong answer is costly and goes unchecked, such as legal or medical guidance. A small model makes mistakes, and a person should review anything important.
Small CPU based VPS plans cover light jobs
A small plan is enough when you run embeddings, classification, automation that calls external APIs, or a small model for a handful of users who accept modest speed.
It also suits learning and prototyping. Spending $10.99 a month to learn how Ollama, Open WebUI and RAG fit together is cheap tuition.
Decision rule: if one user, one small model and a few requests an hour covers your need, a small VPS is plenty.
Be strict about expectations. Slow replies are normal. If you find yourself waiting, that is not a bug in your setup. It is what running a model on shared CPU feels like.
Choose a GPU server when speed or scale matters
Move to a GPU when you need fast answers from a 13B or larger model, serve many people at once, or run heavy jobs on a deadline. At that point CPU memory bandwidth is the bottleneck, and no amount of tuning fixes it. Ask what driver and CUDA versions the provider supports before you commit, because mismatches waste days.
A dedicated server gives you full hardware and more memory channels than a VPS slice, which helps CPU inference. Ask about current specs before assuming it includes a GPU.
Work out hours of use first. A GPU rented only while you need it can cost far less than one running all month.
Cloud AI APIs win in several situations
Choose an API when you need the strongest model, when volume is low, when you have no one to maintain a server or when you need to launch this week. Also choose it when requests spike unpredictably, since an API scales without you buying capacity. Watch provider terms and rate limits, because both can change.
If you only need a website or email, you do not need any of this. A plain shared hosting plan covers that, and AI features can call an API from there.
Our view: paying for an API is not a failure to be technical. It is often the economical choice.
Work through a practical checklist before choosing an AI VPS
Answer these in order. What is the task, and does a small model do it well enough? How many people use it at the same moment? Must the data stay on a server you control?
How many tokens a month do you expect?
What would an API cost at that volume?
Then: who patches the server, how will you secure the endpoint and where will backups live? Can you test a small plan first and measure real speed with your own prompts?
If the answers point to a small, private, steady workload with someone willing to maintain it, a VPS makes sense. If they point to quality, scale or no maintenance, use an API, or a hybrid.
If you are ready to test, start with the smallest plan that fits your model and measure it with your own prompts. You can compare current sizes and prices on our VPS hosting page.