Command-line runtime tuner for Ollama

Find how much context your Ollama model can use on your GPU.

Find the largest context window that fits your Ollama model on your GPU without a measured slowdown. Finetuna tests context and runtime settings, checks GPU residency, and saves a named configuration you can reuse.

Install Finetuna See settings and a residency check

It changes runtime settings, not model weights. Results apply to the tested model, context, and host load.

Keep the settings you checked, then recheck the saved model

Invented Ollama session for llama3.2. Values below illustrate the file and residency messages; they are not measurements from your GPU or recommended settings for every machine.

1 · Check the loaded run Some of the model is off the GPU

finetuna --verify llama3.2

Illustrative console excerpt:

⚠️  Not fully on GPU → size_vram/size=75.0%

This tells you why a different context or batch setting is worth testing.

2 · Test and save a variant The settings survive the tuning session

Illustrative Modelfile-finetuna after testing a smaller context:

FROM llama3.2
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER num_batch 512

Create the named variant llama3.2-ft with the settings that pass on your host.

3 · Verify the saved name Check the configuration you will actually use

finetuna --verify llama3.2-ft

Illustrative successful console excerpt:

✅ Perfect! Model is fully on GPU (size_vram/size=1.000) — it’s hooked! 🐟

You have a named setup to reuse and a command to recheck when host load changes.

These ratios describe residency at check time, not speed or answer quality. Verification loads an existing model; it does not create one. No model weights are trained.

See the create, check, and verify commands

The saved Modelfile records num_ctx, num_batch, and num_gpu. Create the named Ollama variant from it, then run that name. Check the CONTEXT column in ollama ps instead of assuming a fixed default.

What it answers

Does this loaded run look GPU-resident? /api/ps compares size_vram to size. How much context still fits in that search? The largest window that stays on the GPU. Can you keep the settings? A named Modelfile variant.

A residency pass is for the model, context, and host load at check time. It is not a perpetual GPU guarantee. Re-run --verify after you change the model, context, concurrency, or host load.

--check and --dry-run never run ollama create. --verify loads an existing name and does not create a new one.

Leaving the incumbent is valid. The default context search picks the largest window that stayed on the GPU and was not measurably slower than the fastest candidate. A speed or batch search switches only when the win beats measured noise (median + spread).

Quick start

npm install -g finetuna
finetuna --check
finetuna

Non-interactive create: --model --name --ctx. Context fit-search: --auto-tune.

Flags: install and the GitHub README.

With ollanet

Finetuna runs on the machine that hosts Ollama. Each tool works alone. To find and chat with those models from another box, you can use ollanet.

Here: finetuna writes a named variant. There: ollanet scan then ollanet prompt. Same API.

What a run does

Pick a source model and a new name
Pick context / batch / GPU layers (presets through 128K)
Write Modelfile-finetuna and run ollama create
Measure baseline speed; optionally search context that still fits
Suggest a name like gemma4-ctx32k-flash

Built by Catalyst Forge LLC.