Running local image models on a 4 GB or 6 GB GPU
Why your graphics card runs out of memory, and the settings that fix it, in the order worth trying them.
By Hasnain Butt Akhtar, founder and developer ·
If you have ever watched an image generator get most of the way through a picture and then stop with an out of memory error, this guide is for you. The good news is that a 4 GB card can produce genuinely good images. It just needs a few settings that most tools hide, and it helps to know which ones matter.
Where the memory actually goes
An image model does not load into your graphics card as one solid block. Three separate things compete for the same memory, and only the first one is the number people usually quote.
The model itself. This part is fixed and easy to predict. An SD 1.5 style model is around 860 million parameters, which works out at roughly 1.7 GB once loaded. An SDXL style model is 2.6 billion parameters, or about 5.2 GB, and it also loads two text encoders instead of one, which adds another 1.4 GB or so. That arithmetic on its own explains why SDXL is a squeeze on a 6 GB card and simply will not fit on a 4 GB one. No setting changes it.
The working memory during generation. This is the part that catches people out, because it grows much faster than you would expect when you ask for a bigger picture. The model does its work on a small internal version of your image, one eighth the width and height of the final output. At 512 by 512 pixels that internal version has about 4,000 positions to keep track of. At 1024 by 1024 it has about 16,000. Four times the pixels, but the model has to compare every position against every other position, so the memory it needs goes up roughly sixteen times, not four.
That is the single most useful thing to know here. Doubling the size of your image does not double the memory it needs. It can multiply it by sixteen.
The final step that turns it into a picture. When the generation finishes, one last stage converts that internal version into the image you actually see, and it does the whole thing in one go. On a large image this step alone can need over a gigabyte. If your generation reaches 100 per cent and only then fails, this is what ran out of room, not the model.
What each setting does
Most tools offer the same handful of options under slightly different names. They are not interchangeable, and some are much cheaper than others.
| Setting | What it does | What it costs you |
|---|---|---|
| Half precision (fp16 or bf16) | Stores the model in half the space with no visible loss of quality | Nothing. Leave it on. |
| Memory efficient attention | Does the heavy comparison work in small batches, so the biggest memory spike never happens | Nothing, and it is often faster |
| Attention slicing | An older, simpler version of the same idea | Around 10 to 20 per cent speed |
| Tiled decoding | Builds the final image in overlapping sections instead of all at once | A little speed, and very occasionally a faint join |
| Model offloading | Keeps part of the model in system memory and moves it across only when needed | Real speed, often two to three times slower |
| Sequential offloading | The strongest version, keeping only one piece on the card at a time | A lot of speed. Use it last. |
The order is what matters. Half precision and memory efficient attention cost you nothing, so they should already be switched on. Tiled decoding is nearly free and solves the fails at 100 per cent problem specifically. Offloading is the only one that genuinely slows things down, so save it for when the model is simply too big for the card.
The setting nobody mentions: image size
Asking a model for a much bigger image than it was designed for does not just use more memory. It usually makes the picture worse.
An SD 1.5 style model learned on images 512 pixels square. Ask it for 1024 directly and you will often get two heads, an extra arm, or a body that repeats halfway down. The model has never seen a composition at that size, so it fills the space with more of what it does know. People tend to blame their prompt when this happens. The prompt is usually fine.
The fix is what professional workflows do anyway. Generate at the size the model was built for, then enlarge. A 512 pixel image followed by a 2x enlargement gives you a clean 1024 pixel result, uses a fraction of the memory, and will look better than a direct 1024 generation almost every time. For SDXL style models the built for size is 1024, and the same logic applies from there.
A set-up that works, by card size
- 4 GB. Use an SD 1.5 style model with half precision, memory efficient attention on, tiled decoding on, one image at a time, at 512 by 512 (or 512 by 768 for portraits). Then enlarge by 2x. This runs comfortably and does not need offloading. SDXL is not worth attempting on this card, and no combination of settings will change that.
- 6 GB. Everything above, and SDXL becomes possible if you turn on model offloading. Expect it to be noticeably slower. Try it, time it, and decide honestly whether the result is worth the wait. For a lot of work, an SD 1.5 model at its native size with a good enlarger finishes sooner and looks better.
- 8 GB and above. SDXL at 1024 with no offloading, one image at a time. At this point most of the advice on this page stops applying to you.
Two things that catch out Windows users
Your graphics card is doing other jobs too. A web browser with hardware acceleration switched on can hold several hundred megabytes of graphics memory, sometimes a full gigabyte. On a 4 GB card that can be the difference between working and failing. Closing Chrome or Edge before a long run is an unglamorous fix, and it solves a surprising number of out of memory reports.
Windows may not fail. It may just crawl. Recent drivers will quietly spill past your graphics memory into ordinary system memory rather than stopping with an error. Instead of a failure you get a generation that takes eight minutes instead of forty seconds.
You can see this happening. Open Task Manager, go to the Performance tab, select your GPU, and watch two figures separately: dedicated GPU memory and shared GPU memory. If shared memory climbs during a run, your card has run out of its own memory and is borrowing from the rest of the machine. Lower the image size or turn on tiled decoding until shared memory stays flat, and the speed comes back.
Working out what went wrong
When something fails, the moment it fails tells you which of the three things ran out.
| What you see | What ran out | What to try |
|---|---|---|
| Fails before the first step | The model does not fit | Use a smaller model, or turn on offloading |
| Fails partway through | Working memory at this image size | Turn on memory efficient attention, or reduce the size |
| Reaches 100 per cent, then fails | The final conversion step | Turn on tiled decoding |
| Finishes, but takes minutes | Borrowing system memory | Free up graphics memory, or reduce the image size |
| Finishes, but the anatomy is wrong | Generating above the built for size | Generate smaller, then enlarge |
None of this is specific to any one program. The three things competing for memory are simply how these models work, so the same reasoning applies whichever tool you use, and the same five symptoms point to the same five fixes. If a tool gives you no control over tiling or attention, that is worth knowing before you decide your card is the problem.