Local AI Image Generators on Consumer GPUs: What Runs and What Doesn't
Local image generation splits into two very different problems: whether a model fits in VRAM at all, and whether the software stack can reach your GPU. This explainer defines the memory math behind diffusion models, contrasts a 6B model with a 12.9B one, and shows why an 8GB card can run modern generators while a heavy video model on the same card cannot.
admin
The question "will this image model run on my GPU?" sounds like a single question. In practice it is two, and confusing them is why people either give up on hardware that would have worked or buy a card that still won't do what they wanted.
The first question is memory: does the model fit, in whole or in pieces, into the VRAM you have? The second is the software stack: can the tool you want to use actually reach your GPU at all? A card can pass the first test and fail the second, or fail the first and pass the second.
The memory math
Diffusion image models are neural networks with a fixed number of parameters, and parameters take space. A rough working rule is that each parameter costs about one byte at 8-bit precision, two bytes at 16-bit, and half a byte at 4-bit. A 6-billion-parameter model is therefore in the region of 6GB at 8-bit, before you add anything else.
That "anything else" is what trips people up. Generation also needs room for the text encoder that turns your prompt into conditioning, the VAE that decodes the latent into pixels, and the intermediate activations produced during sampling. Those overheads scale with resolution and batch size, not just with parameter count. This is why a model that technically fits can still fail at high resolution, and why the practical number to plan around is always larger than the weights alone.
Two levers change the math without changing the model. Quantisation shrinks the weights: a 4-bit base is roughly an eighth the size of a 16-bit one. Offloading moves components to system RAM and streams them to the GPU as needed, trading speed for fit. Both are now standard features of local generation engines rather than exotic hacks.
Two concrete examples
Take Z-Image, a 6-billion-parameter rectified-flow diffusion transformer from Alibaba Tongyi. It is listed among the models the omnichar-core engine runs locally on your own GPU ¹. At 6B parameters it is the kind of model that sits comfortably in the mid-range consumer card conversation once quantisation and offloading are on the table.
Now take Krea 2 from Krea AI, a 12.9-billion-parameter single-stream MMDiT, shipped as an undistilled RAW base plus an 8-step distilled Turbo variant. More than twice the parameters of Z-Image means roughly twice the weight memory at the same precision. The same source notes something genuinely useful and slightly counterintuitive: training is cheaper than generating, and Krea 2 fits a 16GB card at 512px with a 4-bit base for LoRA training. That is a specific, checkable data point, and it tells you the 16GB class is the realistic entry point for working with a model of this size rather than merely sampling from it.
The pair illustrates the rule cleanly. A 6B model is a fit problem you can usually solve on modest hardware. A 12.9B model is a fit problem that pushes you up a tier, and video models push further still: the same engine lists MiniMax H3 and LTX-2.5, the latter a 22-billion-parameter video model with sound. Video adds a temporal dimension to the activation budget, so a card that handles a 12.9B image model is not automatically a card that handles a 22B video model.
When the stack is the obstacle
The second question is the one that catches people with older or non-Nvidia hardware. An independent studio in São Paulo, AIVisionsLab, documented running a full local pipeline on an AMD Radeon RX 580 8GB with an Intel Xeon E5-2690 v3 and 32GB of DDR4 ECC, deliberately chosen as representative of what independent developers actually own ².
Their image generation path went through stable-diffusion.cpp on Vulkan, and their table routes heavy models, FLUX.1 and AnimateDiff among them, to ComfyUI on the CPU with the Xeon and its 32GB of ECC memory. That split is the whole lesson in miniature. The GPU handles what fits and what the Vulkan backend supports; the CPU absorbs the workloads that don't. No ROCm, no CUDA, no cloud bill.
So "what runs" depends on which backend your tool ships. A model that is perfectly sized for your VRAM may still be unreachable if your generator has no path to your GPU vendor, and a model that overflows VRAM may still complete on the CPU, just slowly.
The practical consequence
Before buying hardware or downloading a checkpoint, answer both questions in order. Estimate the weights from the parameter count and your intended precision, then add headroom for encoders, VAE and activations at the resolution you actually want. Then confirm your chosen tool supports your GPU through a backend it actually ships.
If you are working within a tight memory budget, the constraints are covered in more depth in our guide to running AI image generators on limited VRAM. If you are sizing a machine specifically for local AI work, the Framework Desktop mainboard is worth reading about as a form-factor reference point.
What the sources do not settle is how fast any of this runs on a given card. Neither source provides frame-time or images-per-minute figures for image generation on consumer GPUs, and no benchmark should be assumed from a fit table. Fitting is a necessary condition, not a promise of speed.
Sources
