WHAT COULD A LOCAL AI MACHINE BECOME?

The things that already seem possible

If Qwen3.8-27B is close enough to Luna on routine work, I could run it continuously rather than opening it only when I have a question.

It could crawl public sources, papers, products, markets, and other material I care about. It could remove repetition, check claims, connect new findings with my projects, and maintain an endless filtered information stream. The stream might begin as text and audio. Later it could become a continuously rendered channel with a presenter, charts, maps, and live explanations.

The same system could keep developing software and looking for economic opportunities. It might notice a problem, check whether people care about it, test possible solutions, write a prototype, and monitor the result. I would still decide what deserves commitment and review consequential actions. Much of the search, synthesis, implementation, testing, and follow-up could continue without waiting for me.

Other workloads can use the card when those agents are quiet: private document analysis, image generation and editing, speech, music, simulation, or robot training. The point is not that one GPU does everything simultaneously. A local machine can change roles instead of being tied to one subscription or one interface.

The first machine is already concrete

I have downloaded and hash-verified a 19.54 GB Q5_K_M build of Qwen3.8-27B, plus its vision projector. Q5 should preserve more quality than the smaller 4-bit versions, and it fits on a 24 GB RTX 3090. The tradeoff is that only a few gigabytes remain for the context cache and runtime state.

This creates the first real decision. Q5 is attractive when answer quality matters and the context is moderate. A smaller Q4 build leaves more room for long prompts and concurrent requests. The best quantization depends on the work, so I should not turn “the model fits” into a general conclusion.

A sensible planning range on one 3090 is roughly 50–75 output tokens per second for one request. One highly tuned community setup reports 138 tok/s. The much larger 942 tok/s figure from the same report is total output across 64 concurrent requests. It is useful server capacity, but one agent does not experience that speed.

At 60 tok/s, a 2,000-token answer takes about 33 seconds to decode and an 8,000-token output takes about 2.2 minutes. Real agents also read files, run tests, wait for websites, retry, and sometimes follow the wrong idea. Completed useful tasks per day will matter more than the fastest token number.

There are several kinds of payback

Suppose a €900 card sustains 60 tok/s, the active system draws about 350 W at the wall, electricity costs €0.30/kWh, and every output token replaces a Luna output token of equal value. At continuous useful load, the idealized payback is about 281 active days.

That number is interesting and fragile. At 10% useful utilization, the same arithmetic stretches to roughly 7.7 years. If local output is only half as useful per token, payback roughly doubles. Idle power, cooling, failed tasks, maintenance, my review time, and the risk of a used card all belong in the calculation.

Token replacement is only one kind of return. Privacy and offline access have value. A local model may make experiments affordable that I would never send to an API. And if an agent finds one worthwhile opportunity, creates a useful product, or saves enough development time, that may matter more than millions of cheap tokens.

The measurement I want is cost per accepted result: hardware, electricity, retries, and attention divided by work I actually keep or act on.

A second card changes the choices

Two 3090s do not automatically become one 48 GB GPU. The software has to decide how to use them.

The simplest option is one Qwen server per card. This is well suited to many agents because one job cannot block every other job, and the slow second PCIe slot in my current motherboard matters much less.

The other option is one model split across both cards. With a Q4 model and a compressed KV cache, that should make one context close to Qwen’s 262K-token limit possible. It would let an agent hold a large repository or long body of research in one session. My current B450 motherboard gives the second card only a PCIe 2.0 ×4 path. A llama.cpp layer split should still work because relatively little data crosses between groups of layers, but prompt processing and generation will be slower. Tensor parallelism would be a poor fit.

If long context and shared high-throughput serving become normal rather than occasional, a workstation board with CPU-connected ×8/×8 slots, 128 GB or more system RAM, strong airflow, and a carefully sized PSU becomes part of the AI purchase. The motherboard can be as important as the second GPU.

The same hardware opens several experiments

Images, speech, and music. Strong local image models can run on a 24 GB card through quantization or offloading. Speech models are much smaller and could run on the existing RTX 3070. ACE-Step’s maintainers claim a full song in under ten seconds on a 3090. These are plausible everyday workloads, although quality and commercial licensing still need to be checked model by model.

A genuinely live avatar. This may be closer than I assumed. AVTR-1, MuseTalk, LivePortrait, and PersonaLive target real-time or streaming portrait animation and should fit on a 3090; some may already run on a 3070. A practical chain is speech recognition, Qwen, local text-to-speech, and a lightweight avatar renderer. The newest high-quality Alibaba LiveAvatar is a different class: its published path expects an 80 GB GPU or five GPUs, and the current implementation is CUDA-oriented.

Generated video. LTX-2.5 officially starts at 32 GB VRAM, so a single 3090 is below the supported floor. Quantized and offloaded workflows may work. Two 3090s can place components separately or run two independent candidates, but they are not a clean 48 GB replacement. The practical filmmaking approach is probably several conditioned 5–8 second shots assembled into a longer edit rather than one fragile 30-second generation.

Robot learning. Isaac Lab’s current requirements make a 24 GB card a credible starting point for reinforcement-learning simulation. With two GPUs, one could run simulation and policy training while the other serves a planner or another experiment. Camera-heavy scenes and complex policies change the memory requirement quickly.

The alternatives optimize for different things

Three 32 GB R9700 cards are a good example of why total memory can mislead. They add up to 96 GB, but every card still has only 32 GB and the software must explicitly split the model. My current board cannot host them properly. They might form an interesting LLM workstation on a new platform, but I would not buy them for a CUDA-only avatar model and hope that the port appears later.

Some models belong to another scale entirely

Parameter counts are not always disclosed, and with mixture-of-experts models the active weights tell only part of the memory story. GPT-5.6 Sol’s size is unknown. Kimi K3 makes the opposite case unusually visible: although roughly 104 billion parameters are active per token, the released checkpoint is about 1.56 TB because all experts still have to remain available.

A credible single-node K3 deployment uses eight 288 GB B300 or MI355X GPUs, several terabytes of system RAM, datacenter power and cooling, and a fast internal fabric. Renting such a node can cost roughly $15,000–$40,000 per month depending on commitment and provider. Owning one moves into several hundred thousand dollars.

This changes the local-versus-cloud question. API use is usually rational at low and bursty volume. Dedicated infrastructure becomes interesting when traffic is large, predictable, private, or expensive enough that a fixed machine stays busy. Even then, cost per successful task matters more than cost per token.

What has to be considered before the next purchase

• Quality. Does local Qwen complete my real tasks with the same supervision and acceptance rate as Luna?
• Context. Would I benefit more from the higher-quality Q5 model, a smaller quantization with more context, or a second card for one very long session?
• Concurrency. Do I need one fast conversation, many slower background agents, or two independent model servers?
• The rest of the system. PCIe lanes, RAM, PSU cables, airflow, temperatures, storage, noise, and Linux support can decide whether the configuration is usable.
• Utilization. Continuous generation has value only when the crawling, software work, or media output produces something I keep.
• Attention. An endless information stream can become another source of noise. Filtering quality and interruption policy may matter more than generation speed.
• Security. An agent capable of handling software or economic activity needs sandboxes, spending limits, audit logs, and explicit approval for consequential actions.
• Software risk. Multi-GPU support, quantization, drivers, and model licenses change. Hardware should be useful across several workloads rather than depend on one fragile repository.
• Opportunity cost. A quieter workstation card, rented 80 GB GPU, API budget, or motherboard upgrade may solve the real bottleneck better than another used gaming card.
• Resale and future models. Better small models may make 24 GB more valuable; a new affordable 32–48 GB card could also change used prices quickly.

The next step is still measurement

1. Run the verified Q5 Qwen build on one 3090 and record prompt speed, output speed, context limits, power, temperature, and stability.
2. Give Qwen and Luna the same real coding, research, and opportunity-finding tasks, then compare accepted results and supervision time.
3. Run one bounded always-on crawler and measure whether its filtered stream contains findings I act on.
4. Test concurrency before assuming a second card is necessary; compare one shared server with separate workers.
5. Try a lightweight live-avatar chain on the 3070 or 3090 before planning an 80 GB system.
6. Track displaced API use, electricity, completed software work, and opportunities found over several weeks.

Local AI remains a work in progress. I now have the model and a plausible first card. The important result will be whether the machine quietly produces useful things every day—and which limitation appears first when it does.