Running the GammaVibe pipeline on local models

I've had good success running local AI models - especially the Gemma 4 model family has worked well for me. I often use it with Rewire Text on my desktop, and I had previously experimented with a simple agentic app here: Local AI app with Gemma 4, Pydantic AI, and FLUX.

So I wanted to see if Gemma can handle my GammaVibe autonomous startup idea pipeline. The details are in the video below - bugs and all. In the end, I got it working reasonably well with the following models:

• google/gemma-4-e4b (for the simple, high volume steps - triage and extraction)
• google/gemma-4-12b-qat (for the steps that require more reasoning)

I had to implement some fixes to get this working. In particular, I had to replace UUIDs with small sequential numbers. In my real production pipeline, I already do this for the steps running on smaller Gemini models, but Gemini Flash and Pro can handle UUIDs just fine without truncating them. But gemma-4-12b was struggling with them, so I had to apply that technique to those steps as well.

I also had to make some tweaks to the model settings in LM Studio: Reducing the temperature from 1.0 to 0.35 and Top P from 0.95 to 0.9 was necessary to prevent it from cutting the newsletter post off prematurely. This finding is similar to my previous local model experiment.

After recording the video, I also experimented with the larger Gemma 4 models (first 31B, then 26B). I figured I might be able to replace the two small/medium models with a single large model and use this for all steps. But I wasn't able to get this working; 31B kept chocking on the triage step. I may try this again in the future, but for now, E4B and 12B continue to be the sweet spot for my hardware (Nvidia 4090).

More details in the video:

-Mirko