Gemma 4 26B A4B

My main project is an all in one chatbot that focuses on research with a huge RAG and web browsing abilities (I ingest all my books, most of Wiki, all the big data sets for research papers and such). The idea is to have an auditable process that helps me sort through misinformation and basically the garbage that most of the internet has become. I’ve got RSS feeds feeding the RAG daily from places like Reuters to help get past any model’s knowledge cutoff and just have good information come in. But I don’t want it to be a clipping service that just cuts sentences out of pages and glues them together on a new one to make a scrapbook of paragraphs. I want it to take it all as evidence and reason through it. Be able to go back and forth with me as I drill in deeper into a topic. Additionally have it wired up to things like a big memory system to remember all my chats and piece together personal stuff about me so I can better plan things. Hooked into the documentation for my git projects. Etc. 🚨Gemma 4 26b MoE is matching and/or beating the 31b dense on every damn test I come up with🚨 This sub lead me to believe that dense good, moe bad. Moe dumb. I feel like maybe my testing is wrong or something. The responses I get out of the models take turns on giving me the best responses as well. So confusing. Anybody else with the same type of project that has run into the same thing? Do I just take the 2x/3x speed of moe and skip out on the dense? To note, this project isn’t about coding. Zero roleplaying although I do enjoy creative brainstorming on projects. Hardware: solo 3090. I’m able to fit Gemma 4 31b at Q4 cache with 65k+ ctx with QAT & MTP. From what I understand, the QAT is supposed to make the q4 KV not so much of a bad thing for Gemma. QAT UD-Q4_K_XL Note: I was originally using qwen3.6 and had the same issues of 27b getting outperformed by the 35b. I swapped to Gemma as I kept reading it was much better at writing so figured it would make query results easier to read.

Hey everyone! I'm currently working on a cooking web app where users can post their own recipes. I've already integrated the OpenAI Moderations API to filter out unwanted content (such as NSFW, violence, hate speech, etc.). It works great, but now I need a way to evaluate the overall quality of a post. Specifically, I'd like to check whether the recipe makes sense or is just gibberish, whether it's actually about cooking, whether it's complete, whether the attached image matches the recipe, and so on. I was thinking about using an LLM for this, and I found two free models on OpenRouter: Gemma 4 26B A4B and Gemma 4 31B. I'm looking for something that's accurate, reasonably fast, and suitable for this kind of validation in a production app. Which of these models would you recommend? Or is there another free model on OpenRouter that would be a better choice?

After many rounds of testing, pasting logs into chatgpt, killing my 11 year old ssd (to many log writes finally killed it), I have a pretty good setup working with VSCode. I thought I would share my settings... PC : Intel i7-9700, 32GB DDR4 2666 Ram, Gigabyte H310M-S2H motherboard, ASRock Radeon AI PRO R9700 GPU, Ubuntu 24.04 Llama.cpp server (vulkan) parameters : /app/llama-server -m /models/gemma-264B-8/gemma-4-26B-A4B-it-Q8_0.gguf --ctx-size 80000 --threads 7 --gpu-layers 99 --parallel 1 --flash-attn on --batch-size 2048 --ubatch-size 512 --cache-type-k q8_0 --cache-type-v q8_0 --cache-ram 8192 --ctx-checkpoints 3 --mmap --no-mmproj --reasoning off --reasoning-budget 0 --jinja --chat-template-file /models/gemma-264B-8/chat_template.jinja --temp 0.2 --top-k 64 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.15 --presence-penalty 0 Note that I am using the updated chat template posted a week or so back. With this setup my GPU Shows about 83% Vram Used. The --cache-ram 8192 goes to system ram. CPU Usage shown in webmin stays under 10%, and that is when using OpenWebUI on the same box. I get a about 1600 Prompt tps, and 60 tps for response. This can drop a bit as the context grows. VSCode Insiders Edition setup and results I tried to use the continue plugin, and I hated it. I finally found the fix which is this extension : https://marketplace.visualstudio.com/items?itemName=johnny-zhao.oai-compatible-copilot . It allows you to use your local LLM using coPilot (Agent, Plan and Ask all work). r/LocalLLaMA - Model settings in OIA Extension (https://preview.redd.it/gemma-4-26b-a4b-it-q8-0-results-with-vscode-long-post-v0-2s7qu9vmgxwg1.png?width=1107&format=png&auto=webp&s=8e766574d527718a1b80016547e93203303392e6) Model settings in OIA Extension r/LocalLLaMA - Advanced model settings in extension (https://preview.redd.it/gemma-4-26b-a4b-it-q8-0-results-with-vscode-long-post-v0-csnsdpwtgxwg1.png?width=1091&format=png&auto=webp&s=5c16b43d321b6b1358199409c3da9f7f5c594b14) Advanced model settings in extension I keep the allowable context in vscode below the server setting. The result : I am super impressed. all of the co-pilot features work... code quality is good, and while it does make mistakes i think that is partially my fault for not setting up system prompts, skills, instructions very well (still learning). In use, i create a plan in plan mode and add an instruction to "Keep changes concise and make the plan in small incremental steps" which really helps when it switches to agent mode and doesn't try to change everything at once. It is not perfect by any means... It sometimes gets into loops, or i get tool use exceeded messages. But, while I have been testing my setup I have managed to create a working Asteroids Clone, including tools to generate vector glyphs for the text display in game without writing one line of code (I am a developer btw, but not a game dev): media poster Gameplay I'd love to hear from others who are using a flow like this, get some more tips and help anyone if I can.