展開全文
Link: https://sanityboard.lr7.dev/
Yeah I've been running evals and working on this for over 3 days straight all day to get this all finished. Too tired to do a proper writeup, so I will give some bullet points and a disclaimer.
27 New eval results added in total
Got our first 4 community submissions, which brings us GPT 5.3 Codex Spark results, and a few Droid + Skills results to show us how big of a difference a suitable skills file can make.
3 New OSS coding agents; kilocode cli, cline cli, and pi*
Some site UI improvements, like date slider filter, being able to expand the filter options window, etc.
Interesting pattern I realized. GPT-codex models do really well cause they like to iterate, a lot. These kinds of evals favor models with this kind of tendency. Claude models don't iterate as much, so they sometimes get edged out in these kinds of evals. In an actual interactive coding scenario, I do believe the claude models are still better. Now if you want to just assign a long running task and forget it, that's where the gpt-codex models shine. They just keep going and going until done, they're good at that.
A somewhat important note, the infra used makes a HUGE difference in scores. I noticed this very early on, back when I used to run a ton of terminal bench evals, and especially when I decided to run it against as many different providers as I could to see which one was the best for Kimi K2 thinking. Even the speed affected scores a lot. My bench is no different in this regard, although I tried my best to work around this by having generous retry limits, and manually vetting every run for infra issues (which probably takes up the majority of my time), and rerunning any evals that looked like they may have suffered infra issues. This however isn't perfect, I am human. The reason I mention this is cause z.ai infra is dying. It made it almost impossible to bench against the official api. It was actually more expensive to use than paying standard api rates to claude for opus lol. They ghosted after I asked if I could have credits back for the wasted tokens I never got.. but that's neither here nor there. And also you might see some of the same models but from different providers score differently for infra reasons. Even the date of eval might matter for this, since sometimes providers change, either improving and fixing things, or otherwise. Also worth noting since some runs are older than others, some things might not score as well, being on an older agent version. Hopefully the filter by date slider I added can help with this.
*Pi was a large part of why this took me so much time and reruns. The retry logic had to be changed cause it's the only agent that does not have streaming stdout for some reason, and buffers it all until it's done. It also has 0 iteration whatsoever, it just does everything on one shot and never iterates on it again, leading to very poor scores. No other agents behave like this. These changes introduced bugs, which meant a lot of time spent fixing things and having to rerun things for fair evals. Pi I think is really cool, but since it's headless mode or whatever you want to call it is only a half complete implementation at best, it's almost impossible to get a fair evaluation of it.
媒體與討論
精選與此模型相關的公開影片和貼文。
X (Twitter)
Qwen ·
Qwen3.5-Plus is now on Poe!
查看原文Vercel Developers ·
Qwen 3.5 Plus is now available on AI Gateway. 1 million token context window, built-in adaptive tool use, and strong visual reasoning. Use 𝚖𝚘𝚍𝚎𝚕: '𝚊𝚕𝚒𝚋𝚊𝚋𝚊/𝚚𝚠𝚎𝚗𝟹.𝟻-𝚙𝚕𝚞𝚜'
查看原文Qwen ·
Qwen 3.5 Plus is now available on AI Gateway. Thanks @vercel_dev team. 🤝 Use model: 'alibaba/qwen3.5-plus' Try it now!
查看原文Reddit
Qwen 3.5 Plus is actually a really good cheap model
Kenjirio ·
展開全文
Been using it since reading a post on here through Alibaba Cloud's subscription. It being China aside for those who want the best bang for buck, I think Qwen 3.5Plus is the best you can get atm. I've tried Kimi, GLM, MiniMax and ngl they all weren't as good as the hype. Kimi came close, but I'd prefer ChatGPT but my multi agent workflow burns too many tokens for plus so I had to find alternatives. Since using Qwen it's actually responded well and doesn't go off the rails so far with my setup. It's only $3 to try it so might as well. Still yet to see if I burn through the usage on this but at that point it's on me to optimise better since it's very generous. Didn't expect it to be any good but hey I'm pleasantly surprised so far. Nothing can still beat opus etc. but its the best cheap model I've used so far. I'll update this post if I find something better. here's the guide I was referring to (https://www.reddit.com/r/AIToolsPerformance/comments/1rgue16/openclaw_alibaba_cloud_coding_plan_8_frontier/)
Qwen3.5 Plus, GLM 5, Gemini 3.1 Pro, Sonnet 4.6, three new open source agents, and a lot more added to SanityBoard
lemon07r ·
展開全文
Link: https://sanityboard.lr7.dev/ Yeah I've been running evals and working on this for over 3 days straight all day to get this all finished. Too tired to do a proper writeup, so I will give some bullet points and a disclaimer. 27 New eval results added in total Got our first 4 community submissions, which brings us GPT 5.3 Codex Spark results, and a few Droid + Skills results to show us how big of a difference a suitable skills file can make. 3 New OSS coding agents; kilocode cli, cline cli, and pi* Some site UI improvements, like date slider filter, being able to expand the filter options window, etc. Interesting pattern I realized. GPT-codex models do really well cause they like to iterate, a lot. These kinds of evals favor models with this kind of tendency. Claude models don't iterate as much, so they sometimes get edged out in these kinds of evals. In an actual interactive coding scenario, I do believe the claude models are still better. Now if you want to just assign a long running task and forget it, that's where the gpt-codex models shine. They just keep going and going until done, they're good at that. A somewhat important note, the infra used makes a HUGE difference in scores. I noticed this very early on, back when I used to run a ton of terminal bench evals, and especially when I decided to run it against as many different providers as I could to see which one was the best for Kimi K2 thinking. Even the speed affected scores a lot. My bench is no different in this regard, although I tried my best to work around this by having generous retry limits, and manually vetting every run for infra issues (which probably takes up the majority of my time), and rerunning any evals that looked like they may have suffered infra issues. This however isn't perfect, I am human. The reason I mention this is cause z.ai infra is dying. It made it almost impossible to bench against the official api. It was actually more expensive to use than paying standard api rates to claude for opus lol. They ghosted after I asked if I could have credits back for the wasted tokens I never got.. but that's neither here nor there. And also you might see some of the same models but from different providers score differently for infra reasons. Even the date of eval might matter for this, since sometimes providers change, either improving and fixing things, or otherwise. Also worth noting since some runs are older than others, some things might not score as well, being on an older agent version. Hopefully the filter by date slider I added can help with this. *Pi was a large part of why this took me so much time and reruns. The retry logic had to be changed cause it's the only agent that does not have streaming stdout for some reason, and buffers it all until it's done. It also has 0 iteration whatsoever, it just does everything on one shot and never iterates on it again, leading to very poor scores. No other agents behave like this. These changes introduced bugs, which meant a lot of time spent fixing things and having to rerun things for fair evals. Pi I think is really cool, but since it's headless mode or whatever you want to call it is only a half complete implementation at best, it's almost impossible to get a fair evaluation of it.
Qwen3.5 Plus and Qwen3.5 397B A17B Comparison
sirjoaco ·
查看原文YouTube
Qwen3.5 Plus is Here: Native Vision-Language Model on Chinese New Year
Fahd Mirza
查看原文Qwen3.5 Plus Vibe Coding Test - Building A Smart Inventory Agent!
Bijan Bowen
查看原文Alibaba Cloud Model Studio Full Tutorial: Build an AI Agent with Qwen3.5 Plus
Eric Tech
查看原文