How LLM Launches Land on Hacker News

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patie...

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example:>Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not all...

I just finished reading the 'paper' and I'm astonished that they aren't even publishing the # of parameters or even a vague outline of the architecture changes. It feels like such a slap in the face to all the academic AI researchers that their work is built off over the years...

That footnote on page 15 is the scariest thing i've read about AI/ML to date."To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and...

From the livestream video, the tax part was incredibly impressive. After ingesting the entire tax code and a specific set of facts for a family and then calculating their taxes for them, it then was able to turn that all into a rhyming poem. Mind blown. Here it is in its entir...

They also added both JSON and function support to the vision model - previously it didn't have those.This means you can now use gpt-4-turbo vision to extract structured data from an image!I was previously using a nasty hack where I'd run the image through the vision model to e...

Their naming and versioning choices have definitely created some confusion. Apparently GPT-4 Turbo doesn't just come with the the new vision capabilities, but also with improved non-vision capabilities:> "I’m hoping we can get evals out shortly to help quantify this. Until ...

People like to say AI is moving blazingly fast these days but this has been like a year in the waiting que. Guessing sora will take equally long if not way longer before the general audience gets to touch it.

Here's a blog post about what I've been building with GPT-4 Turbo + Vision for structured data extraction into SQLite database tables from unstructured text and images:https://www.datasette.cloud/blog/2024/datasette-extract/YouTube video demo here: https://www.youtube.com/watc...

Can I ask, how are people affording to play around with GPT4? Are you all doing it at work? Or is there some way I am unaware of to keep the costs down enough to play around with it for experimenting? It's so expensive!

The most impressive part is that the voice uses the right feelings and tonal language during the presentation. I'm not sure how much of that was that they had tested this over and over, but it is really hard to get that right so if they didn't fake it in some way I'd say that ...

Very interesting and extremely impressive!I tried using the voice chat in their app previously and was disappointed. The big UX problem was that it didn't try to understand when I had finished speaking. English is a second language and I paused a bit too long thinking of a wor...

Very, very impressive for a "minor" release demo. The capabilities here would look shockingly advanced just 5 years ago.Universal translator, pair programmer, completely human sounding voice assistant and all in real time. Scifi tropes made real.But: Interesting next to see ho...

I found these videos quite hard to watch. There is a level of cringe that I found a bit unpleasant.It’s like some kind of uncanny valley of human interaction that I don’t get on nearly the same level with the text version.

We've had voice input and voice output with computers for a long time, but it's never felt like spoken conversation. At best it's a series of separate voice notes. It feels more like texting than talking.These demos show people talking to artificial intelligence. This is new. ...

[dupe]Some more discussion: https://news.ycombinator.com/item?id=40996248

The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k.I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as eac...

Here's something interesting to think about: In ML we do a lot of bootstrapping. If a model is 51% wrong on a binary problem you flip the answer and train a 51% correct model then work your way up from there.Small models are trained from synthetic and live data curated and gen...

GPT-4o mini is $0.15/1M input tokens, $0.60/1M output tokens. In comparison, Claude Haiku is $0.25/1M input tokens, $1.25/1M output tokens.There's no way this price-race-to-the-bottom is sustainable.

@dang: This post isn't on the 1st or 2nd page of hacker news. Did it trip some automated controversy detection code for too many comments in the first hour?Edit: it says 181 points, 6 hours ago, and eyeballing the 1st page it should be in the top 5 right now.

So google will eventually be mostly indexing the output of LLMs, and at that point they might as well skip the middleman and generate all search results by themselves, which incidentally, this is how I am using Kagi today - I basically ask questions and get the answers, and I ...

That's an inflection point, all right. OpenAI's customers can now at least break even.Of course, it means a flood of crap content.

> For example, putting in 50k page views a month, with a Finance category, gives a potential yearly earnings of $2,000.> I'm going to take the median across all categories, which is an estimated annual revenue of $1,550 for 50,000 monthly page views.> This is approxim...

This article is weird clickbait which, even weirder, worked.It seems to assume a world where SEO entrepreneurs where ready to churn out million-page sites, but the cost per query were blocking them. There is no marginal cost, no SEO cost to adding another page, as long as a co...

This assumes a future where users are still depending on search engines or some comparative tool. Profiting off the current status quo. I would also be curious how user behavior will evolve to identify, evade, and ignore AI generated content. Some quasi arms race we'll be in f...

related tweet: https://x.com/OpenAIDevs/status/1834278699388338427OpenAI o1-preview and o1-mini are rolling out today in the API for developers on tier 5.o1-preview has strong reasoning capabilities and broad world knowledge.o1-mini is faster, 80% cheaper, and competitive with...

Related:OpenAI O1 Modelhttps://news.ycombinator.com/item?id=41523070

No GPT in the name, so maybe not transformer based?

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae...For 18,936 input, 2,905 output it cost 3.3612 cents.Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

I just pushed a new release of my LLM CLI tool with support for the new model and the reasoning_effort option: https://llm.datasette.io/en/stable/changelog.html#v0-21Example usage: llm -m o3-mini 'write a poem about a pirate and a walrus' \ -o reasoning_effort high Output (com...

For AI coding, o3-mini scored similarly to o1 at 10X less cost on the aider polyglot benchmark [0]. This comparison was with both models using high reasoning effort. o3-mini with medium effort scored in between R1 and Sonnet. 62% $186 o1 high 60% $18 o3-mini high 57% $5 DeepSe...

For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still co...

So far, it seems like this is the hierarchyo1 > GPT-4o > o3-mini > o1-mini > GPT-4o-minio3 mini system card: https://cdn.openai.com/o3-mini-system-card.pdf

GPT 4.5 pricing is insane: Price Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokensGPT 4o pricing for comparison: Price Input: $2.50 / 1M tokens Cached input: $1.25 / 1M tokens Output: $10.00 / 1M tokensIt sounds like it's so expensive and t...

Is it official then?Most of us have been waiting for this moment for a while. The transformer architecture as it is currently understood can't be milked any further. Many of us knew this since last year. GPT-5 delays eventually led to non-tech voices to suggest likewise. But w...

I got gpt-4.5-preview to summarize this discussion thread so far (at 324 comments): hn-summary.sh 43197872 -m gpt-4.5-preview Using this script: https://til.simonwillison.net/llms/claude-hacker-news-themes...Here's the result: https://gist.github.com/simonw/5e9f5e94ac8840f698c...

Considering both this blog post and the livestream demos, I am underwhelmed. Having just finished the stream, I had a real "was that all" moment, which on one hand shows how spoiled I've gotten by new models impressing me, but on another feels like OpenAI really struggles to s...

First impression of GPT-4.5:1. It is very very slow, for some applications where you want real time interactions is just not viable, the text attached below took 7s to generate with 4o, but 46s with GPT4.52. The style it writes is way better: it keeps the tone you ask and make...

Pricing: $150 / 1M input tokens, $600 / 1M output tokens. (Not a typo.)Very expensive, but I've been using it with my ChatGPT Pro subscription and it's remarkably capable. I'll give it 100,000 token codebases and it'll find nuanced bugs I completely overlooked.(Now I almost fe...

This is their first model to only be available via the new Responses API - if you have code that uses Chat Completions you'll need to upgrade to Responses in order to support this.Could take me a while to add support for it to my LLM tool: https://github.com/simonw/llm/issues/839

It cost me 94 cents to render a pelican riding a bicycle SVG with this one!Notes and SVG output here: https://simonwillison.net/2025/Mar/19/o1-pro/

Assuming a highly motivated office worker spends 6 hours per day listening or speaking, at a salary of $160k per year, that works out to a cost of ≈$10k per 1M tokens.OpenAI is now within an order of magnitude of a highly skilled humans with their frontier model pricing. o3 pr...

It has a 2023 knowledge cut-off, and 200k context window... ? That's pretty underwhelming.

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between- 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought)- o3-mini (web search, CoT, canvas, but no i...

Numbers for SWE-bench Verified, Aider Polyglot, cost per million output tokens, output tokens per second, and knowledge cutoff month/year: SWE Aider Cost Fast Fresh Claude 3.7 70% 65% $15 77 8/24 Gemini 2.5 64% 69% $10 200 1/25 GPT-4.1 55% 53% $8 169 6/24 DeepSeek R1 49% 57% $...

don't miss that OAI also published a prompting guide WITH RECEIPTS for GPT 4.1 specifically for those building agents... with a new recommendation for:- telling the model to be persistent (+20%)- dont self-inject/parse toolcalls (+2%)- prompted planning (+4%)- JSON BAD - use X...

I have been trying GPT-4.1 for a few hours by now through Cursor on a fairly complicated code base. For reference, my gold standard for a coding agent is Claude Sonnet 3.7 despite its tendency to diverge and lose focus.My take aways:- This is the first model from OpenAI that f...

From OpenAI's announcement:> Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better s...

Ok, I’m a bit underwhelmed. I’ve asked it a fairly technical question, about a very niche topic (Final Fantasy VII reverse engineering): https://chatgpt.com/share/68001766-92c8-8004-908f-fb185b7549...With right knowledge and web searches one can answer this question in a matte...

Interesting... I asked o3 for help writing a flake so I could install the latest Webstorm on NixOS (since the one in the package repo is several months old), and it looks like it actually spun up a NixOS VM, downloaded the Webstorm package, wrote the Flake, calculated the SHA ...

Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1]Incredible how resilient Claude models have been for best-in-coding class.[1] But by only about 1%, and inclusive of C...

I have a very basic / stupid "Turing test" which is just to write a base 62 converter in C#. I would think this exact thing would be in github somewhere (thus in the weights) but has always failed for me in the past (non-scientific / didn't try every single model).Using o4-min...

To plan a visit to a dark sky place, I used duck.ai (Duckduckgo's experimental AI chat feature) to ask five different AIs on what date the new moon will happen in August 2025.GPT-4o mini: The new moon in August 2025 will occur on August 12.Llama 3.3 70B: The new moon in August...

I'm really hoping GPT5 is a larger jump in metrics than the last several releases we've seen like Claude3.5 - Claude4 or o3-mini-high to o3-pro. Although I will preface that with the fact I've been building agents for about a year now and despite the benchmarks only showing sl...

The guys in the other thread who said that OpenAI might have quantized o3 and that's how they reduced the price might be right. This o3-pro might be the actual o3-preview from the beginning and the o3 might be just a quantized version. I wish someone benchmarks all of these mo...

The benchmarks don’t look _that_ much better than o3. Does that mean Pro models are just incrementally better than base models, or are we approaching the higher end of a sigmoid function, with performance gains leveling off?

I am still not willing to upgrade to a Pro account. I pay $20 a month for both Gemini and ChatGPT, and for what I need this is currently enough.I have dreamed of having powerful AI ever since I read Bertram Raphael's great book Mind Inside Matter around 1978, getting hooked on...

here's a nice user review we published: https://www.latent.space/p/o3-prosama's highlight[0]:> "The plan o3 gave us was plausible, reasonable; but the plan o3 Pro gave us was specific and rooted enough that it actually changed how we are thinking about our future."I kept nu...

It is frequently suggested that once one of the AI companies reaches an AGI threshold, they will take off ahead of the rest. It's interesting to note that at least so far, the trend has been the opposite: as time goes on and the models get better, the performance of the differ...

GPT-5 knowledge cutoff: Sep 30, 2024 (10 months before release).Compare that toGemini 2.5 Pro knowledge cutoff: Jan 2025 (3 months before release)Claude Opus 4.1: knowledge cutoff: Mar 2025 (4 months before release)https://platform.openai.com/docs/models/comparehttps://deepmin...

Going by the system card at: https://openai.com/index/gpt-5-system-card/> GPT‑5 is a unified system . . .OK> . . . with a smart and fast model that answers most questions, a deeper reasoning model for harder problems, and a real-time router that quickly decides which mod...

I'm not really convinced, the benchmark blunder was really strange but the demos were quite underwhelming, and it appears this was reflected by a huge market correction in the betting markets as to who will have the best AI by end of the year.What excites me now is that Gemini...

The marketing copy and the current livestream appear tautological: "it's better because it's better."Not much explanation yet why GPT-5 warrants a major version bump. As usual, the model (and potentially OpenAI as a whole) will depend on output vibe checks.

Interesting, the new model's prompt is ~half the size (10KB vs. 23KB) of the previous prompt[0][1].SWE-bench performance is similar to normal gpt-5, so it seems the main delta with `gpt-5-codex` is on code refactors (via internal refactor benchmark 33.9% -> 51.3%).As someon...

I've been a hardcore claude-4-sonnet + Cursor fan for a long time, but in the last 2 months my usage went through the roof. I started with the basic Cursor subscription, then upgraded to pro, until I hit usage limits again. Then I started using my own Claude API key but I was ...

Codex CLI IDE just works, very impressed with the quality. If you tried it a while back and didn’t like it, try it again via the vscode extension generous usage included with plus.Ditched my Claude code max sub for the ChatGPT pro $200 plan. So much faster, and not hit any lim...

From my observation of the past 2 weeks is that Claude Code is getting dramatically worse and super low usage quota's while OpenAI Codex is getting great and has a very generous usage quota in comparison.For people that have not tried it in say ~1 month, give Codex CLI a try.

It's been interesting reading this thread and seeing that others have also switched to using Codex over Claude Code. I kept running into a huge issue with Claude Code creating mock implementations and general fakery when it was overwhelmed. I spent so much time tuning my input...

I've been extremely impressed (and actually had quite a good time) with GPT-5 and Codex so far. It seems to handle long context well, does a great job researching the code, never leaves things half-done (with long tasks it may leave some steps for later, but it never does 50% ...

This should probably be merged with the other GPT-5-Codex thread at https://news.ycombinator.com/item?id=45252301 since nobody in this thread is talking about the system card addendum.

My problem _still_ with all of the codex/gpt based offerings is that they think for way too long. After using Claude 4 models through cursor max/ampcode I feel much more effective given it's speed. Ironically, Claude Code feels just as slow as codex/gpt (even with my company p...

Interesting, the new model uses a different prompt in Codex CLI that's ~half the size (10KB vs. 23KB) of the previous prompt[0][1].SWE-bench performance is similar to normal gpt-5, so it seems the main delta with `gpt-5-codex` is on code refactors (via internal refactor benchm...

I signed up to OpenAI, verified my identity, and added my credit card, bought $10 of credits.But when I installed Codex and tried to make a simple code bugfix, I got rate limited nearly immediately. As in, after 3 "steps" the agent took.Are you meant to only use Codex with the...

I managed to get GPT-5-Codex-Mini to draw me a pelican. It's not a very good one! https://static.simonwillison.net/static/2025/codex-hacking-m...For comparison, here's GPT-5-Codex (not mini) https://static.simonwillison.net/static/2025/codex-hacking-d... and full GPT-5: https:...

Since they also limited the usage of codex CLI quite a bit, this might help. I really like the gpt-5-codex model. It has the most impressive dynamic reasoning effort I've seen. Responding instantly to simple questions while thinking very long when necessary. It's also interest...

Codex isn’t even a good model.

GPT-5 and GPT-5-Codex are already not clever enough for anything interesting.

I don’t want more conversational, I want more to the point. Less telling me how great my question is, less about being friendly, instead I want more cold, hard, accurate, direct, and factual results.It’s a machine and a tool, not a person and definitely not my friend.

All the examples of "warmer" generations show that OpenAI's definition of warmer is synonymous with sycophantic, which is a surprise given all the criticism against that particular aspect of ChatGPT.I suspect this approach is a direct response to the backlash against removing 4o.

> what romanian football player won the premier league> The only Romanian football player to have won the English Premier League (as of 2025) is Florin Andone, but wait — actually, that’s incorrect; he never won the league.> ...> No Romanian footballer has ever won...

I’ve seen various older people that I’m connected with on Facebook posting screenshots of chats they’ve had with ChatGPT.It’s quite bizarre from that small sample how many of them take pride in “baiting” or “bantering” with ChatGPT and then post screenshots showing how they “g...

Seems like people here are pretty negative towards a "conversational" AI chatbot.Chatgpt has a lot of frustrations and ethical concerns, and I hate the sycophancy as much as everyone else, but I don't consider being conversational to be a bad thing.It's just preference I guess...

I've been using a lot of Claude and Codex recently.One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character...

Rest assured that we are better at training models than naming them ;D- New benchmark SOTAs with 77.9% on SWE-Bench-Verified, 79.9% on SWE-Lancer, and 58.1% on TerminalBench 2.0- Natively trained to work across many hours across multiple context windows via compaction- 30% mor...

Today I did some comparisons of GPT-5.1-Codex-Max (on high) in the Codex CLI versus Gemini 3 Pro in the Gemini CLI.- As a general observation, Gemini is less easy to work with as a collaborator. If I ask the same question to both models, Codex will answer the question. Gemini ...

OpenAI likes to time their announcements alongside major competitor announcements to suck up some of the hype. (See for instance the announcement of GPT-4o a single day before Google's IO conference)They were probably sitting on this for a while. That makes me think this is a ...

Thinking level medium: https://tools.simonwillison.net/svg-render#%3Csvg%20xmlns%3D...Thinking level xhigh: https://tools.simonwillison.net/svg-render#%20%20%3Csvg%20xm...

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine.The thing that would now make the biggest difference isn't "more intelligence", whatever that...

Is it me, or did it still get at least three placements of components (RAM and PCIe slots, plus it's DisplayPort and not HDMI) in the motherboard image[0] completely wrong? Why would they use that as a promotional image?0: https://images.ctfassets.net/kftzwdyauwt9/6lyujQxhZDnO...

I feel there is a point when all these benchmarks are meaningless. What I care about beyond decent performance is the user experience. There I have grudges with every single platform and the one thing keeping me as a paid ChatGPT subscriber is the ability to sort chats in "pro...

Looks like they've begun censoring posts at r/Codex and not allowing complaint threads so here is my honest take:- It is faster which is appreciated but not as fast as Opus 4.5- I see no changes, very little noticeable improvements over 5.1- I do not see any value in exchange ...

I've benchmarked it on the Extended NYT Connections benchmark (https://github.com/lechmazur/nyt-connections/):The high-reasoning version of GPT-5.2 improves on GPT-5.1: 69.9 → 77.9.The medium-reasoning version also improves: 62.7 → 72.1.The no-reasoning version also improves: ...

Okay results are in for GenAI Showdown with the new gpt-image 1.5 model for the editing portions of the site!https://genai-showdown.specr.net/image-editingConclusions- OpenAI has always had some of the strongest prompt understanding alongside the weakest image fidelity. This u...

I have a Nano Banana Pro blog post in the works expanding on my experiments with Nano Banana (https://news.ycombinator.com/item?id=45917875). Running a few of my test cases from that post and the upcoming blog post through this new ChatGPT Image model, this new model is better...

If this was a farm of sweatshop Photoshopers in 2010, who download all images from the internet and provide a service of combining them on your request, this would escalate pretty quickly.Question: with copyright and authorship dead wrt AI, how do I make (at least) new content...

I am very impressed a benchmark I like to run is have it create sprite maps, uv texture maps for an imagined 3d modelNoticed it captured a megaman legends vibe ....https://x.com/AgentifySH/status/2001037332770615302and here it generated a texture map from a 3d characterhttps:/...

It's really weird to see "make images from memories that aren't real" as a product pitch

If anyone from OpenAI is reading this -- a plea to not screw with the reasoning capabilities!Codex is so so good at finding bugs and little inconsistencies, it's astounding to me. Where Claude Code is good at "raw coding", Codex/GPT5.x are unbeatable in terms of careful, metho...

I was very skeptical about Codex at the beginning, but now all my coding tasks start with Codex. It's not perfect at everything, but overall it's pretty amazing. Refactoring, building something new, building something I'm not familiar with. It is still not great at debugging t...

Since they are not showing you how this model compares against the benchmarks they are showing, here is a quick view with the public numbers from Google and Anthropic. At least this gives some context: SWE-Bench (Pro / Verified) Model | Pro (%) | Verified (%) -----------------...

The GPT models, in my experience, have been much better for backend than the Claude models. They're much slower, but produce logic that is more clear, and code that is more maintainable. A pattern I use is, setup a Github issue with Claude plan mode, then have Codex execute it...

I’ve been using Codex CLI heavily after moving off Claude Code and built a containerized starter to run Codex in different modes: timers/file triggers, API calls, or interactive/single-run CLI. A few others are already using it for agentic workflows. If you want to run Codex s...

Whats interesting to me is that these gpt-5.3 and opus-4.6 are diverging philosophically and really in the same way that actual engineers and orgs have diverged philosophicallyWith Codex (5.3), the framing is an interactive collaborator: you steer it mid-execution, stay in the...

I think Anthropic rushed out the release before 10am this morning to avoid having to put in comparisons to GPT-5.3-codex!The new Opus 4.6 scores 65.4 on Terminal-Bench 2.0, up from 64.7 from GPT-5.2-codex.GPT-5.3-codex scores 77.3.

,,GPT‑5.3-Codex is the first model we classify as High capability for cybersecurity-related tasks under our Preparedness Framework , and the first we’ve directly trained to identify software vulnerabilities. While we don’t have definitive evidence it can automate cyber attacks...

Something that caught my eye from the announcement:> GPT‑5.3‑Codex is our first model that was instrumental in creating itself. The Codex team used early versions to debug its own trainingI'm happy to see the Codex team moving to this kind of dogfooding. I think this was cr...

I remember when AI labs coordinated so they didn't push major announcements on the same day to avoid cannibalizing each other. Now we have AI labs pushing major announcements within 30 minutes.

Wow, I wish we could post pictures to HN. That chip is HUGE!!!!The WSE-3 is the largest AI chip ever built, measuring 46,255 mm² and containing 4 trillion transistors. It delivers 125 petaflops of AI compute through 900,000 AI-optimized cores — 19× more transistors and 28× mor...

I love this! I use coding agents to generate web-based slide decks where “master slides” are just components, and we already have rules + assets to enforce corporate identity. With content + prompts, it’s straightforward to generate a clean, predefined presentation. What I’d r...

First thoughts using gpt-5.3-codex-spark in Codex CLI:Blazing fast but it definitely has a small model feel.It's tearing up bluey bench (my personal agent speed benchmark), which is a file system benchmark where I have the agent generate transcripts for untitled episodes of a ...

Continue to believe that Cerebras is one of the most underrated companies of our time. It's a dinner-plate sized chip. It actually works. It's actually much faster than anything else for real workloads. Amazing

My stupid pelican benchmark proves to be genuinely quite useful here, you get a visual representation of the quality difference between GPT-5.3-Codex-Spark and full GPT-5.3-Codex: https://simonwillison.net/2026/Feb/12/codex-spark/

The headline may make it seem like AI just discovered some new result in physics all on its own, but reading the post, humans started off trying to solve some problem, it got complex, GPT simplified it and found a solution with the simpler representation. It took 12 hours for ...

It's interesting to me that whenever a new breakthrough in AI use comes up, there's always a flood of people who come in to handwave away why this isn't actually a win for LLMs. Like with the novel solutions GPT 5.2 has been able to find for erdos problems - many users here (e...

"An internal scaffolded version of GPT‑5.2 then spent roughly 12 hours reasoning through the problem, coming up with the same formula and producing a formal proof of its validity."When I use GPT 5.2 Thinking Extended, it gave me the impression that it's consistent enough/has a...

AI can be an amazing productivity multiplier for people who know what they're doing.This result reminded me of the C compiler case that Anthropic posted recently. Sure, agents wrote the code for hours but there was a human there giving them directions, scoping the problem, fin...

It would be more accurate to say that humans using GPT-5.2 derived a new result in theoretical physics (or, if you're being generous, humans and GPT-5.2 together derived a new result). The title makes it sound like GPT-5.2 produced a complete or near-complete paper on its own,...

The single biggest issue for me with ChatGPT right now is how absolutely awful it sounds in every answer. "Why it matters", "the big picture", "it's not jut you", the awful emphasis, the quotations with rhetorical questions, etc.. I don't know if it's intentional so you can ea...

I'm a bit confused by this branding (never even noticed that there was a 5.2-Instant), it's not a super fast 1000tok/s Cerebras based model which they have for codex-spark, it's just 5.2 w/out the router / "non-thinking" mode?I feel like openai is going to get right back to wh...

Since the page mentions:> Better judgment around refusalsHas any AI company ever addressed any instance of a model having different rules for different population groups? I've seen many examples of people asking questions like, "make up a joke about <group>" and then ...

I kind of chuckled when I read the headline "GPT‑5.3 Instant: Smoother, more ..."LLM companies starting to sound like cigarette advertisements.

Is nobody else unsettled by the example? Strange timing to talk about calculating trajectories on long range projectiles?

The marquee feature is obviously the 1M context window, compared to the ~200k other models support with maybe an extra cost for generations beyond >200k tokens. Per the pricing page, there is no additional cost for tokens beyond 200k: https://openai.com/api/pricing/Also per...

I am running gpt-5.4 as one of my coding agents, and something interesting has happened: it's the first time I've seen an agent unfairly shift blame to a team mate:"Bob’s latest mail is actually the source of the confusion: he changed shared app/backend text to aweb/atlas. I’m...

I've only used 5.4 for 1 prompt (edit: 3@high now) so far (reasoning: extra high, took really long), and it was to analyse my codebase and write an evaluation on a topic. But I found its writing and analysis thoughtful, precise, and surprisingly clearly written, unlike 5.3-Cod...

I find it quite funny how this blog post has a big "Ask ChatGPT" box at the bottom. So you might think you could ask a question about the contents of the blog post, so you type the text "summarise this blog post". And it opens a new chat window with the link to the blog post f...

So let me get this straight, OpenAi previously had an issue with LOTS of different models snd versions being available. Then they solved this by introducing GPT-5 which was more like a router that put all these models under the hood so you only had to prompt to GPT-5, and it w...

Comments moved to https://news.ycombinator.com/item?id=47265045.

More discussion: https://news.ycombinator.com/item?id=47265005

I checked the current speed over the API, and so far I'm very impressed. Of course models are usually not as loaded on the release day, but right now:- Older GPT-5 Mini is about 55-60 tokens/s on API normally, 115-120 t/s when used with service_tier="priority" (2x cost).- GPT-...

To me, mini releases matter much more and better reflect the real progress than SOTA models.The frontier models have become so good that it's getting almost impossible to notice meaningful differences between them.Meanwhile, when a smaller / less powerful model releases a new ...

I quite like the GPT models when chatting with them (in fact, they're probably my favorites), but for agentic work I only had bad experiences with them.They're incredibly slow (via official API or openrouter), but most of all they seem not to understand the instructions that I...

Here's a grid of pelicans for the different models and reasoning levels: https://static.simonwillison.net/static/2026/gpt-5.4-pelican...

According to their benchmarks, GPT 5.4 Nano > GPT-5-mini in most areas, but I'm noticing models are getting more expensive and not actually getting cheaper?GPT 5 mini: Input $0.25 / Output $2.00GPT 5 nano: Input: $0.05 / Output $0.40GPT 5.4 mini: Input $0.75 / Output $4.50G...

Related blog post: https://blog.google/technology/ai/google-gemini-ai/ (via https://news.ycombinator.com/item?id=38544746, but we merged the threads)

Very impressive! I noticed two really notable things right off the bat:1. I asked it a question about a feature that TypeScript doesn't have[1]. GPT4 usually does not recognize that it's impossible (I've tried asking it a bunch of times, it gets it right with like 50% probabil...

For others that were confused by the Gemini versions: the main one being discussed is Gemini Ultra (which is claimed to beat GPT-4). The one available through Bard is Gemini Pro.For the differences, looking at the technical report [1] on selected benchmarks, rounded score in %...

This demo is nuts: https://youtu.be/UIZAiXYceBI?si=8ELqSinKHdlGlNpX

One observation: Sundar's comments in the main video seem like he's trying to communicate "we've been doing this ai stuff since you (other AI companies) were little babies" - to me this comes off kind of badly, like it's trying too hard to emphasize how long they've been doing...

The white paper is worth a read. The things that stand out to me are:1. They don't talk about how they get to 10M token context2. They don't talk about how they get to 10M token context3. The 10M context ability wipes out most RAG stack complexity immediately. (I imagine creat...

One interesting tidbit from the technical report:>HumanEval is an industry standard open-source evaluation benchmark (Chen et al., 2021), but we found controlling for accidental leakage on webpages and open-source code repositories to be a non-trivial task, even with conser...

Massive whoa if true from technical report"Studying the limits of Gemini 1.5 Pro's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens"https://storage.googleapis.com/deepmind-media/gemini/g...

0 trust to what they put out until I see it live. After the last "launch" video which was fundamentally a marketing edit not showing the real product, I don't trust anything coming out of Google that isn't an instantly testable input form.

Personally, I've given up on Gemini, as it seems to have been censored to the point of uselessness. I asked it yesterday [0] about C++ 20 Concepts, and it refused to give actual code because I'm under 18 (I'm 17, and AFAIK that's what the age on my Google account is set to). I...

Benchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | ...

The terms of use: https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policySomething that caught my eye in the terms:> Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma.One of the...

Hello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models.Opinions are our own and not of Google DeepMind.

I notice a few divergences to common models:- The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x;- The vocabulary size is 10x (256K vs. Mistral’s 32K);- The training token count is tripled (6T vs. Llama2's 2T)Apart from that, it uses the ...

Is there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.

Within a couple sentences of each other he says (paraphrasing), “if you test any model (Gemini, ChatGPT, Grok) you’ll see it says far left things” and “we don’t know why [Gemini] is far left leaning”.I think he’s being honest here. At least at the executive level of having no ...

Transcript:https://lifearchitect.ai/sergey/

> The Gemini art? Yeah, with the… Okay. That wasn’t really expected [...] > So, once again, we haven’t fully understood why it means “left” in many cases. And that’s not our intention.Riiiight.

Anyone except me thinks he doesn't look very healthy? Its strange he is kind of slow on the video where he enters the room. Maybe some biohacking.

Let's be clear. This isn't a left or right thing. That's just a convenient argument to avoid the real problem. It's about moral relativism.I am about as left wing as you can be and gemini is simply offensive. It didn't want to say if Hitler was worse than Elon or Trump. It did...

I upgraded my llm-gemini plugin to provide CLI access to Gemini Flash: pipx install llm # or brew install llm llm install llm-gemini --upgrade llm keys set gemini # paste API key here llm -m gemini-1.5-flash-latest 'a short poem about otters' https://github.com/simonw/llm-gemi...

Looking at MMLU and other benchmarks, this essentially means sub-second first-token latency with Llama 3 70B quality (but not GPT-4 / Opus), native multimodality, and 1M context.Not bad compared to rolling your own, but among frontier models the main competitive differentiator...

1M token context by default is the big feature here IMO, but we need better benchmarks to measure what that really means.My intuition is that as contexts get longer we start hitting the limits of how much comprehension can be embedded in a single point of vector space, and wil...

A lightweight model that you can only use in the cloud? That is amusing. These tech megacorps are really intent on owning your usage of AI. But we must not let that be the future.

One thing OpenAI is beating Google at is actually publishing pricing for its APIs (and being consistent as to what they are called.)Google has I think 10 models available (there’s more than ten model names, but several of the models have multiple aliases) through what the Goog...

https://aistudio.google.com/live is by far the coolest thing here. You can just go there and share your screen or camera and have a running live voice conversation with Gemini about anything you're looking at. As much as you want, for free.I just tried having it teach me how t...

I released a new llm-gemini plugin with support for the Gemini 2.0 Flash model, here's how to use that in the terminal: llm install -U llm-gemini llm -m gemini-2.0-flash-exp 'prompt goes here' LLM installation: https://llm.datasette.io/en/stable/setup.htmlWorth noting that the...

Big companies can be slow to pivot, and Google has been famously bad at getting people aligned and driving in one direction.But, once they do get moving in the right direction the can achieve things that smaller companies can't. Google has an insane amount of talent in this sp...

Buried in the announcement is the real gem — they’re releasing a new SDK that actually looks like it follows modern best practices. Could be a game-changer for usability.They’ve had OpenAI-compatible endpoints for a while, but it’s never been clear how serious they were about ...

Beats Gemini 1.5 Pro at all but two of the listed benchmarks. Google DeepMind is starting to get their bearings in the LLM era. These are the minds behind AlphaGo/Zero/Fold. They control their own hardware destiny with TPUs. Bullish.

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large...

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result.You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images.You...

Isn't it amazing how one company invented a universally spread format that takes structured data from an editor (except images obviously) and converts it into a completely fucked-up unstructured form that then requires expensive voodoo magic to convert back into structured data.

We are driving full speed into a xerox 2.0 moment and this time we are doing so knowingly. At least with xerox, the errors were out of place and easy to detect by a human. I wonder how many innocent people will lose their lives or be falsely incarcerated because of this. I won...

(disclaimer I am CEO of llamaindex, which includes LlamaParse)Nice article! We're actively benchmarking Gemini 2.0 right now and if the results are as good as implied by this article, heck we'll adapt and improve upon it. Our goal (and in fact the reason our parser works so we...

Gemma 3 is out! Multimodal (image + text), 128K context, supports 140+ languages, and comes in 1B, 4B, 12B, and 27B sizes with open weights & commercial use.Gemma 3 model overview: https://ai.google.dev/gemma/docs/coreHuggingface collection: https://huggingface.co/collecti...

Greetings from the Gemma team! We just got Gemma 3 out of the oven and are super excited to show it to you! Please drop any questions here and we'll answer ASAP.(Opinions our own and not of Google DeepMind.)PS we are hiring: https://boards.greenhouse.io/deepmind/jobs/6590957

Lots to be excited about here - in particular new architecture that allows subquadratic scaling of memory needs for long context; looks like 128k+ context is officially now available on a local model. The charts make it look like if you have the RAM the model is pretty good ou...

Linking to he announcement (which links to his PDF) would probably be more useful.Introducing Gemma 3: The most capable model you can run on a single GPU or TPUhttps://blog.google/technology/developers/gemma-3/

Very cool open release. Impressive that a 27b model can be as good as the much bigger state of the art models (according to their table of Chatbot Arena, tied with O1-preview and above Sonnet 3.7).But the example image shows that this model still makes dumb errors or has a poo...

It just tells me 'content not permitted'.For context, I was attempting to put a cup of hot chocolate into the hands of an anime character.

Ever since OpenAI showed (but did not release) this type of multimodal output with 4o, I have been waiting for this to be available to the general public.It seems like really combining visuals at the level of generation capability means language understanding is fully grounded...

Curious that they use "Use Gemini 2.0 Flash to tell a story and it will illustrate it with pictures, keeping the characters and settings consistent throughout", and their example is not consistent at all between two pictures.

I was really hoping that there would be more character consistency, given the fact they mention it in the blog. It also doesn't seem to reliably follow styles like "watercolor illustration" or "line and wash".

One of the biggest problems with hands off LLM writing (for long horizon stuff like novels) is that you can't really give them any details of your story because they get absolutely neurotic with it.Imagine for instance you give the LLM the profile of the love interest for your...

I've been using a math puzzle as a way to benchmark the different models. The math puzzle took me ~3 days to solve with a computer. A math major I know took about a day to solve it by hand.Gemini 2.5 is the first model I tested that was able to solve it and it one-shotted it. ...

I'm impressed by this one. I tried it on audio transcription with timestamps and speaker identification (over a 10 minute MP3) and drawing bounding boxes around creatures in a complex photograph and it did extremely well on both of those.Plus it drew me a very decent pelican r...

Tops our benchmark in an unprecedented way.https://help.kagi.com/kagi/ai/llm-benchmark.htmlHigh quality, to the point. Bit on the slow side. Indeed a very strong model.Google is back in the game big time.

Gemini 2.5 Pro set the SOTA on the aider polyglot coding leaderboard [0] with a score of 73%.This is well ahead of thinking/reasoning models. A huge jump from prior Gemini models. The first Gemini model to effectively use efficient diff-like editing formats.[0] https://aider.c...

Google making Gemini 2.5 Pro (Experimental) free was a big deal. I haven't tried the more expensive OpenAI models so I can't even compare, only to the free models I have used of theirs in the past.Gemini 2.5 Pro is so much of a step up (IME) that I've become sold on Google's m...

An often overlooked feature of the Gemini models is that they can write and execute Python code directly via their API.My llm-gemini plugin supports that: https://github.com/simonw/llm-gemini uv tool install llm llm install llm-gemini llm keys set gemini # paste key here llm -...

Gemini flash models have the least hype, but in my experience in production have the best bang for the buck and multimodal tooling.Google is silently winning the AI race.

One hidden note from Gemini 2.5 Flash when diving deep into the documentation: for image inputs, not only can the model be instructed to generated 2D bounding boxes of relevant subjects, but it can also create segmentation masks! https://ai.google.dev/gemini-api/docs/image-und...

For a non programmer like me google is becoming shockingly good. It is giving working code the first time. I was playing around with it asked it to write code to scrape some data of a website to analyse. I was expecting it to write something that would scrape the data and late...

I think gemma-3-27b-it-qat-4bit is my new favorite local model - or at least it's right up there with Mistral Small 3.1 24B.I've been trying it on an M2 64GB via both Ollama and MLX. It's very, very good, and it only uses ~22Gb (via Ollama) or ~15GB (MLX) leaving plenty of mem...

I have a few private “vibe check” questions and the 4 bit QAT 27B model got them all correctly. I’m kind of shocked at the information density locked in just 13 GB of weights. If anyone at Deepmind is reading this — Gemma 3 27B is the single most impressive open source model I...

First graph is a comparison of the "Elo Score" while using "native" BF16 precision in various models, second graph is comparing VRAM usage between native BF16 precision and their QAT models, but since this method is about doing quantization while also maintaining quality, isn'...

Indeed!! I have swapped out qwen2.5 for gemma3:27b-it-qat using Ollama for routine work on my 32G memory Mac.gemma3:27b-it-qat with open-codex, running locally, is just amazingly useful, not only for Python dev, but for Haskell and Common Lisp also.I still like Gemini 2.5 Pro ...

Gemma 3 is way way better than Llama 4. I think Meta will start to lose its position in LLM mindshare. Another weakness of Llama 4 is its model size that is too large (even though it can run fast with MoE), which greatly limits the applicable users to a small percentage of ent...

You can try it on Android right now:Download the Edge Gallery apk from github: https://github.com/google-ai-edge/gallery/releases/tag/1.0.0Download one of the .task files from huggingface: https://huggingface.co/collections/google/gemma-3n-preview-6...Import the .task file in ...

Probably a better link: https://developers.googleblog.com/en/introducing-gemma-3n/Gemma 3n is a model utilizing Per-Layer Embeddings to achieve an on-device memory footprint of a 2-4B parameter model.At the same time, it performs nearly as well as Claude 3.7 Sonnet in Chatbot ...

According to the readme here - https://huggingface.co/google/gemma-3n-E4B-it-litert-previewE4B has a score of 44.4 in the Aider polyglot dashboard. Which means its on-par with gemini-2.5-flash (not the latest preview but the version used for the bench on aider's website), gpt4...

On Hugging face I see 4B and 2B versions now -https://huggingface.co/collections/google/gemma-3n-preview-6...Gemma 3n Previewgoogle/gemma-3n-E4B-it-litert-previewgoogle/gemma-3n-E2B-it-litert-previewInteresting, hope it comes on LMStudio as MLX or GGUF. Sparse and or MoE model...

It seems to work quite well on my phone. One funny side effect I've found is that it's much easier to bypass the censorship in these smaller models than in the larger ones, and with the complexity of the E4B variant I wouldn't have expected the "roleplay as my father who is ex...

Hi all, I built these models with a great team. They're available for download across the open model ecosystem so give them a try! I built these models with a great team and am thrilled to get them out to you.From our side we designed these models to be strong for their size o...

My lovely interaction with the 270M-F16 model:> what's second tallest mountain on earth?The second tallest mountain on Earth is Mount Everest.> what's the tallest mountain on earth?The tallest mountain on Earth is Mount Everest.> whats the second tallest mountain?The ...

I've got a very real world use case I use DistilBERT for - learning how to label wordpress articles. It is one of those things where it's kind of valuable (tagging) but not enough to spend loads on compute for it.The great thing is I have enough data (100k+) to fine-tune and r...

This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything.Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: ...

Apple should be doing this. Unless their plan is to replace their search deal with an AI deal -- it's just crazy to me how absent Apple is. Tim Cook said, "it's ours to take" but they really seem to be grasping at the wind right now. Go Google!

This is the gpt 4 moment for image editing models. Nano banana aka gemini 2.5 flash is insanely good. It made a 171 elo point jump in lmarena!Just search nano banana on Twitter to see the crazy results. An example. https://x.com/D_studioproject/status/1958019251178267111

I've updated the GenAI Image comparison site (which focuses heavily on strict text-to-image prompt adherence) to reflect the new Google Gemini 2.5 Flash model (aka nano-banana).https://genai-showdown.specr.netThis model gets 8 of the 12 prompts correct and easily comes within ...

Unfortunately, it suffers from the same safetyism than other many releases. Half of the prompts get rejected. How can you have character consistency if the model is forbidden from editing any human. And most of my photo editing involves humans, so basically this is just a usel...

There is one thing Gemini 2.5 Flash Image can do that no other edit model can do: incorporate multiple images simultaneously without shenanigans due to its multimodality, e.g. for Flux Kontext, if you want to "put the person in the first image into the second image", you have ...

I digitised our family photos but a lot of them were damaged (shifted colours, spills, fingerprints on film, spots) that are difficult to correct for so many images. I've been waiting for image gen to catch up enough to be able to repair them all in bulk without changing detai...

This really captures something I've been experiencing with Gemini lately. The models are genuinely capable when they work properly, but there's this persistent truncation issue that makes them unreliable in practice.I've been running into it consistently, responses that just s...

I added support to these models to my llm-gemini plugin, so you can run them like this (using uvx so no need to install anything first): export LLM_GEMINI_KEY='...' uvx --isolated --with llm-gemini llm -m gemini-flash-lite-latest 'An epic poem about frogs at war with ducks' Re...

Serious question: If it's an improved 2.5 model, why don't they call it version 2.6? Seems annoying to have to remember if you're using the old 2.5 or the new 2.5. Kind of like when Apple released the third-gen iPad many years ago and simply called it the "new iPad" without a ...

Google seems to be the main foundation model provider that's really focusing on the latency/TPS/cost dimensions. Anthropic/OpenAI are really making strides in model intelligence, but underneath some critical threshold of performance, the really long thinking times make workflo...

Non-AI Summary:Both models have improved intelligence on Artificial Analysis index with lower end-to-end response time. Also 24% to 50% improved output token efficiency (resulting in lower cost).Gemini 2.5 Flash-Lite improvements include better instruction following, reduced v...

Benchmarks from page 4 of the model card: | Benchmark | 3 Pro | 2.5 Pro | Sonnet 4.5 | GPT-5.1 | |-----------------------|-----------|---------|------------|-----------| | Humanity's Last Exam | 37.5% | 21.6% | 13.7% | 26.5% | | ARC-AGI-2 | 31.1% | 4.9% | 13.6% | 17.6% | | GPQ...

It is interesting that the Gemini 3 beats every other model on these benchmarks, mostly by a wide margin, but not on SWE Bench. Sonnet is still king here and all three look to be basically on the same level. Kind of wild to see them hit such a wall when it comes to agentic coding

One benchmark I would really like to see: instruction adherence.For example, the frontier models of early-to-mid 2024 could reliably follow what seemed to be 20-30 instructions. As you gave more instructions than that in your prompt, the LLMs started missing some and your outp...

There needs to be a sycophancy benchmark in these comparisons. More baseless praise and false agreement = lower score.

Curiously, this website seems to be blocked in Spain for whatever reason, and the website's certificate is served by `allot.com/[email protected]` which obviously fails...Anyone happen to know why? Is this website by any change sharing information on safe medical abo...

Out of curiosity, I gave it the latest project euler problem published on 11/16/2025, very likely out of the training dataGemini thought for 5m10s before giving me a python snippet that produced the correct answer. The leaderboard says that the 3 fastest human to solve this pr...

This is wild. I gave it some legacy XML describing a formula-driven calculator app, and it produced a working web app in under a minute:https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...I spent years building a compiler that takes our custom XML format and generat...

Well, I tried a variation of a prompt I was messing with in Flash 2.5 the other day in a thread about AI-coded analog clock faces. Gemini Pro 3 Preview gave me a result far beyond what I saw with Flash 2.5, and got it right in a single shot.[0] I can't say I'm not impressed, e...

Static Pelican is boring. First attempt:Generate SVG animation of following:1 - There is High fantasy mage tower with a top window a dome2 - Green goblin come in front of tower with a torch3 - Grumpy old mage with beard appear in a tower window in high purple hat4 - Mage sends...

I'm sure this is a very impressive model, but gemini-3-pro-preview is failing spectacularly at my fairly basic python benchmark. In fact, gemini-2.5-pro gets a lot closer (but is still wrong).For reference: gpt-5.1-thinking passes, gpt-5.1-instant fails, gpt-5-thinking fails, ...

WellIt is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs.In fact GPT5 wrote an edge detection...

I do some electrical drafting work for construction and throw basic tasks at LLMs.I gave it a shitty harness and it almost 1 shotted laying out outlets in a room based on a shitty pdf. I think if I gave it better control it could do a huge portion of my coworkers jobs very soon

These OCR improvements will almost certainly be brought to google books, which is great. Long term it can enable compressing all non-digital rare books into a manageable size that can be stored for less than $5,000.[0] It would also be great for archive.org to move to this fro...

Interesting "ScreenSpot Pro" results: 72.7% Gemini 3 Pro 11.4% Gemini 2.5 Pro 49.9% Claude Opus 4.5 3.50% GPT-5.1 ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usehttps://arxiv.org/abs/2504.07981

In case the article author sees this, the "HTML transcription" link is broken - it goes to https://aistudio-preprod.corp.google.com/prompts/1GUEWbLIlpX... which is a Google-employee-only URL.

Don’t let the “flash” name fool you, this is an amazing model.I have been playing with it for the past few weeks, it’s genuinely my new favorite; it’s so fast and it has such a vast world knowledge that it’s more performant than Claude Opus 4.5 or GPT 5.2 extra high, for a fra...

This is awesome. No preview release either, which is great to production.They are pushing the prices higher with each release though: API pricing is up to $0.5/M for input and $3/M for outputFor comparison:Gemini 3.0 Flash: $0.50/M for input and $3.00/M for outputGemini 2.5 Fl...

Feels like Google is really pulling ahead of the pack here. A model that is cheap, fast and good, combined with Android and gsuite integration seems like such powerful combination.Presumably a big motivation for them is to be first to get something good and cheap enough they c...

These flash models keep getting more expensive with every release.Is there an OSS model that's better than 2.0 flash with similar pricing, speed and a 1m context window?Edit: this is not the typical flash model, it's actually an insane value if the benchmarks match real world ...

This model is breaking records on my benchmark of choice, which is 'the fraction of Hacker News comments that are positive.' Even people who avoid Google products on principle are impressed. Hardly anyone is arguing that ChatGPT is better in any respect (except brand recogniti...

Arc-AGI-2: 84.6% (vs 68.8% for Opus 4.6)Wow.https://blog.google/innovation-and-ai/models-and-research/ge...

Is it me or is the rate of model release is accelerating to an absurd degree? Today we have Gemini 3 Deep Think and GPT 5.3 Codex Spark. Yesterday we had GLM5 and MiniMax M2.5. Five days before that we had Opus 4.6 and GPT 5.3. Then maybe two weeks I think before that we had K...

I’ve been using Gemini 3 Pro on a historical document archiving project for an old club. One of the guys had been working on scanning old handwritten minutes books written in German that were challenging to read (1885 through 1974). Anyways, I was getting decent results on a f...

Google is absolutely running away with it. The greatest trick they ever pulled was letting people think they were behind.

Here is the methodologies for all the benchmarks: https://storage.googleapis.com/deepmind-media/gemini/gemini_...The arc-agi-2 score (84.6%) is from the semi-private eval set. If gemini-3-deepthink gets above 85% on the private eval set, it will be considered "solved">Submi...

I hope this works better than 3.0 ProI'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development.It's stunningly good at reasoning, design, and generatin...

People underrate Google's cost effectiveness so much. Half price of Opus. HALF.Think about ANY other product and what you'd expect from the competition thats half the price. Yet people here act like Gemini is dead weight____Update:3.1 was 40% of the cost to run AA index vs Opu...

If it’s any consolation, it was able to one-shot a UI & data sync race condition that even Opus 4.6 struggled to fix (across 3 attempts).So far I like how it’s less verbose than its predecessor. Seems to get to the point quicker too.While it gives me hope, I am going to pl...

Price is unchanged from Gemini 3 Pro: $2/M input, $12/M output. https://ai.google.dev/gemini-api/docs/pricingKnowledge cutoff is unchanged at Jan 2025. Gemini 3.1 Pro supports "medium" thinking where Gemini 3 did not: https://ai.google.dev/gemini-api/docs/gemini-3Compare to Op...

These models are so powerful.It's totally possible to build entire software products in the fraction of the time it took before.But, reading the comments here, the behaviors from one version to another point version (not major version mind you) seem very divergent.It feels lik...

Lots of comments about the price change, but Artifical Analysis reports that 3.1 Flash-Lite (reasoning) used fewer than half of the tokens of 2.5 Flash-Lite (reasoning).This will likely bring the cost below 2.5 flash-lite for many tasks (depends on the ratio of input to output...

Unfortunate, significant price increase for a 'lite' model: $0.25 IN / $1.50 OUT vs. Gemini 2.5 Flash-Lite $0.10 IN / $0.40 OUT.

For the last 2 years, startup wisdom has been that models will continue to get cheaper and better. Claude first, and now Gemini has shown that it's not the case.We priced an enterprise contract using Flash 1.5 pricing last summer, and today that contract would be unit economic...

That's a 150% increase in the input costs and 275% increase on output costs over the same sized previous generation (2.5-flash-lite) model

You can test Gemini 3.1 Lite transcription capabilities in https://ottex.ai — the only dictation app supporting Gemini models with native audio input.We benchmarked it for real-life voice-to-text use cases: <10s 10-30s 30s-1m 1-2m 2-3m Flash 2548 2732 3177 4583 5961 Flash L...

Thinking / reasoning + multimodal + tool calling.We made some quants at https://huggingface.co/collections/unsloth/gemma-4 for folks to run them - they work really well!Guide for those interested: https://unsloth.ai/docs/models/gemma-4Also note to use temperature = 1.0, top_p ...

I ran these in LM Studio and got unrecognizable pelicans out of the 2B and 4B models and an outstanding pelican out of the 26b-a4b model - I think the best I've seen from a model that runs on my laptop.https://simonwillison.net/2026/Apr/2/gemma-4/The gemma-4-31b model is compl...

Comparison of Gemma 4 vs. Qwen 3.5 benchmarks, consolidated from their respective Hugging Face model cards: | Model | MMLUP | GPQA | LCB | ELO | TAU2 | MMMLU | HLE-n | HLE-t | |----------------|-------|-------|-------|------|-------|-------|-------|-------| | G4 31B | 85.2% | ...

Prompt:> what is the Unix timestamp for this: 2026-04-01T16:00:00ZQwen 3.5-27b-dwq> Thought for 8 minutes 34 seconds. 7074 tokens.> The Unix timestamp for 2026-04-01T16:00:00Z is:> 1775059200 (my comment: Wednesday, 1 April 2026 at 16:00:00)Gemma-4-26b-a4b> Thou...

Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can

Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING.Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.Aider 0.75.0 is out with support for 3.7 Sonnet [1].Thinking supp...

Hi everyone! Boris from the Claude Code team here. @eschluntz, @catherinewu, @wolffiex, @bdr and I will be around for the next hour or so and we'll do our best to answer your questions about the product.

Kagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7.https://help.kagi.com/kagi/ai/llm-benchmark.htmlAppears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, abo...