The whole “it’s just autocomplete” is just a comforting mantra. A sufficiently advanced autocomplete is indistinguishable from intelligence. LLMs provably have a world model, just like humans do. They build that model by experiencing the universe via the medium of human-generated text, which is much more limited than human sensory input, but has allowed for some very surprising behavior already.

We’re not seeing diminishing returns yet, and in fact we’re going to see some interesting stuff happen as we start hooking up sensors and cameras as direct input, instead of these models building their world model indirectly through purely text. Let’s see what happens in 5 years or so before saying that there’s any diminishing returns.

Gary Marcus should be disregarded because he’s emotionally invested in The Bitter Lesson being wrong. He really wants LLMs to not be as good as they already are. He’ll find some interesting research about “here’s a limitation that we found” and turn that into “LLMS BTFO IT’S SO OVER”.

The research is interesting for helping improve LLMs, but that’s the extent of it. I would not be worried about the limitations the paper found for a number of reasons:

• There doesn’t seem to be any reason to believe that there’s a ceiling on scaling up
• LLM’s reasoning abilities improve with scale (notice that the example they use for kiwis they included the answers from o1-mini and llama3-8B, which are much smaller models with much more limited capabilities. GPT-4o got the problem correct when I tested it, without any special prompting techniques or anything)
• Techniques such as RAG and Chain of Thought help immensely on many problems
• Basic prompting techniques help, like “Make sure you evaluate the question to ignore extraneous information, and make sure it’s not a trick question”
• LLMs are smart enough to use tools. They can go “Hey, this looks like a math problem, I’ll use a calculator”, just like a human would
• There’s a lot of research happening very quickly here. For example, LLMs improve at math when you use a different tokenization method, because it changes how the model “sees” the problem

Until we hit a wall and really can’t find a way around it for several years, this sort of research falls into the “huh, interesting” territory for anybody that isn’t a researcher.

Gary Marcus is an AI crank and should be disregarded

Using it in pipes looks cool. IMO the usage in writing git commit messages is actually not useful. Almost always you should be writing the why, not the what. Same thing for comments. Unless the code has a good reason to be written inscrutably e.g. for performance, write simple code and comment why you’re doing something as necessary. Which is not to say “the code comments itself”, but the “what” comments should be higher level at a function or file level

The collect’s in the middle aren’t necessary, neither is splitting by ": ". Here’s a simpler version

fn main() {
    let text = "seeds: 79 14 55 13\nwhatever";
    let seeds: Vec<_> = text
        .lines()
        .next()
        .unwrap()
        .split_whitespace()
        .skip(1)
        .map(|x| x.parse::<u32>().unwrap())
        .collect();
    println!("seeds: {:?}", seeds);
}

It is simpler to bang out a [int(num) for num in text.splitlines()[0].split(' ')[1:]] in Python, but that just shows the happy path with no error handling, and does a bunch of allocations that the Rust version doesn’t. You can also get slightly fancier in the Rust version by collecting into a Result for more succinct error handling if you’d like.

EDIT: Here’s also a version using anyhow for error handling, and the aforementioned Result collecting:

use anyhow::{anyhow, Result};

fn main() -> Result<()> {
    let text = "seeds: 79 14 55 13\nwhatever";
    let seeds: Vec<u32> = text
        .lines()
        .next()
        .ok_or(anyhow!("No first line!"))?
        .split_whitespace()
        .skip(1)
        .map(str::parse)
        .collect::<Result<_, _>>()?;
    println!("seeds: {:?}", seeds);
    Ok(())
}

You probably wouldn’t be committing this, unless you’re backing up a heavily WIP branch. The issue is that if you’re developing locally and need to make a temporary change, you might comment something out, which then requires commenting another now-unused variable, which then requires commenting out yet another variable, and so on. Go isn’t helping you here, it’s wasting your time for no good reason. Just emit a warning and allow CI to be configured to reject warnings.

This is the best explanation I’ve ever seen of monads: https://www.adit.io/posts/2013-04-17-functors,_applicatives,_and_monads_in_pictures.html

For some reason, you’ll find a lot of really bad explanations of monads, like “programmable semi-colons”. Ignore those, and check out the link.

It’s not a half-arsed copy, it’s borrowing a limited subset of HKT for a language with very different goals. Haskell can afford a lot of luxuries that Rust can’t.

That’s 👏 what 👏 CI 👏 is 👏 for

Warn in dev, enforce stuff like this in CI and block PRs that don’t pass. Go is just being silly here, which is not surprising given that Rob Pike said

Syntax highlighting is juvenile. When I was a child, I was taught arithmetic using colored rods. I grew up and today I use monochromatic numerals.

The Go developers need to get over themselves.

Thanks! That’s the sort of thing I’m looking for

Are there any good services like bandcamp, but for video? Even if it doesn’t have blockbuster/popular movies I’d be interested. I spend a lot of money on bandcamp because it’s easy and simple: I give them money and in return get bits that I do what I want with.