Rendered at 21:48:01 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Almondsetat 1 days ago [-]
I think Casey is currently the most informed person to make a series of books or articles summarizing the history of SW Engineering, all the lessons learned and forgotten, and all the good stuff that was published and still hasn't gained traction in the practice
layla5alive 1 days ago [-]
You think there aren't other people equally or more qualified? I mean Casey may be the most visible person with those qualifications - but lack of visibility doesn't imply lack of competence, plenty of brilliant people work in the background..
Almondsetat 1 days ago [-]
Feel free to point to anyone doing these deep dives that specifically tackle the lost knowledge of the early decades of computer science and the ideas that have not yet come to fruition despite being extremely old
bob1029 1 days ago [-]
> Feel free to point to anyone doing these deep dives
Most of this kind of content comes off as relevant/topical but mindless entertainment. This is not a "deep dive" of anything that I could practically apply in my daily work. It feels good to think this content might add value and then to subsequently consume it, but it's effectively junk food.
I used to spend a lot of time watching crap like lex and primagen before it became obvious that these kinds of information channels are not meaningfully enriching despite sounding like they should be to a casual observer.
bsoles 1 days ago [-]
As much as I like to watch Casey, Prime, etc. on YouTube, they are not going to teach anybody anything. They are just entertainment for software people with a certain viewpoint.
ntstatusquo 1 days ago [-]
My one data point is that Casey’s handmade hero series, and his immediate mode gui video from way back in 2005, are what introduced me and some friends to an entirely new way to build graphical user interfaces, and I do feel a great appreciation for that. Guys like Ryan Fleury (of RadDbg) and Vjekoslav Krajacic of Filepilot similarly credit Casey with their “radicalization” :)
To your point though, it was less about specific technical knowledge conveyed by these videos and more about him evangelizing a high level approach that many folks otherwise wouldn’t have considered
pitched 1 days ago [-]
I want more things like that old hand made hero to exist in the world, but, from an interview with Casey himself, it doesn’t pay well enough to justify. Someone with the chops to put that together is in a position to be doing something much better incentivized.
ranger_danger 18 hours ago [-]
That series also produced this gem:
> Open source is the very worst thing currently going on because it is so incredibly exploitative, it's far more exploitative than any actual company is of the workers who work at the company.
> Even the people who are getting paid in open source are getting massively underpaid to do it compared to how much the people who are using their code are making, it's nothing compared to the power that is accreted by the people who have co-opted that work thanks to the open source model. And then mark zuckerberg gets to define how the internet works despite having paid for almost none of the software that his company actually needed to make that work.
> It's like feudalism or serfdom, these people did the work and got nothing for it. It's like you took the worst aspects of capitalism for workers and the worst aspects of socialism for workers and put them together, that's open source. You get no power and you get no money.
> It's exploitative whether the people chose to be exploited, just because someone chooses to let you exploit them does not mean that you didn't exploit them. And for the record that's how most exploitation works; convincing people to do something that turns out to be very bad for them and very good for you, and that's exactly what the open source movement has turned out to be.
> I really don't see the "we post stuff on github under a gpl2 or lgpl or apache or mit license", all that is to me now is just exploitation. You can say that there's solutions but until someone demonstrates that those solutions work, it's the standard "real communism has never been tried" argument. AGPL is the only thing that I've seen so far that's an attempt to fix these fundamentally unfair compensation practices.
LtdJorge 1 days ago [-]
You don’t know Computer Enhance then
megagpt3 1 days ago [-]
Do you?
Almondsetat 1 days ago [-]
You are making many assumptions. First of all, he is doing the deep dives by researching all those articles, and of course the resulting presentation will be very superficial. The point is exactly to publish a book to allow the reader to partake in the deep dive.
Secondly, nobody said anything about applying that stuff to your daily work.
jddndnfnfn 1 days ago [-]
[flagged]
Almondsetat 1 days ago [-]
Thank you for creating an account just for making stuff up nobody has said or even hinted at
1 days ago [-]
chamomeal 1 days ago [-]
agree regarding primagen but I don’t think that’s a good comparison at all
bob1029 1 days ago [-]
Casey is a frequent guest on his shows. It's hard to not conflate the two.
josephg 22 hours ago [-]
Prime is an entertainer first and foremost. He talks with confidence, even when he doesn’t know what he’s talking about - which is very often.
Casey is far more measured, careful and accurate. Casey feels like an engineering educator first, who happens to make YouTube videos. I don’t always agree with Casey’s technical opinions, but he seems like a thoughtful guy who thinks things through.
HexDecOctBin 1 days ago [-]
> that I could practically apply in my daily work
When has History of any kind ever held practical value?
mariusor 1 days ago [-]
Is this.. is this a real question?
ykonstant 1 days ago [-]
I hope the above user is being sarcastic.
mariusor 15 hours ago [-]
Based on the answers below, I think that perhaps not. :(
ranger_danger 18 hours ago [-]
When people use it to prevent repeating the same mistakes.
HexDecOctBin 18 hours ago [-]
History wouldn't keep repeating if they did so.
mariusor 15 hours ago [-]
Do you know the meme with the airplane with holes in its fuselage? Think of that, please.
agumonkey 1 days ago [-]
Yeah I was a sense that some people would be more into serious technical explorations and not just distanced review of the past (even though it can bring some knowledge).
bena 1 days ago [-]
I watch primeagen mostly for awareness of general tech news items.
Jach 20 hours ago [-]
Pretty much any book or long paper that decides to take a historical turn is going to be at least as good if not better than these talks. There are a lot of historical resources out there that go into different parts of "computing history" more generally. I don't know what would meet your criteria for computer science or software engineering more narrowly. To me they are all very interconnected. I couldn't say whether it'd be more important to have a nice presentation that includes Babbage and his machines vs. a neat presentation that covers Codd's work going directly into Chamberlin and Boyce's work on System R. A historical presentation has to decide what it includes and excludes. In recent times I've enjoyed Fabien Sanglard and his work on the histories of Wolfenstein 3D, Doom, the hardware environments behind them (development and deployment), and the CP-System. Lots of "lost" technical knowledge in them. Of course the earlier and higher level Masters of Doom book contains a lot of neat history as well, though less technical. The HOPL (History of Programming Language) papers are always insightful and contain a lot more "plain" computer science, though of course with a focus on programming language things. Richard Gabriel could write a lot more of history than he already has; here's one of his papers that (among other things) details a history of mixins in object oriented languages, and how they were "rediscovered" poorly in a way that obscured their actual historical nature, demonstrating an idea of incommensurability. https://dreamsongs.com/Files/Incommensurability.pdf
I should also add that even interviews are valuable. Coders at Work is fantastic at that and drawing out technical opinions and reasoning; notably the subjects interviewed don't all agree and have sometimes drastically different work styles and preferences. (And also adding from another comment, Turing Award Lectures are great history, and Uncle Bob even has a history book out now on influential programmers.)
bckr 1 days ago [-]
Who is also a working engineer (and programmer of the highest caliber) who is in touch with the zeitgeist of engineering.
jancsika 1 days ago [-]
Even without knowing your specific criteria, #2 on your list must be Alan Kay.
But I'm curious who you think #3 is.
Barrin92 1 days ago [-]
>Feel free to point to anyone doing these deep dives that specifically tackle the lost knowledge of the early decades of computer science
I skimmed through the video because I don't have two hours but there's no lost knowledge in it, he's going over well known papers from Dijkstra, Hoare or Knuth. The paper he bases most of the talk around Knuth's Structured Programming with go to Statements is his most often cited work, there's a very high chance you'll encounter that early in any undergrad course. So the answer to your question is, pretty much at any local uni with a CS course.
Kranar 23 hours ago [-]
The well known papers are not what the talk is about though. The talk is titled "The root of the root..." specifically to go over the specific and incredibly/almost impossible to find citations those well known papers used and built upon to see what historical insights that might have been lost went into how those papers are now interpreted (or as the video argues, misinterpreted).
znpy 1 days ago [-]
Knuth could do them, in theory. The guy was literally there when stuff was happening.
cratermoon 1 days ago [-]
Kevlin Henney
my-next-account 1 days ago [-]
Me, I do them.
pitched 1 days ago [-]
Where do you publish?
my-next-account 18 hours ago [-]
I don't. I do them for my own satisfaction, not yours.
spawrks 1 days ago [-]
I feel like the comments that are giving you pushback haven't actually listened to Casey before. I agree with you, while opinionated his perspective is very pragmatic and misunderstood. He's done such a great job giving his experience out to the broader world too.
DanielHB 10 hours ago [-]
It is kinda funny that some world-renowed people who invented genius algorithms like quicksort and graph shortest-path were mulling over "hey would be nice if we kept related data together in a struct" and some people were raging over it saying "why you want to change how I work?"
Casey points out that many of the lessons they came to were so obvious that it just became how it is done that no one even remembers it was done in any other way. In the talk the laments it makes it really hard to track down who originally came up with these ideas.
nextaccountic 23 hours ago [-]
The trouble is, his knowledge covers just a slice of it. He won't talk about stuff like functional programming for instance
josephg 22 hours ago [-]
Yeah. Casey is just some guy who’s good at software and YouTube. I’ve worked with dozens of people at his level technically. But very few with his interest and skill at making videos.
It’s a pity there’s not more people like him. I’ve met some incredibly smart people in the FP world. I wish more of them made approachable YouTube videos explaining what they know. I recently learned about “Safe Haskell” which, if adapted to rust and other languages, could solve a lot of the security problems with npm and cargo. But nobody knows about it! We need FP Casey.
nextaccountic 12 hours ago [-]
Safe Haskell is just a worst version of Rust's unsafe. What it actually does is equivalent to Rust's #![forbid(unsafe_code)] which immediately lead to a question: so Haskell has unsafe, just like Rust? And of course it does. Any practical systems language has unsafe in one form or another, not only for FFI but also for performance, Rust is just honest about it.
There's two reasons Safe Haskell is substantially worse than what Rust does. Safe Haskell works by annotating code that is safe, but that's entirely backwards. We need to annotate unsafe code, and explain in plain English exactly why it is okay in that particular instance (of course it would be better to explain in code, like, give a formal proof that is checked by the compiler, but then it isn't unsafe anymore!). Safe Haskell answer for that is to annotate good unsafe code as trustworthy, but that doesn't work because it doesn't goes in detail on why the code is trustworthy (to do so you really need to go into the details, you can't handwave it). It's the // SAFETY comments that are at the heart of Rust's unsafe, carefully explaining safety invariants that must be kept (specially important if we are modifying code), not unsafe { } blocks.
The second reason is much simpler. It's optional, and approximately nobody uses Safe Haskell or cares about it. If people used it we would have something to improve upon. So Rust achievement here is mostly sociological, it's a community of programmers that care about safety. Which is good given that a Rust program typically have much more unsafe code than a Haskell program.
The thing I really want is a language or environment with no implicit access rights. So, if I call add(a, b) then the add function doesn’t have implicit access to the filesystem, network or global variables in other parts of the program. If you want to give a function access to a subdirectory, you should pass a handle to that subdirectory as an argument and use openat() or equivalent.
This would guarantee - at a language level - that leftpad or log4j can’t root my computer.
Safe rust doesn’t give this guarantee. Safe code can still make arbitrary syscalls. Safe rust can convert a path string to a File. Or open arbitrary network sockets. Rust also doesn't have a way to import a crate but forbid the use of any unsafe blocks.
I want to be able to use 3rd party code from cargo without getting hacked. Right now rust does not keep me safe from these supply chain attacks.
I don’t know enough about safe Haskell to know how close it tacks to this. But that’s what I want.
tome 5 hours ago [-]
I think nextaccountic is not technically fully correct, but he/she is at least correct in the most important part: Safe Haskell is not really practical.
But if you don't want an ironclad guarantee and instead you're content with making wrong code obviously wrong even if it's not formally verified, then I recommend Haskell with a capability system (what the Haskell world calls an "effect system"). As far is I'm concerned there are two practical choices in 2026, Bluefin (mine) and effectful (one of Bluefin's inspirations)
This is my objection as well. His talk last year on OOP was heavily focused purely on the style of OOP that led to C++'s version. It completely ignored everything else going on outside that historical thread. As history it was interesting but doesn't inform much outside of a C++ view of the world and it missed so much relevant history about OOP more generally.
While I suspect he could do research, he also just isn't personally experienced in large tracts of SWE culture as others. Others have worked a far larger variety of jobs in the software industry and thus tend to have a broader idea of the cultures and how they've evolved. In a recent clip I saw of him and others discussing how BS are various phrases engineers say (the root of all evil phrase being one of them), he hadn't heard of the YAGNI acronym/idea before. That's no fault on him, programmers live in different worlds. But regardless of one's feelings about such ideas, or whether they've heard of them or not, a history of SWE would have to cover them and the schools of thought that originated them. And this is just one minor thing in living memory and recent careers, there is so much from before he was born that has to be researched and if possible also talked about with the older practitioners who are still around and remember things. If you don't already have some awareness of things that should be covered in a history, whether from personal experience or just general curiosity about history that has led you to see such things before somewhere, the only hope is further research uncovering them, and I don't trust his researching that much.
Reading The Psychology of Computer Programming from the 70s was such an interesting read for me from a historical perspective as well as being able to see how some ideas they were thinking about back then came to fruition in some way or another. (e.g. Stack Overflow.) It's just one small part of history. A "complete" history as the top comment suggests would be better as a multi-author endeavor, would probably need many books, and that's if it could be done at all. In the meantime at least one can always go read primary sources and develop an idea of the history on one's own.
josephg 22 hours ago [-]
If you know this stuff, please make videos / talks yourself going through it. Casey doesn’t have the background for a lot of this. He’s doing great as a C++ dev but as you say, he’s just one guy with one perspective.
Jach 20 hours ago [-]
I really think videos and talks are the least effective ways to get these things out there. I'd rather just point people to references I've read myself about specific topics rather than try to create a structured synthesis or presentation. I tend to think that's sort of best anyway for a curious individual: go read primary sources about some topic, and if interested further, citations or other related works. Every Turing Award Lecture transcription I've read has been insightful. I just recalled that Uncle Bob was writing a book about influential programmers, he was calling for suggestions on names a few years ago: https://x.com/unclebobmartin/status/1714972421269119190 One could just read through those names and the names of those the comments added (I had several more) and the work they are most known for, and by the time one is done with that, one will have a very nice historical picture of computer science up to the 70s. Nowhere near complete, of course. It seems he did publish his book at the end of 2024 (https://www.amazon.com/We-Programmers-Chronicle-Coders-Rober...), I'll have to check it out at some point. (Edit: and I'm reminded of another funny thing. Early in the book Clean Code, he asked some other programmers what they thought was meant by the idea. I remember penciling in some objections in the margins that he should have asked x,y,z other people who I would rather have read from than some of his sample.)
It's just absurd to me to point at Casey, of all people, as the most informed person who would be best positioned to publish something containing all of the history of anything broad.
kloop 1 days ago [-]
I'd put Kevlin Henney up there with him. His talk on 1968 alone is pretty amazing
PeterStuer 1 days ago [-]
He's entertaining and well spoken. Love it. Not sure the book medium would be the best avenue. My own sweet spot is presentations, live interviews and meetings. I can't write for sh*t, even if I tried. I know.
CogDisco 18 hours ago [-]
Possibly good at starting such a series, then kinda fizzle out.
JamesSwift 1 days ago [-]
Ehh I love Casey and have learned a ton by watching how he thinks about things in his handmade hero series, but he is fairly narrow minded in his views of dev. Not that theres anything wrong with that, for the kinds of dev he does his approach is very good. But its not generalizable.
Almondsetat 1 days ago [-]
How is this relevant? I didn't say he should make a book about his teachings about SWE, I said he could compile a comprehensive review of the foundational literature of compsci
shimman 1 days ago [-]
Well the comment was about the history of SWE, if you want a comprehensive history you don't focus on the lived memory of one single individual.
Almondsetat 1 days ago [-]
Where have I said I want his lived memories?
isbvhodnvemrwvn 16 hours ago [-]
When you immediately dismissed his narrow focus.
OtomotO 1 days ago [-]
Absolutely.
Like I agree with most everything he says and I like to optimise my own software, but for my day to day jobs and contracts it's simply not feasible.
That level of performance and rigor is not what is demanded nor paid or appreciated.
E.g. in Enterprise circles it's still OOP from top to bottom.
Mixed with a tad more functional style due to varying adoption of that paradigm in the languages used by enterprise.
But low level optimizations are not interesting to these customers at all, even though they could help them with some problems.
hirvi74 1 days ago [-]
> not what is demanded nor paid or appreciated.
I suppose it is a matter of what one works on, but that has been consistent in my career.
I have seen many be rewarded for choosing what I would deem intentional, gross negligence. But hey, those devs get work finished faster, and that's all that matters to the non-technical folks.
yoyohello13 1 days ago [-]
It’s not a coincidence that Casey and other high profile performance minded people are in game dev. Thats a field where performance is important (or at least a consideration). In web dev or regular enterprise app dev it’s simply not a consideration that needs to be made most of the time.
FacelessJim 1 days ago [-]
Terrific presentation. But I have a comment:
His dismissal of the argument Knuth makes regarding the hot loops could have been explored a bit better. I found it weird he didn’t mention the difference of types of programs of then vs now. Even today, in scientific code it is still absolutely the case a lot of the time that a huge chunk of the runtime comes from a single very very hot loop. It might be hidden in a library, but it’s there.
Instead he focuses only on “program size”.
Knuth samples where very small FORTRAN programs (compared to today’s standards). Today’s program are bigger but the fundamental number crunching primitive of “let’s compute stuff in a loop” remains. It’s just buried under a pile of extra cruft (data loading, parallelism, dispatching etc).
Now we just deal with a lot more programs that are of a whole different class compared to what they where doing with computers in the 70s. We have much more I/O involved. And hot loops don’t like being I/O bound.
collinstevens 1 days ago [-]
> I found it weird he didn’t mention the difference of types of programs of then vs now.
iirc, in the talk casey in fact does goes on about how he tried to find examples, but couldn't. in the q&a, he was also asked about this further.
Realizing I didn't quite answer your question about types of programs then and now. In the Q&A, they do discuss what types of (Fortran only) programs were measured in the study, and there is some speculation, but unfortunately the documentary record wasn't particularly solid.
Pannoniae 1 days ago [-]
Being I/O bound is usually a result of bad engineering practices though.
If you're I/O bound, that either means the problem doesn't require much computation - which is possible but fairly rare, or more likely that your code is so unoptimised that barely any computation gets carried out while your code is waiting on memory/disk/network.
"I can't do anything because my program is I/O-bound" is more of an excuse / mental justification of why your program is slow instead of an honest reason for so.
zbentley 1 days ago [-]
> If you're I/O bound, that either means the problem doesn't require much computation - which is possible but fairly rare
This is backwards. I bet that by count, many more programs are written in domains where they're necessarily IO bound than the inverse. Anything that uses the network for its core functionality, anything reliant on a datasource whose aggregate contents are O(memory)+ size, or anything reliant on slow peripherals (lots of embedded software) are in this class.
Scientific simulations, HFT algorithms, video games, LLMs, etc.--the stuff in the other class--aren't inconsequential, but they're dwarfed in number by the class of software that spends 99+% of its time waiting for IO. Hell, entire programming languages (node.js) have been created in response to that proportion.
groundzeros2015 24 hours ago [-]
Bottleneck is a misleading word here. Yes the network is slow. But you can still save 200ms in response time by working on your CPU.
zbentley 21 hours ago [-]
Neither I nor GP used that word.
You're not wrong in that there are some programs that could be doing more work in CPU/memory while waiting for IO, but those, too, are dwarfed by the number of programs that can't really do anything meaningful until IO completes. Anything that RPCs or IPCs data is generally going to be waiting for a complete RPC IO to finish before doing compute (and even the most granular RPC protocols tend to communicate in pretty big, slow chunks to maximize throughput). Lots of software waiting on local hardware (e.g. storage) is similarly doing IOs in pretty big pieces--maybe page-sized, or disk-block-sized, or file-sized--and can't do much meaningful CPU work until that's done. In embedded, it often behooves programs to get as much IO-sourced data read or written as is possible with available resources before switching back to CPU work--doing this increases throughput on slow hardware, and can also improve power efficiency.
Put another way: modelling IO as a stream with something like io_uring won't save the end user much latency if the completions inside the ring wait for slow, batched IO, or if application code needs to see completed transactions before proceeding.
Latency, throughput, power, hardware cost--those often trade off, and there's no free lunch.
megagpt3 1 days ago [-]
Modern networks can transfer data more quickly than CPUs can process it. Starting at 100Gbps.
SideQuark 1 days ago [-]
That’s completely backwards by orders of magnitude. Modern CPUs outrun L1 cache speed by a lot, L1 cache speed is on order of 1000-5000 GB/s, a GB/s is real bits used, is 8x of Gbps, and network speeds include significant overhead bits for frame packing and other data unused for the actual payloads.
Is that the caliber of hardware I can expect to be working with on a cheap vps?
ykonstant 1 days ago [-]
What in tarnation?
orojackson 1 days ago [-]
ETL processes are heavily I/O bound, especially when you're trying to shuttle data from one enterprise system to another enterprise system. It's also common when the culture of data exchange from the regulator all the way down to the companies doing the actual work is batch processing where large amounts of data are shared once a day as opposed to real time. Excel spreadsheets are the norm, not the exception. Requests for data to be sent over via XML or JSON are mainly because my employer wanted to make it easier to process the data ourselves, but the regulators actually expect spreadsheets.
Most of the stuff I work on is almost exclusively network I/O bound. I wouldn't say it's a _result_ of bad engineering practices, though. One group decided on a particular system that's also public-facing, and the group I actually support prefers a more internal-facing system. It also doesn't help that the budgets for both projects are completely separate and firewalled from each other by law. Growth opportunities don't apply here because I deal with a captive market with legally-forced customers.
wavemode 1 days ago [-]
No, I don't think the way you're characterizing this is accurate. I/O is inherently very slow compared to computation. And many programs genuinely don't have any useful computation to do while waiting for I/O - because the result of that I/O operation contains the information needed for the program to even make its next decision.
Such programs are not necessarily impossible to optimize. One common optimization is to use an event loop, allowing just a few threads to handle thousands of concurrent operations. Because while a thread is waiting for I/O in one request or unit of work, in the meantime it moves on to work on processing another request/unit. Another common optimization is batching/grouping of I/O calls.
adgjlsfhk1 1 days ago [-]
> I/O is inherently very slow compared to computation
This isn't really true anymore. IO has bad latency, but modern SSD bandwidth is ~5-15GB/s. If your program is IO latency bound and processing less that 5GB/s you aren't IO bound, you aren't hiding your latency well enough.
wavemode 1 days ago [-]
> modern SSD bandwidth is ~5-15GB/s
That's nothing compared to modern memory bandwidth.
mananaysiempre 1 days ago [-]
> I/O is inherently very slow compared to computation.
Not anymore, no. Your SSD, before any caching, does gigabytes per second of sequential reads. For any bytewise processing, except the most trivial of tasks, you’ll struggle to get above a few hundred megabytes per second with scalar (native) code. To actually keep up with a modern SSD, you’ll virtually always have to hand-write SIMD loops, minimize the number of syscalls with tools like io_uring, or possibly be smart about distributing tasks across cores without ruining the access pattern.
For instance, simdjson is famously fast but I don’t believe it can keep up with say a high-end PCIe Gen 4 SSD like a Samsung 990 PRO, let alone the latest-and-greatest (and, literally, hottest) Gen 5 stuff. And I know of no Unicode normalizer that would be able to do a gigabyte per second on general inputs (not ASCII, not Latin-1) simply because the latency for dependent lookup table accesses is absolute murder.
wavemode 1 days ago [-]
No, you're comparing apples and oranges. All an SSD sequential read is doing is copying data from one place to another. So you should be comparing SSD bandwidth to memory bandwidth, not SSD bandwidth to (time it takes to execute some arbitrary algorithm). Or you should be comparing SSD bandwidth when performing millions of tiny random non-sequential reads and writes, to the algorithm time.
What your comment demonstrates is that it is possible in some cases for I/O to be fast enough to not be a performance bottleneck for certain kinds of programs. But not that I/O is not slow.
mananaysiempre 23 hours ago [-]
Muratori et al. like comparing speeds to (single-core) memory bandwidth (dozens of GB/s) and that’s a reasonable upper bound, but generally it seems to me that, unless you operate on huge elements and don’t do very much with them, you won’t get within an order of magnitude of it. Even if you think about RAM exclusively, the headline numbers are for sequential reads and things will slow down dramatically if you actually perform random accesses (IIRC, DDR5 is about as slow as DDR4 there in terms of physical time units, so much slower in terms of bus cycles). Meanwhile, in a real situation, you’re going to be bound by compute long before that.
And I think you’re being unfair labelling my couple of examples “some arbitrary algorithm[s]”: my choice was indeed arbitrary, but it’s also immaterial. The general setup would be that you’re processing elements in a loop and that your iterations are serialized (as they usually more or less are before you get around to optimization). A loop body of even three lines of C is likely to have a latency of 5–10 cycles or so, and you’re running on a core clocked somewhere from 5 GHz (desktop) to half that (server). So the best you should expect is ~500 MB/s if your elements are bytes, ~2 GB/s if they’re 32-bit integers, etc. For very simple tasks (that are also somehow not susceptible to vectorization), it is possible to not lose this order of magnitude and get down to almost 1 cycle/element in scalar code, but that requires heroic effort[1].
Your analysis is correct if and only if the data is on the same machine as the calculations. If the data comes from another machine, it comes at network speed. If it comes from the internet, it comes at non-local network speed. That's very different from SSD speed.
duped 1 days ago [-]
Classic latency vs throughput problem. 10s of GB/s of disk bandwidth doesn't help when my problem is serialized durable writes.
groundzeros2015 24 hours ago [-]
Cases where your cpu work is not a step in that pipeline are rare. (Cannot be parallelized)
jamiejquinn 24 hours ago [-]
Plenty of scientific simulations end up being IO or communications bound when at scale (e.g. 30k CPU cores). Can hide latency to a certain degree but basically any algorithm that uses timestepping must halt at some point to allow data to flow around the nodes, or to dump data to disk for visualisation or checkpointing.
In saying that there are some novel and very clever algorithms that continue on without seemingly necessary boundary data, that then self correct when the data comes through, thus completely hiding the latency at the cost (in both accuracy and time) of running a correction process.
kenfox 1 days ago [-]
The computers at the time had very little ram. The IBM System/360 didn’t get 1MB until 1968. I suspect a lot of programs were I/O bound just to be able to work at all. Most modern engineers cannot conceive of doing anything useful with 64KB and I think it’s a mistake to project modern practices 50 years into the past.
torginus 1 days ago [-]
Personally I'm quite sure this is super interesting, but I don't really have 3 hours to listen to this, even 1.5h at 2x speed is too much.
I would very much prefer something written down, so I could absorb this at my own pace. I know, gift horse, but still.
xen0 1 days ago [-]
Thanks to modern playback technology, you can pause it and resume play later at your convenience.
DanielHB 10 hours ago [-]
Casey mentions that at some point one of these great minds were upset that it was so difficult to write programs because they didn't have things like keyboards or screens. One of them even brings up exciting new developments in electronic typewriters that could be used to improve their workflows.
1 days ago [-]
pton_xd 1 days ago [-]
It's worth the listen if you're even mildly interested in the history of computer science. He's a great presenter. I guess at some point you do have to prioritize how to spend your time, though.
MobiusHorizons 1 days ago [-]
It is quite long, but I thought it was worth it if you can find the time. Short of that I think just reading the Knuth article the quote is from might bring similar insights.
andai 1 days ago [-]
Maybe go for a long drive? Long walk? Whatever floats your boat.
I used to do manual labor and I would work my way through like eight hours of audiobooks per day.
torginus 1 days ago [-]
I've noticed after I've switched to audiobooks that my retention is horrible compared to reading. For actual complex topics, it's even worse.
pitched 1 days ago [-]
I’ve found this too but I think audiobooks still have a place, they just can’t something you rely on being able to perfectly understand. A second book on the same topic or something in a topic loosely related but interesting. Or fiction to help stay awake on long road trips.
Rendello 1 days ago [-]
It's likely the same for me, but I tend to relisten to audiobooks I like, sometimes dozens of times. I have a hard time focusing on physical books, and I've probably only re-read a handful of them, so the net effect is that I take in more with a good audiobook.
I used read a lot of books when I was younger, then slowly stopped, and found getting back to reading quite hard as well. Armed with the knowledge that I used to be able to do this, I just forced myself to. And after a while, I found I had no trouble committing any more.
Rendello 6 hours ago [-]
I think it really is a matter of distraction. When I was using public transit a lot, I managed to get through books really quickly by substituting them for my phone. And there was a period where I had no phone for a few months in a new city, and I just read constantly. Even outside while walking. I do the same on my phone, after all.
yoyohello13 1 days ago [-]
I agree. I use audiobooks for fiction. For learning I need to read the thing to really retain anything.
knollimar 1 days ago [-]
If you want a spoilery TLDR:
It's more about the journey. He tracks down the origin, finds the support, finds the support flawed, and leaves you to your own conclusion rather than make a new flawed one.
The basic idea is that the origin assumes a highly critical inner hot loop, don't assume where it is, and optimize there.
There's some other time spent saying this justifies slower abstractions for maintainability elsewhere.
abainbridge 1 days ago [-]
Another point I liked was that there was, apparently, an influential book called Structured Programming, whose content was so universally agreed upon, that all programming became Structured Programming. Nobody needs the book anymore.
Some day null, statements (rather than expressions) and side-effects will have always been wrong.
tialaramex 1 days ago [-]
It's not the focus of the talk and so it's hard to tell if Casey understands (the choice to separate the words GO TO in several places suggests he does) but the `goto` keyword you've seen in several modern languages is not the problematic "GO TO statement", it's a de-fanged remnant, the toy poodle to GO TO's wolf pack.
The actual GO TO complained of is, like the jump instruction in machine code, just entirely unbothered by context. Want to go from the middle of this code about employee payroll processing to mid-way through initializing a weather simulation? No problem. Well. No problem for the machine, for a human programmer it's a complete nightmare. Actually that's putting it mildly, nightmares have more structure. You cannot do anything like that with for example C's goto.
You compiler can, and in a few cases (that's what the discussion about the tail-call optimisation is about for example) it will, but the program you wrote doesn't do this and so you don't have to try to keep the whole program in your head.
So in that sense GOTO died out with, maybe BASICs? I think the BASICs tend to have that wolf nature GOTO feature, but nothing modern has it.
bogomog 5 hours ago [-]
In the early 90s I was doing IT type C with SQL database software. There was one pattern of goto usage common: clean up at the end of a function if something went wrong (close db cursor, log something, return error code, and the like). When that space went to Java and to some extent C++ (and later C#) exceptions could be used instead. A defer statement would have been a more direct replacement for the pattern, though.
1 days ago [-]
hirvi74 1 days ago [-]
So, I have never seen anyone actually use GOTOs, so maybe a lot of the stigma comes from excessive, poor usage or something.
However, having written a good chunk of ASM in my life. I don't think jumps or branches are really that hard to follow. Jumps/Branches and GOTOs specify the next location. It is not as though one has to guess where.
It's not the arrow, it's the archer that is the problem.
tialaramex 1 days ago [-]
Never as in, in BASIC, or you've never seen goto in C? The de-fanged C "goto" is all over the place in Linux and in similar close-to-metal C software. C does not (yet, likely C2Y will fix this) have labelled break, so goto is used to say "I am inside a mess of nested loops, we're done, end the loops" and as a catch-all failure handler in some codebases.
My guess is that your ASM is inflected by structured programming everywhere. Yes, unlike COME FROM we can see where we're going next with GO TO but if you go back 60+ years there is some scary code that even today's optimizers probably wouldn't emit because it's too crazy. Suppose calculate-total-fuel ends with three CPU instructions which copy register F into register H then add register C to it and multiply the sum by four. Over in locate-horizon it so happens we need to add two things together and multiply them by four and we could do that last. So, if those two things were in registers F and C we could just GO TO that last part of calculate-total-fuel.
You're correct that we don't need to "guess where" it goes, but good luck understanding why the program works when it's like this, let alone the ordinary maintenance work of making small modifications.
AnimalMuppet 1 days ago [-]
> My guess is that your ASM is inflected by structured programming everywhere.
I think you're probably right. To expand on this:
In asm, you can have things that are clearly functions. You have a stack discipline going in and out of them. They end with stack cleanup, then a RET or some such, which pops the return address off of the stack and jumps to it. Within that function, you have JMP instructions (or whatever) that move around within the function. You may also call other functions, by pushing variables on the stack, and then calling JSR or whatever to push the program counter on the stack and jump, and when those functions return, you'll be right where you were in this function. That's all sane, and it's "structured assembly".
Non-structured assembly would be like the example in your second paragraph. You're in one function, and you JMP (not JSR) into the interior of a second function. Or, you simply don't have functions, just labels that you jump around do. That's not structured, and not sane.
hirvi74 24 hours ago [-]
I have no idea what the parent meant by, "My guess is that your ASM is inflected by structured programming everywhere."
(I am the GP)
Your response was far better than mine. If those instructions were executed more than a couple of times and the constraints/assertions were identical, I'd absolutely throw them in a function. I'll handle what I need to before/after the function.
I would never branch to some coincidental label. That is why I love ASM so much. If you are sloppy or lazy, you will most likely be punished severely for those choices.
(Tangential, but when I first learned ASM in college, I felt like I learned more in that one semester than all the sum of all classes in my entire degree.)
tialaramex 22 hours ago [-]
By "inflected by..." I'm suggesting that Structured Programming has altered how you write assembler even though of course assembler doesn't inherently provide that structure.
When you need to do X here and there, you write code to X and then you call it where it was useful - you don't have code in one function just jump to a label in a completely different function because, in this era where structured programming is taken for granted - that seems crazy. And it is, but only the same way that chattel slavery seems crazy today, in the Antebellum South it was just usual and likewise in 1950s computer software just jumping into unrelated code was normal.
Jach 21 hours ago [-]
One of the principles of structured programming is that you organize your code into functions, and that they have single points of entry, and single points of exit. We still have a lot of multiple points of exit in functions these days, but the reasons for them usually fall into either those of clarity (less nesting/branching) or performance by skipping unnecessary work, and the consequences of them are less dire in modern high level languages because such languages have GC (no fear of skipping a memory cleanup step) and features like try..finally to ensure code closing file handlers or logging or whatever is executed at the end. But we have largely eliminated multiple points of entry from most code. The main remnant I can think of is in the form of coroutines, though they are structured and limited enough (like modern gotos) and have well defined lifecycles that it's more forgivable.
hirvi74 1 days ago [-]
To be honest, I do not have much experience with C nor BASIC. I only know the extreme basics of C, and I have never written a line of BASIC in my entire life. My first real exposure to programming was in high school (Java). From there, I basically went from high-level languages -> x86 -> high-level languages again -> starting to play with AArch64 in an attempt to but some grooves back in my smooth brain.
> My guess is that your ASM is inflected by structured programming everywhere.
I am not certain what you mean.
> if you go back 60+ years there is some scary code that even today's optimizers probably wouldn't emit because it's too crazy.
Why was this as common as the allegations lead me to believe? Was it just a product of time? No better way to handle such logic at the time?
As for your example, I know it is intended to be interpreted trivially, but I just want to clarify I understand you correctly. Is the issue due to locate-horizon depending on logic from a completely orthogonal operation?
If so, I see the red flag immediately. However, I do not see an issue with creating a separate function. I'd just have to be damn sure H was not storing anything used for some other upcoming operation prior to the function call, I'd check F and C for valid state (if necessary), take other potential side-effects into account, etc.
It feels a lot like the DRY advice being abused in higher-level languages, you know? I agree with expression, "duplication is better than the wrong abstraction."
tialaramex 13 hours ago [-]
> Was it just a product of time?
Same reason that nobody did the Fosbury Flop at the 1948 Olympics - Dick Fosbury hasn't shown this technique yet and it's not obvious if you've never seen it that this is a better way to perform an Olympic regulation high jump. Twenty years later, he does exactly that and wins Gold and by the 1988 Olympics this is how everybody does a high jump, but in 1948 nobody knows it.
Your instinct to "create a separate function" is exactly Structured Programming.
cure_42 1 days ago [-]
Windows batch, no?
tialaramex 1 days ago [-]
COMMAND.COM in its batch mode? Kinda, sorta. The interpreter can AIUI "call" other files but the "goto" is restricted to the same file.
So we can jump over a variable declaration which is pretty confusing, but we can't jump into unrelated code.
Also while COMMAND.COM can't be as old as BASIC it must be pretty old.
knollimar 1 days ago [-]
No, they're not sarcastic. It was an interesting point; if an idea succeeds well enough people just do it and make it "common sense". It's a point in the talk.
brabel 1 days ago [-]
This is one talk that is worth the 3 hours (the talk is 2 hours, plus a qa with none other than Ginger Bill of Odin fame)!
bena 1 days ago [-]
The last hour is a Q&A. I don't know if that changes your perception of how watchable it is.
It's also a little fluffy. It's basically the history of how the book Structured Programming came about, because it's that book that essentially caused Knuth to write an article that contained the quote.
A lot of it is fairly interesting but it also highlights a big problem I do have with Muratori. He generalizes from self. He didn't fully understand or appreciate the quote about premature optimization, so no one did. And even though he's been told what it actually means, and was probably pointed to the full quote, he would stick on his interpretation. It's likely someone he views as "higher", like Jon Blow, challenged him and then he took the time to actually look at it.
Pannoniae 1 days ago [-]
"He didn't fully understand or appreciate the quote about premature optimization, so no one did."
He ain't wrong though.... most people definitely don't get the jive for sure.
Knuth was talking about a massively different kind of optimisation than what we do today yet people keep continuing to parrot the 97% figure uncritically like as it was some gospel. [0]
He made middleware used by countless games over at RAD. I'd say that counts as contributing to game development.
bena 1 days ago [-]
I didn't say he didn't contribute. I said he wasn't famous for it. And he's not. He's famous for his streams and talks.
kshallvari 1 days ago [-]
There is a 45 min version at Primeagen's "The Standup"
philippta 1 days ago [-]
Having watched both, they cover completey different topics.
wpm 1 days ago [-]
Audio transcription has been around for a while, you could solve this problem for yourself quite easily.
andai 1 days ago [-]
I get the auto-transcript with yt-dlp then ask a cheap LLM like DeepSeek to clean it up.
Though lately I've been uploading the audio to AssemblyAI, I somehow still haven't used up my credits after several years lol
At one point I built a system that would summarize the transcript and I'd be able to ask questions about it, but Gemini can do that natively now so I usually just use that.
It’s on YouTube, pull the transcript and format it with an LLM. If there’s no transcript, there’s Whisper. All of this can be done with local models too.
JavierFlores09 1 days ago [-]
Audio transcription is nowhere near the same as a properly written piece of text. Videos are simply a different format, and thus a lot of the communication is held by the fact that there's visual cues to go by. If one were to write an article about the same, you'd use much denser language as well as references/links around the topic. Maybe even restructure the whole thing to communicate it better through text.
It is something you can maybe whip up with the help of an LLM but then you're left wondering whether it is actually communicating what the video intended to begin with, as you wouldn't be as knowledgeable about the topic, one wouldn't be able to spot the parts where an LLM interpreted something horribly and thus reaching the wrong conclusions.
For all of the above, I do not consider audio transcript to be of any help when one prefers written content. It is useful for translation or people with hearing issues, but both of those receive help of the visual parts to convey the topic.
gessha 21 hours ago [-]
Yes but in the absence of a blog version, what are you going to do?
kshallvari 1 days ago [-]
THE LEGENDARY GAME PROGRAMMER
nchmy 1 days ago [-]
Isn't that Jonathan blow?
(to be clear, I'm a big fan of Casey)
arnorhs 1 days ago [-]
I believe this is referencing a meme on the primeagen's standup podcast, where Casey is referred to as legendary while he feels undeserving of this title.
Titles aside, his talk is really insightful and it is super interesting to do a deep dive on these old computer/programming topics as the modern concepts were being discovered
nchmy 1 days ago [-]
Hah, I wasn't aware of that. I was sort of referring to what seems to be not such a meme that jblow is always introduced like that. It just seems weird, even if true...
Yeah Ive been meaning to watch that talk - I love listening to pretty much anything Casey says/does. He's extremely thoughtful and fair.
kshallvari 1 days ago [-]
> I believe this is referencing a meme on the primeagen's standup podcast
Exactly!
frou_dh 1 days ago [-]
[flagged]
kshallvari 1 days ago [-]
Oh really, just for a funny joke?
miyoji 1 days ago [-]
Yes, programmers are exactly like everyone else in this way.
ModernMech 1 days ago [-]
> devfluencer
Thank you for bringing a term to a thing that has been bugging me: people who purport to be devs but spend most of their time on podcasts and blogs talking and writing about being devs, rather than just being devs.
dgellow 1 days ago [-]
They are both legendary game programmers
inigyou 1 days ago [-]
Are they? Casey programmed the walk monster for Jon's game, a brute force tester to make sure you could always walk both ways and never get stuck, instead of building the walking system using a nav mesh. There was one place you could walk where you weren't supposed to, despite the walk monster, that would have been obvious if it was a nav mesh.
dgellow 1 days ago [-]
He’s known for his Handmade hero series (unfortunately archived now) where he developed his game over 500+ episodes: https://hero.handmade.network/
I’m sure you can find bugs in the work of any game dev you would consider legendary, game implementation is generally very messy
bena 1 days ago [-]
660+ and the game was never finished
squh 7 hours ago [-]
You have to consider that each episode is only ~2 hours and every line of code was written on stream. That's about 7 months or so of full-time work. Within that time a lot of the hours was spent teaching instead of coding, too.
There is massive amount of knowledge in there for anyone bothering to actually learn something and it was all provided free of charge. Hats off to Casey for sticking to it as long as he did.
inigyou 1 days ago [-]
Doesn't mean you can't learn a lot from the attempt. Technical craftsmanship and project management skill are two completely separate skills. Definitely don't learn project management from Casey.
tialaramex 1 days ago [-]
Sure, but the biggest reason to direct anybody to these videos is that they should actually make that video game they want to make, not just think about it - and so the fact that Handmade Hero just trails off and doesn't end up producing a finished video game is a problem.
The craftsmanship is dubious. I think it's a problem that people assume Casey knows what he's doing when so often he's like "We're doing it live" and Casey's hand coded solution is pretty bad whereas the thing which came in the box is very good. Casey understands Casey's version, and that's an upside if you are Casey but you are not. If the result was a finished game then hey, whatever, the game was finished and that matters. But that part didn't happen either, so then it's just like watching Tsoding or something.
ahaferburg 1 days ago [-]
At some point I stopped caring about Handmade Hero because he just never got out of the global illumination rabbit hole. He should have followed that as a side quest, behind a feature flag. Maybe one week for progress on the game, one week for rendering experiments. He didn't seem to make much progress, and it just fizzled out as he ran out of steam. Finishing should have been a bit higher on the priority list. It was just not setting a good example from a project or self management perspective.
guitarlimeo 1 days ago [-]
Well Billy Basso was hugely inspired by the Handmade Hero series and went on to make the critically acclaimed Animal Well because of it. That's enough of a win for that series in my eyes.
tialaramex 22 hours ago [-]
Animal Well is a great game, there's a lot in there and I appreciate it when a game is deep rather than big but I don't see much connection there. Maybe it's like how "Maze: Solve The World's Most Challenging Puzzle" (a book I disliked immensely) inspired Tonda Ros to make "Blue Prince" a game which I enjoyed tremendously.
guitarlimeo 15 hours ago [-]
Animal Well wouldn't exist without Handmade Hero, Billy himself said it in an interview. The first game architecture he made was RAII etc and full of spaghetti and then he actually followed Handmade Hero and started to getting somewhere.
bena 5 hours ago [-]
To be fair, there's another adage that comes into play here: Build one to throw away, you will anyhow.
His second attempt, no matter how he made it, was made with the context of his earlier mistakes.
RAII doesn't lead to spaghetti code necessarily. If anything, "unity builds" would more encourage spaghetti as it's easier to reference anything else in your build.
andersonpico 23 hours ago [-]
animal well is such an incredible game and it shows how talented Billy Basso is in multiple disciplines
3eb7988a1663 1 days ago [-]
John Carmack is a legendary game programmer. If I can identify a superior algorithmic approach to something in Doom, does that disqualify Carmack's credentials?
tialaramex 1 days ago [-]
First I think I'd want to see you identify a superior algorithm known when Doom shipped and then we can have that discussion about who's credentials should be disqualified.
3eb7988a1663 1 days ago [-]
I never made claims to be any kind of game programmer, but sure.
Fine, as I recall, Doom data assets are searched linearly whenever they need to pull new data. So if you switch to the chainsaw, and need to rev, the game does a linear scan of all graphics, maps, and sounds looking for the vroom noise.
There are many different data structures that could perform this lookup faster. These have been known since the earliest days of computing.
Does it matter? No. Engineering is all about trade-offs. A linear scan was obviously fast enough and simple to implement.
Similarly, if walk monster manually trolls the map, that does not say anything about Casey in isolation. I believe all of Blow's games use a custom game engine, so unless a nav mesh system was already implemented, that was going to take additional work. The walk monster may have been better bang-for-buck.
tialaramex 22 hours ago [-]
You probably have a point on linear search for assets (I haven't checked but I've assumed you're right about how it works in the released source)
There are some tradeoffs here, my instinct would be to bring hash tables into play and that's perhaps a mistake because it means you're doing fewer reads but more arithmetic and on some hardware that's a bad trade. But "linear search" is only the simplest and probably not the smartest option.
dgellow 1 days ago [-]
I think you’re both saying the same thing
dist-epoch 1 days ago [-]
No, that would be John Carmack
flumpcakes 1 days ago [-]
Death-frightening scion capable of seeing beyond the illusionary world before our eyes John Carmack?
fantasizr 1 days ago [-]
I took notice when he breaks down the ethics of gen ai as it pertains to online theft, that AI broke the natural order of putting content online where you'd trade exposure for use https://youtu.be/bjO-s4rNPlY?si=uAtBTb6V_Sw1puBU&t=2244
cuechan 1 days ago [-]
He is just legendary when it comes to game programming
socalgal2 1 days ago [-]
What games has he shipped?
sirwhinesalot 1 days ago [-]
The legendary games programmer thing is a meme.
Casey is professionally best known for his work at RAD game tools (a highly successful middleware provider for game development back in the day), not games he himself developed.
What he is most highly regarded for is his teaching, particularly the Handmade Hero series on YouTube, which various programmers directly attribute as being responsible for their own professional success to a large extent.
He did work on the video games Dungeon Siege and The Witness afaik, but that's not what he is known for.
He also made one of the fastest terminal emulators (refterm), purely out of spite, to show Microsoft they were full of shit in their answer to an issue he posted to the Windows Terminal github. This directly led to performance improvements in Windows Terminal.
tancop 1 days ago [-]
"Highly successful middleware provider back in the day" is an understatement.
Bink was the cutscene video codec for a long time and it's still the most popular option by far. Kraken compression is so good Sony worked with AMD to make a hardware decoder for the PS5 and paid for a general license that lets games use it for free. They are probably stronger than ever in terms of games using their tech and collected license fees.
DanielHB 9 hours ago [-]
I remember using it to encode some videos for a school presentation. I think I needed to make it fit into a CD and play-on-double-click and Bink was the easiest tool to do it in.
socalgal2 14 hours ago [-]
Are you conflating what Casey worked on with what RadTools worked on?
9 hours ago [-]
georgemcbay 23 hours ago [-]
> They are probably stronger than ever in terms of games using their tech and collected license fees.
True, though they've been acquired by Epic since 2021. They keep the RAD Game Tools branding, but aren't their own separate company anymore.
Oodle/Kraken is mostly the work of Charles Bloom and Fabian Giesen, a couple of other "legendary game programmers" at RAD, but who aren't quite as visibly prominent as Casey has been in the modern software developer mediasphere.
inigyou 1 days ago [-]
He mostly evangelizes non-pessimal software. He advocates that your program should only be a small factor like 3x slower than the hypothetical optimum - instead of 10000x slower as today's software often is. He compared Visual C++ 6's debugger on hardware from the time to current Visual Studio's debugger on current hardware, and found the former much faster when performing the exact same tasks on the exact same project file and code files.
sirwhinesalot 1 days ago [-]
I have recently gotten a 45x on a shitty vibe coded Python project by using mypyc (2x), replacing datastructures with more CPU-friendly alternatives (15x) and using some smarter heuristics rather than just brute force on top of that (45x total).
And I never left Python, the slowest language around.
AI is only going to make the problem worse unless you know what to look for.
andersonpico 23 hours ago [-]
> He compared Visual C++ 6's debugger on hardware from the time to current Visual Studio's debugger on current hardware, and found the former much faster when performing the exact same tasks on the exact same project file and code files.
this shouldn't be right, wtf
DanielHB 9 hours ago [-]
From what I heard from Visual Studio folk it is surprising it is not orders of magnitude faster rather than "much faster".
Also I learned to program in Turbo C and Turbo Pascal with its debugger and nothing comes faster (at least nothing single threaded)
fwlr 22 hours ago [-]
Yes, it really shouldn’t, and so Casey goes to great and careful lengths to fully evidence these results. This is a nuance that’s easy to miss in evangelizing “Casey Muratori content” to others. It’s easy to gloss “abjectly unacceptable performance” as “bad performance”.
Jach 21 hours ago [-]
And people still come out of the woodwork with a parade of excuses, and sometimes he addresses those too. But it's no wonder that he gets a bit brisk or blunt with people, and turns off others who haven't been following along.
imron 9 hours ago [-]
VC++ 6 was an amazing IDE. My favourite of all time, with the debugger being one of the highlights. Still unmatched today.
inigyou 12 hours ago [-]
It's right.
When did you start using computers? I'm getting older now, and some people weren't alive when I first used a computer.
It's because the older programming approach was to just tell the computer how to do the thing - and the modern approach is to write a framework inside which you write a framework inside which you write an inefficient version of the thing because it's constrained to the tools available in the framework stack.
Old-school implementation of "scroll up": copy pixels then redraw the pixels at the bottom.
New-school implementation of "scroll up": delete virtualized rows from the beginning of a DOM table and add more to the end. Change the height properties on different parts of the scrollbar. Relayout the whole DOM, several times if you're not careful. Redraw the whole screen.
He works in the engine/tool side of things. He worked on some widely used libraries, mainly Bink 2 (video codec) and Granny 3D (3D animation) used in a ton of shipped games.
namuol 23 hours ago [-]
He did some work for The Witness and wrote some nice technical blog posts about it [1].
I don’t know the extent to which he contributed beyond that, but he is therefore by some reasonable definition _a_ game developer.
[1] the-witness.net/news/author/casey/
dundarious 1 days ago [-]
He did middleware at RAD, home of a lot of good stuff, and worked directly on at least The Witness
singleshot_ 1 days ago [-]
Very interesting that the Christmas Disk still has not been released.
ex-aws-dude 1 days ago [-]
What I don't understand with both Muratori/Blow is that they seem to be incredulous that someone wouldn't just make software high quality and fast for the sake of it
Like they can't comprehend the fact that these things only happen if there is an incentive for it
I think its because they work in games where there is a business incentive for performance, users care a lot if framerate suffers in a game
They are so used to working in games that they take it for granted that the business incentive for performance exists
As someone who has worked in both AAA games and big tech it just comes across as a bit naive to me
softwaredoug 1 days ago [-]
I think there could be a version of our industry with people doing better work. And it’s worthwhile to study why we don’t have that industry.
Yes it’s incentives - but why these incentives exist and not others is an interesting question to study.
In the 2010s we pushed everyone to learn how to code and maybe lost some of our higher standards. Maybe we’re doing that now with AI?
Maybe broader social and economic incentives have had a lot of short term thinking of late? It’s hard to organize society or a company to go in one direction very long before incentives completely change everything. Who knows, maybe that’s a feature, not a bug.
It’s worth asking though why we have this software industry and not another one. I’m not convinced it’s for legitimate reasons.
stereolambda 1 days ago [-]
I think the thesis would be there's a colossal structural/market failure in software. This because of many degrees of freedom, consumer ignorance, network effects and such. Compare this to physical things. The difference between the absolutely shoddiest couch sellable and the practically best one is tiny, compared to the difference between the worst software you could impose on the market and what could be achieved by competent engineers if we somehow removed the wrong incentives from capital and bureaucracy. With most software most people use, we are near the bottom of this range. I don't know if I fully buy that, even if we completely ignore whether removing these incentives is at all realistic. But would also say there's something there.
Then we can talk about how we all pay the increased cost in energy and materials, and so forth.
groundzeros2015 23 hours ago [-]
The fallacy is assuming it takes more work to get good performance. When what’s really lacking is clarity about the problem and solution.
Optimizing code for max performance does take work. But just not doing incredibly dumb things and writing simple programs just takes education (and re-education).
I can’t count how many times I’ve replaced a distributed system with for loop.
DanielHB 9 hours ago [-]
You are assuming a greenfield scenario, in large brownfield projects it is quite easy to degrade performance by innocent changes. That is usually when the perf goes to trash.
It takes a lot more work to keep performance good in a large project than just letting it degrade.
groundzeros2015 3 hours ago [-]
If you aren’t willing to change the system (have no business reason to) then you can’t implement or benefit from any new paradigms or advice and are necessarily bound by prior decisions.
danpalmer 22 hours ago [-]
Setting aside incentives – because I think that's very fair to push back on, we should strive for better craft...
I find Blow in particular a bit naive when it comes to the complexity of modern non-game software. Games are very complex, but Blow puts them on a pedestal separate from other software, as if a web backend can't be as complex. He calls software slow while simultaneously ignoring most of the functionality.
He's essentially never had to deal with distributed systems, or anything that allows multiple users to use the same system, with all the performance and anti-abuse issues that come with that.
His opinions about game development are insightful, and his thoughts on craft can be quite inspiring, but I think he could benefit from a stint on a (good) big tech server team to see some of the things he's currently missing.
ButlerianJihad 22 hours ago [-]
Heh heh heh. In actuality, Jon Blow has plenty of experience with distributed and multi-user systems. I worked closely with him while we were developing TinyMUCK 2.x, and the MUF programming language, a stack-based, simplified Forth.
In fact, Jon played a crucial role in the invention of the first MUF worm. We had just finished an extension to MUF that permitted triggering programs wherever a text string was expected. This gave a great flexibility and functionality to run MUF programs passively, by third parties, all kinds of new ways.
Unfortunately this also gave rise to a "worm" type of MUF that replicated itself whenever it was triggered passively. We went a little crazy when this was discovered, but it was all part and parcel of a multi-user game system, where there were permissions to consider, and multiple read/write/execute access to objects that should be owned by their respective player characters.
Jon Blow also inspired a lot of my exploits as I tested other systems of MUD programmability. In that time, TinyMUSH was also experimenting with a language that was more akin to scripting, and boy was it vulnerable. TinyMOO was also coming out, with a rather revolutionary programming paradigm. So in the mid-1990s, just in the insular TinyMU* world, there were at least 3 branches of programmable MUDs being actively developed.
Hopefully Jon could recall my greatest contribution to the corpus of early MUF programs. On Atlantis MUCK, I programmed a simulation of Conway's Game of Life. Of course, there was no animation possible and there was no multithreaded execution possible, either. So my program was given a starting board of Life, and then computed each frame in sequence, output it into the text stream of the game, and terminated after N iterations. Meanwhile, the entire game server locked up, because it could only run one MUF program at a time. Everyone else's commands and input would be held until my MUF program finished running.
This was a real issue, of course, and Jon and I worked to mitigate the problems caused. I think, ultimately, we just limited the number of iterations or the number of instructions that could be executed by any one MUF program at a time.
But Jon and I collaborated on a tutorial for player-programmers, and he helped us find and extinguish many bugs. And these were very complex systems, with 64 or even 256 simultaneous players.
Perhaps Jon has forgotten some of those lessons, but perhaps not.
danpalmer 20 hours ago [-]
Fair enough! That's some good experience and should address some of what I've heard from him, but perhaps he has forgotten.
That said, and this is coming from someone who (sadly) missed the era of computing you're describing, I think there's still a difference between dealing with a 64-256 player multiplayer system, and scaling a server application across dozens? thousands? of machines, across the world.
My limited research suggests that MUDs run almost exclusively on a single machine (?), or even just having a small number in one location drastically simplifies the problem space.
Aeolun 22 hours ago [-]
I mean, it’s a thing in pretty much all lower level software that you want it to be fast. I guarantee you postgres and redis are trying to optimize the software to the cycle, because wasted cycles are wasted for all their users.
mberning 1 days ago [-]
Will have to give this a watch after the kids go to bed. I like a lot of Casey’s views even if I don’t agree with them.
dkersten 1 days ago [-]
I enjoyed this talk. It’s long, but it’s interesting and goes into a lot of “lost” history.
register 1 days ago [-]
What is the thesis of his talk? Is it worth viewing?
Tuna-Fish 15 hours ago [-]
He tracks down where the statement "Premature optimization is the root of all evil" actually comes from, and whether the assumptions set out in the text where it was published still hold.
It doesn't really contain anything new or earthshattering, but if you find diving into the history of programming as a discipline interesting, it might be for you.
I would recommend it to anyone who liked the rambling style of "The big OOPs", but imo that one was better. I still liked it.
Panzerschrek 1 days ago [-]
It's a common situation for many quotes of such kind. Taken out of context they loose or completely change their initial meaning.
jdw64 1 days ago [-]
Do not guess. Measure, but only measure the bottlenecks that threaten the business
tialaramex 1 days ago [-]
tl;dr the saying is that "premature optimisation is the root of all evil", and Casey burrows into contemporary data to show that really although the claim was 3% of the code takes up 90% of the runtime even then it was more likely 4% takes 50%.
The best thing you could take away from this lecture is something a reasonable person should take away from the original "root of all evil" saying anyway. Measure. Measure. Measure. If you aren't measuring that's not optimization it's masturbation.
Ironically in the process of measuring for a third time yesterday I tripped a bug in Bill's language for which I opened an issue. This is not the goal of measuring three times but merely a happy accident.
Along the way Casey discovers (?) that Structured Programming means just what we today call programming†, that software was a lot smaller in the days when 4096 bytes of RAM was a good entry level option and that loads of these famous people from 1970s computer science knew each other.
† And knowing about this is one reason the Structured Concurrency people want that everywhere. Very possibly there's a future where it seems silly that people once wrote programs which did not use structured concurrency.
brewmarche 1 days ago [-]
In other videos Casey argues against the profile–fix–repeat workflow (I’m not saying that you necessarily meant this by measuring), instead arguing for estimating the theoretical maximum, then trying to get close enough to it. His argument is that the former might push you towards a local minimum without realising that you could do much better
tialaramex 1 days ago [-]
When "Estimating the theoretical maximum" you're going to bring your assumptions to that estimate and so you're equally likely to get stuck in some local optimal point because you didn't see what else was possible, so that doesn't persuade me much. Worse, now it's a local optimal which might be entirely in your head.
There is some space for this "Estimate the theoretical maximum" as a high level insight, if the software needs to do a thing which your estimate says is impossible that's important to address up front but I think for practical software engineering you're much more likely to need profile-fix-repeat
Actually measuring is crucial and an estimate is not a measurement. I think one of the reasons many in the Handmade Community don't like measuring is that it too often ruins their "estimate" of how the crap they've written is achieving a "theoretical maximum" in favour of a boring fact that it's much worse than alternatives.
zipy124 10 hours ago [-]
Usually the ideas it that the theoretical maximum is not a local optima though, but some ideal global optima which you know you realistically cannot reach.
That is you know the data is some size, and memory throughput is some rate, and clock cycles is some rate etc...
You are entirely correct about the line of thinking if your assumptions are down a different line of thinking such as algorithmic estimates though, where you might miss some better way of doing things.
zipy124 10 hours ago [-]
I've always used this method of working and can stand by it. For instance let's say you are working on a high-speed real-time image processing system. If you know that each image is 1080p for instance, that's 2 million pixels, and we will have 3 bytes per pixel for colour, then that's like 6MB per image (without compression). If you have 25 GB/s of RAM bandwidth, then there is a hard limit of ~4 thousand frames per second if you aren't even doing any processing. Therefore if you strip out your processing and are only getting say 400 frames per second, you know something is going wrong.
This can be applied roughly to anything. I find it very handy when thinking about CPU/GPU performance as well. For instance knowing that at 5GHz you have 5 billion clock cycles per second, which depending on instructions and pipe lining etc... can be roughly 5-30 billion operations per second per core tells you how long some process should take if you know roughly how many operations are required and the data size you are operating on.
Obviously as you drill down things get much more complicated, but they give you some rough idea about how fast things could be under some set of assumptions. Very similar to how a physicist works with models and assumptions.
Pannoniae 1 days ago [-]
Sure, but there are performance issues no profiler will catch in a straightforward flamegraph reading.
Some examples:
1. Your hottest loop is spilling registers which only shows up as non-local cache thrashing (i.e. some other random code becomes slow) or randomly slow instructions i.e. "why is this xorps to initialise this float suddenly slow" due to pipeline stalls
2. Your code stops fitting into cache due to the code bloat, there's no one method which is slow, everything is slowed down by a percentage factor
3. A lot of useless work being done like temporary strings being copied everywhere
4. Your code is "I/O bound" because all the data you're accessing is scattered all across memory, leading to completely predictable TLB stalls
It's very easy to make a large program, quite a bit harder to make a small one...
smj-edison 1 days ago [-]
How do you even measure those? Will that show up under something like VTune?
Pannoniae 1 days ago [-]
Yes, VTune is good.
I'm on AMD and uProf isn't that good (it often fails to match PDB to source, it's slow / crashy, stuff like that) so I mostly use Superluminal myself (has godly UX) and only drop down to uProf when I have a suspicion and want to gather hardware stats.
LtdJorge 1 days ago [-]
You can do all that with perf on Linux, but everything is manual.
inigyou 1 days ago [-]
Casey generally, across all his material, advocates for non-pessimisation. Measurement takes too long to apply it to your entire program. He advocates for thinking about how much work the computer should actually have to do, then not making it do much more than that, at all times. This means avoiding serial dependency chains on the network, and huge towers of abstractions, and redundant work. He allows for writing lazy slow code as an intentional tradeoff that you may revise later if it becomes a bottleneck. He does not allow for inherently slow architecture.
socalgal2 1 days ago [-]
> Measure. Measure. Measure.
The problem is knowing what to measure. There's another saying
"When a measure becomes a target, it ceases to be a good measure."
As an example from memory, there was a game dev company that celebrated they had maxed out the cores on the PS3. That didn't mean anything though, anyone can max out the cores by filing them with bad code. But hey, their "measurement" told them they had maxed out the machine
tialaramex 1 days ago [-]
> The problem is knowing what to measure.
This can be a problem, but much less so because so often we're doing "easy mode" where we don't need a proxy. The "it ceases to be a good measure" is because you're measuring a proxy. You wanted to deliver happiness, you measured wealth because it was easier to measure but seemed correlated and now you've got rich miserable people, oops. But software engineers can often measure the actual thing they want to improve directly, not a proxy and so it cannot cease to be a good measure.
ahartmetz 1 days ago [-]
With a few caveats, though they do tend to get fixed over time. For example, frame rate. Higher frame rate is better, the end, right? Wellll...
Latency and hitching are annoying to measure, so for a long time, they were pretty much ignored. That has improved and hopefully will continue to improve. There is the bufferbloat initiative, gaming magazines take frame time histograms now, input devices and screens commonly have their latency measured. But latency is still under-measured - for GPUs / GPU drivers, for all software, for games and particularly for websites.
tialaramex 22 hours ago [-]
Yeah, frame stutter is definitely one of the cases where somebody may end up targeting a proxy and that saying about targets happens as expected. A game which typically hits 50-60fps but has occasional half second stutters may not fix the stutters if your proxy was "time to render 100 frames" and you throw out outliers - since every stutter will be an outlier.
groundzeros2015 24 hours ago [-]
That quote mostly applies to human organizations where the connection between outcome and metric is questionable. Or where incentives unexpectedly change behavior.
If your goal is fast software it can be measured quantively and you’re likely to improve the actual thing using those metrics.
socalgal2 14 hours ago [-]
The goal above was to write fast software. The developers made the mistake of seeing that they had saturated all the cores as "We did a good job of using the entire machine" = "our code is fast". They had a measurement and misinterpreted what it meant.
groundzeros2015 3 hours ago [-]
This is just dumb. You wanted to improve performance but measured memory instead? Whoops.
Why not measure performance for which there are many good metrics.
perkinsResearch 1 days ago [-]
Legend
fizlebit 24 hours ago [-]
I think I kept skipping each 5 minutes until I got to something interesting and then was at the end of the video. What is his point?
thedevbirb 8 hours ago [-]
[dead]
jeffrallen 1 days ago [-]
It's premature optimization, according to the video description.
Most of this kind of content comes off as relevant/topical but mindless entertainment. This is not a "deep dive" of anything that I could practically apply in my daily work. It feels good to think this content might add value and then to subsequently consume it, but it's effectively junk food.
I used to spend a lot of time watching crap like lex and primagen before it became obvious that these kinds of information channels are not meaningfully enriching despite sounding like they should be to a casual observer.
> Open source is the very worst thing currently going on because it is so incredibly exploitative, it's far more exploitative than any actual company is of the workers who work at the company.
> Even the people who are getting paid in open source are getting massively underpaid to do it compared to how much the people who are using their code are making, it's nothing compared to the power that is accreted by the people who have co-opted that work thanks to the open source model. And then mark zuckerberg gets to define how the internet works despite having paid for almost none of the software that his company actually needed to make that work.
> It's like feudalism or serfdom, these people did the work and got nothing for it. It's like you took the worst aspects of capitalism for workers and the worst aspects of socialism for workers and put them together, that's open source. You get no power and you get no money.
> It's exploitative whether the people chose to be exploited, just because someone chooses to let you exploit them does not mean that you didn't exploit them. And for the record that's how most exploitation works; convincing people to do something that turns out to be very bad for them and very good for you, and that's exactly what the open source movement has turned out to be.
> I really don't see the "we post stuff on github under a gpl2 or lgpl or apache or mit license", all that is to me now is just exploitation. You can say that there's solutions but until someone demonstrates that those solutions work, it's the standard "real communism has never been tried" argument. AGPL is the only thing that I've seen so far that's an attempt to fix these fundamentally unfair compensation practices.
Casey is far more measured, careful and accurate. Casey feels like an engineering educator first, who happens to make YouTube videos. I don’t always agree with Casey’s technical opinions, but he seems like a thoughtful guy who thinks things through.
When has History of any kind ever held practical value?
I should also add that even interviews are valuable. Coders at Work is fantastic at that and drawing out technical opinions and reasoning; notably the subjects interviewed don't all agree and have sometimes drastically different work styles and preferences. (And also adding from another comment, Turing Award Lectures are great history, and Uncle Bob even has a history book out now on influential programmers.)
But I'm curious who you think #3 is.
I skimmed through the video because I don't have two hours but there's no lost knowledge in it, he's going over well known papers from Dijkstra, Hoare or Knuth. The paper he bases most of the talk around Knuth's Structured Programming with go to Statements is his most often cited work, there's a very high chance you'll encounter that early in any undergrad course. So the answer to your question is, pretty much at any local uni with a CS course.
Casey points out that many of the lessons they came to were so obvious that it just became how it is done that no one even remembers it was done in any other way. In the talk the laments it makes it really hard to track down who originally came up with these ideas.
It’s a pity there’s not more people like him. I’ve met some incredibly smart people in the FP world. I wish more of them made approachable YouTube videos explaining what they know. I recently learned about “Safe Haskell” which, if adapted to rust and other languages, could solve a lot of the security problems with npm and cargo. But nobody knows about it! We need FP Casey.
There's two reasons Safe Haskell is substantially worse than what Rust does. Safe Haskell works by annotating code that is safe, but that's entirely backwards. We need to annotate unsafe code, and explain in plain English exactly why it is okay in that particular instance (of course it would be better to explain in code, like, give a formal proof that is checked by the compiler, but then it isn't unsafe anymore!). Safe Haskell answer for that is to annotate good unsafe code as trustworthy, but that doesn't work because it doesn't goes in detail on why the code is trustworthy (to do so you really need to go into the details, you can't handwave it). It's the // SAFETY comments that are at the heart of Rust's unsafe, carefully explaining safety invariants that must be kept (specially important if we are modifying code), not unsafe { } blocks.
The second reason is much simpler. It's optional, and approximately nobody uses Safe Haskell or cares about it. If people used it we would have something to improve upon. So Rust achievement here is mostly sociological, it's a community of programmers that care about safety. Which is good given that a Rust program typically have much more unsafe code than a Haskell program.
Here's a thread about Safe Haskell issues https://www.reddit.com/r/haskell/comments/zwkqke/deprecating... that links to https://discourse.haskell.org/t/deprecating-safe-haskell-or-...
And an older thread https://www.reddit.com/r/haskell/comments/msa3oq/safe_haskel...
This would guarantee - at a language level - that leftpad or log4j can’t root my computer.
Safe rust doesn’t give this guarantee. Safe code can still make arbitrary syscalls. Safe rust can convert a path string to a File. Or open arbitrary network sockets. Rust also doesn't have a way to import a crate but forbid the use of any unsafe blocks.
I want to be able to use 3rd party code from cargo without getting hacked. Right now rust does not keep me safe from these supply chain attacks.
I don’t know enough about safe Haskell to know how close it tacks to this. But that’s what I want.
But if you don't want an ironclad guarantee and instead you're content with making wrong code obviously wrong even if it's not formally verified, then I recommend Haskell with a capability system (what the Haskell world calls an "effect system"). As far is I'm concerned there are two practical choices in 2026, Bluefin (mine) and effectful (one of Bluefin's inspirations)
* https://hackage.haskell.org/package/bluefin
* https://hackage.haskell.org/package/effectful
While I suspect he could do research, he also just isn't personally experienced in large tracts of SWE culture as others. Others have worked a far larger variety of jobs in the software industry and thus tend to have a broader idea of the cultures and how they've evolved. In a recent clip I saw of him and others discussing how BS are various phrases engineers say (the root of all evil phrase being one of them), he hadn't heard of the YAGNI acronym/idea before. That's no fault on him, programmers live in different worlds. But regardless of one's feelings about such ideas, or whether they've heard of them or not, a history of SWE would have to cover them and the schools of thought that originated them. And this is just one minor thing in living memory and recent careers, there is so much from before he was born that has to be researched and if possible also talked about with the older practitioners who are still around and remember things. If you don't already have some awareness of things that should be covered in a history, whether from personal experience or just general curiosity about history that has led you to see such things before somewhere, the only hope is further research uncovering them, and I don't trust his researching that much.
Reading The Psychology of Computer Programming from the 70s was such an interesting read for me from a historical perspective as well as being able to see how some ideas they were thinking about back then came to fruition in some way or another. (e.g. Stack Overflow.) It's just one small part of history. A "complete" history as the top comment suggests would be better as a multi-author endeavor, would probably need many books, and that's if it could be done at all. In the meantime at least one can always go read primary sources and develop an idea of the history on one's own.
It's just absurd to me to point at Casey, of all people, as the most informed person who would be best positioned to publish something containing all of the history of anything broad.
Like I agree with most everything he says and I like to optimise my own software, but for my day to day jobs and contracts it's simply not feasible.
That level of performance and rigor is not what is demanded nor paid or appreciated.
E.g. in Enterprise circles it's still OOP from top to bottom.
Mixed with a tad more functional style due to varying adoption of that paradigm in the languages used by enterprise.
But low level optimizations are not interesting to these customers at all, even though they could help them with some problems.
I suppose it is a matter of what one works on, but that has been consistent in my career.
I have seen many be rewarded for choosing what I would deem intentional, gross negligence. But hey, those devs get work finished faster, and that's all that matters to the non-technical folks.
His dismissal of the argument Knuth makes regarding the hot loops could have been explored a bit better. I found it weird he didn’t mention the difference of types of programs of then vs now. Even today, in scientific code it is still absolutely the case a lot of the time that a huge chunk of the runtime comes from a single very very hot loop. It might be hidden in a library, but it’s there. Instead he focuses only on “program size”. Knuth samples where very small FORTRAN programs (compared to today’s standards). Today’s program are bigger but the fundamental number crunching primitive of “let’s compute stuff in a loop” remains. It’s just buried under a pile of extra cruft (data loading, parallelism, dispatching etc).
Now we just deal with a lot more programs that are of a whole different class compared to what they where doing with computers in the 70s. We have much more I/O involved. And hot loops don’t like being I/O bound.
iirc, in the talk casey in fact does goes on about how he tried to find examples, but couldn't. in the q&a, he was also asked about this further.
"I can't do anything because my program is I/O-bound" is more of an excuse / mental justification of why your program is slow instead of an honest reason for so.
This is backwards. I bet that by count, many more programs are written in domains where they're necessarily IO bound than the inverse. Anything that uses the network for its core functionality, anything reliant on a datasource whose aggregate contents are O(memory)+ size, or anything reliant on slow peripherals (lots of embedded software) are in this class.
Scientific simulations, HFT algorithms, video games, LLMs, etc.--the stuff in the other class--aren't inconsequential, but they're dwarfed in number by the class of software that spends 99+% of its time waiting for IO. Hell, entire programming languages (node.js) have been created in response to that proportion.
You're not wrong in that there are some programs that could be doing more work in CPU/memory while waiting for IO, but those, too, are dwarfed by the number of programs that can't really do anything meaningful until IO completes. Anything that RPCs or IPCs data is generally going to be waiting for a complete RPC IO to finish before doing compute (and even the most granular RPC protocols tend to communicate in pretty big, slow chunks to maximize throughput). Lots of software waiting on local hardware (e.g. storage) is similarly doing IOs in pretty big pieces--maybe page-sized, or disk-block-sized, or file-sized--and can't do much meaningful CPU work until that's done. In embedded, it often behooves programs to get as much IO-sourced data read or written as is possible with available resources before switching back to CPU work--doing this increases throughput on slow hardware, and can also improve power efficiency.
Put another way: modelling IO as a stream with something like io_uring won't save the end user much latency if the completions inside the ring wait for slow, batched IO, or if application code needs to see completed transactions before proceeding.
Latency, throughput, power, hardware cost--those often trade off, and there's no free lunch.
And a GPU does orders of magnitude past this.
Networks are not even close.
https://chipsandcheese.com/p/a-new-year-and-new-tests-gpu-l1...
Most of the stuff I work on is almost exclusively network I/O bound. I wouldn't say it's a _result_ of bad engineering practices, though. One group decided on a particular system that's also public-facing, and the group I actually support prefers a more internal-facing system. It also doesn't help that the budgets for both projects are completely separate and firewalled from each other by law. Growth opportunities don't apply here because I deal with a captive market with legally-forced customers.
Such programs are not necessarily impossible to optimize. One common optimization is to use an event loop, allowing just a few threads to handle thousands of concurrent operations. Because while a thread is waiting for I/O in one request or unit of work, in the meantime it moves on to work on processing another request/unit. Another common optimization is batching/grouping of I/O calls.
This isn't really true anymore. IO has bad latency, but modern SSD bandwidth is ~5-15GB/s. If your program is IO latency bound and processing less that 5GB/s you aren't IO bound, you aren't hiding your latency well enough.
That's nothing compared to modern memory bandwidth.
Not anymore, no. Your SSD, before any caching, does gigabytes per second of sequential reads. For any bytewise processing, except the most trivial of tasks, you’ll struggle to get above a few hundred megabytes per second with scalar (native) code. To actually keep up with a modern SSD, you’ll virtually always have to hand-write SIMD loops, minimize the number of syscalls with tools like io_uring, or possibly be smart about distributing tasks across cores without ruining the access pattern.
For instance, simdjson is famously fast but I don’t believe it can keep up with say a high-end PCIe Gen 4 SSD like a Samsung 990 PRO, let alone the latest-and-greatest (and, literally, hottest) Gen 5 stuff. And I know of no Unicode normalizer that would be able to do a gigabyte per second on general inputs (not ASCII, not Latin-1) simply because the latency for dependent lookup table accesses is absolute murder.
What your comment demonstrates is that it is possible in some cases for I/O to be fast enough to not be a performance bottleneck for certain kinds of programs. But not that I/O is not slow.
And I think you’re being unfair labelling my couple of examples “some arbitrary algorithm[s]”: my choice was indeed arbitrary, but it’s also immaterial. The general setup would be that you’re processing elements in a loop and that your iterations are serialized (as they usually more or less are before you get around to optimization). A loop body of even three lines of C is likely to have a latency of 5–10 cycles or so, and you’re running on a core clocked somewhere from 5 GHz (desktop) to half that (server). So the best you should expect is ~500 MB/s if your elements are bytes, ~2 GB/s if they’re 32-bit integers, etc. For very simple tasks (that are also somehow not susceptible to vectorization), it is possible to not lose this order of magnitude and get down to almost 1 cycle/element in scalar code, but that requires heroic effort[1].
[1] https://github.com/powturbo/Turbo-Histogram
In saying that there are some novel and very clever algorithms that continue on without seemingly necessary boundary data, that then self correct when the data comes through, thus completely hiding the latency at the cost (in both accuracy and time) of running a correction process.
I would very much prefer something written down, so I could absorb this at my own pace. I know, gift horse, but still.
I used to do manual labor and I would work my way through like eight hours of audiobooks per day.
I would like to learn to focus... (https://youtu.be/1IRn8BN3Qhw?t=331)
I used read a lot of books when I was younger, then slowly stopped, and found getting back to reading quite hard as well. Armed with the knowledge that I used to be able to do this, I just forced myself to. And after a while, I found I had no trouble committing any more.
The basic idea is that the origin assumes a highly critical inner hot loop, don't assume where it is, and optimize there.
There's some other time spent saying this justifies slower abstractions for maintainability elsewhere.
See also
https://dl.acm.org/doi/epdf/10.1145/356635.356640
I think the GOTOers just died out.
Some day null, statements (rather than expressions) and side-effects will have always been wrong.
The actual GO TO complained of is, like the jump instruction in machine code, just entirely unbothered by context. Want to go from the middle of this code about employee payroll processing to mid-way through initializing a weather simulation? No problem. Well. No problem for the machine, for a human programmer it's a complete nightmare. Actually that's putting it mildly, nightmares have more structure. You cannot do anything like that with for example C's goto.
You compiler can, and in a few cases (that's what the discussion about the tail-call optimisation is about for example) it will, but the program you wrote doesn't do this and so you don't have to try to keep the whole program in your head.
So in that sense GOTO died out with, maybe BASICs? I think the BASICs tend to have that wolf nature GOTO feature, but nothing modern has it.
However, having written a good chunk of ASM in my life. I don't think jumps or branches are really that hard to follow. Jumps/Branches and GOTOs specify the next location. It is not as though one has to guess where.
It's not the arrow, it's the archer that is the problem.
My guess is that your ASM is inflected by structured programming everywhere. Yes, unlike COME FROM we can see where we're going next with GO TO but if you go back 60+ years there is some scary code that even today's optimizers probably wouldn't emit because it's too crazy. Suppose calculate-total-fuel ends with three CPU instructions which copy register F into register H then add register C to it and multiply the sum by four. Over in locate-horizon it so happens we need to add two things together and multiply them by four and we could do that last. So, if those two things were in registers F and C we could just GO TO that last part of calculate-total-fuel.
You're correct that we don't need to "guess where" it goes, but good luck understanding why the program works when it's like this, let alone the ordinary maintenance work of making small modifications.
I think you're probably right. To expand on this:
In asm, you can have things that are clearly functions. You have a stack discipline going in and out of them. They end with stack cleanup, then a RET or some such, which pops the return address off of the stack and jumps to it. Within that function, you have JMP instructions (or whatever) that move around within the function. You may also call other functions, by pushing variables on the stack, and then calling JSR or whatever to push the program counter on the stack and jump, and when those functions return, you'll be right where you were in this function. That's all sane, and it's "structured assembly".
Non-structured assembly would be like the example in your second paragraph. You're in one function, and you JMP (not JSR) into the interior of a second function. Or, you simply don't have functions, just labels that you jump around do. That's not structured, and not sane.
(I am the GP)
Your response was far better than mine. If those instructions were executed more than a couple of times and the constraints/assertions were identical, I'd absolutely throw them in a function. I'll handle what I need to before/after the function.
I would never branch to some coincidental label. That is why I love ASM so much. If you are sloppy or lazy, you will most likely be punished severely for those choices.
(Tangential, but when I first learned ASM in college, I felt like I learned more in that one semester than all the sum of all classes in my entire degree.)
When you need to do X here and there, you write code to X and then you call it where it was useful - you don't have code in one function just jump to a label in a completely different function because, in this era where structured programming is taken for granted - that seems crazy. And it is, but only the same way that chattel slavery seems crazy today, in the Antebellum South it was just usual and likewise in 1950s computer software just jumping into unrelated code was normal.
> My guess is that your ASM is inflected by structured programming everywhere.
I am not certain what you mean.
> if you go back 60+ years there is some scary code that even today's optimizers probably wouldn't emit because it's too crazy.
Why was this as common as the allegations lead me to believe? Was it just a product of time? No better way to handle such logic at the time?
As for your example, I know it is intended to be interpreted trivially, but I just want to clarify I understand you correctly. Is the issue due to locate-horizon depending on logic from a completely orthogonal operation?
If so, I see the red flag immediately. However, I do not see an issue with creating a separate function. I'd just have to be damn sure H was not storing anything used for some other upcoming operation prior to the function call, I'd check F and C for valid state (if necessary), take other potential side-effects into account, etc.
It feels a lot like the DRY advice being abused in higher-level languages, you know? I agree with expression, "duplication is better than the wrong abstraction."
Same reason that nobody did the Fosbury Flop at the 1948 Olympics - Dick Fosbury hasn't shown this technique yet and it's not obvious if you've never seen it that this is a better way to perform an Olympic regulation high jump. Twenty years later, he does exactly that and wins Gold and by the 1988 Olympics this is how everybody does a high jump, but in 1948 nobody knows it.
Your instinct to "create a separate function" is exactly Structured Programming.
So we can jump over a variable declaration which is pretty confusing, but we can't jump into unrelated code.
Also while COMMAND.COM can't be as old as BASIC it must be pretty old.
It's also a little fluffy. It's basically the history of how the book Structured Programming came about, because it's that book that essentially caused Knuth to write an article that contained the quote.
A lot of it is fairly interesting but it also highlights a big problem I do have with Muratori. He generalizes from self. He didn't fully understand or appreciate the quote about premature optimization, so no one did. And even though he's been told what it actually means, and was probably pointed to the full quote, he would stick on his interpretation. It's likely someone he views as "higher", like Jon Blow, challenged him and then he took the time to actually look at it.
He ain't wrong though.... most people definitely don't get the jive for sure. Knuth was talking about a massively different kind of optimisation than what we do today yet people keep continuing to parrot the 97% figure uncritically like as it was some gospel. [0]
[0] http://www.joshbarczak.com/blog/?p=580
Though lately I've been uploading the audio to AssemblyAI, I somehow still haven't used up my credits after several years lol
At one point I built a system that would summarize the transcript and I'd be able to ask questions about it, but Gemini can do that natively now so I usually just use that.
https://rentry.co/2ttr46r9
It is something you can maybe whip up with the help of an LLM but then you're left wondering whether it is actually communicating what the video intended to begin with, as you wouldn't be as knowledgeable about the topic, one wouldn't be able to spot the parts where an LLM interpreted something horribly and thus reaching the wrong conclusions.
For all of the above, I do not consider audio transcript to be of any help when one prefers written content. It is useful for translation or people with hearing issues, but both of those receive help of the visual parts to convey the topic.
(to be clear, I'm a big fan of Casey)
Titles aside, his talk is really insightful and it is super interesting to do a deep dive on these old computer/programming topics as the modern concepts were being discovered
Yeah Ive been meaning to watch that talk - I love listening to pretty much anything Casey says/does. He's extremely thoughtful and fair.
Exactly!
Thank you for bringing a term to a thing that has been bugging me: people who purport to be devs but spend most of their time on podcasts and blogs talking and writing about being devs, rather than just being devs.
I’m sure you can find bugs in the work of any game dev you would consider legendary, game implementation is generally very messy
There is massive amount of knowledge in there for anyone bothering to actually learn something and it was all provided free of charge. Hats off to Casey for sticking to it as long as he did.
The craftsmanship is dubious. I think it's a problem that people assume Casey knows what he's doing when so often he's like "We're doing it live" and Casey's hand coded solution is pretty bad whereas the thing which came in the box is very good. Casey understands Casey's version, and that's an upside if you are Casey but you are not. If the result was a finished game then hey, whatever, the game was finished and that matters. But that part didn't happen either, so then it's just like watching Tsoding or something.
His second attempt, no matter how he made it, was made with the context of his earlier mistakes.
RAII doesn't lead to spaghetti code necessarily. If anything, "unity builds" would more encourage spaghetti as it's easier to reference anything else in your build.
Fine, as I recall, Doom data assets are searched linearly whenever they need to pull new data. So if you switch to the chainsaw, and need to rev, the game does a linear scan of all graphics, maps, and sounds looking for the vroom noise.
There are many different data structures that could perform this lookup faster. These have been known since the earliest days of computing.
Does it matter? No. Engineering is all about trade-offs. A linear scan was obviously fast enough and simple to implement.
Similarly, if walk monster manually trolls the map, that does not say anything about Casey in isolation. I believe all of Blow's games use a custom game engine, so unless a nav mesh system was already implemented, that was going to take additional work. The walk monster may have been better bang-for-buck.
There are some tradeoffs here, my instinct would be to bring hash tables into play and that's perhaps a mistake because it means you're doing fewer reads but more arithmetic and on some hardware that's a bad trade. But "linear search" is only the simplest and probably not the smartest option.
Casey is professionally best known for his work at RAD game tools (a highly successful middleware provider for game development back in the day), not games he himself developed.
What he is most highly regarded for is his teaching, particularly the Handmade Hero series on YouTube, which various programmers directly attribute as being responsible for their own professional success to a large extent.
He did work on the video games Dungeon Siege and The Witness afaik, but that's not what he is known for.
He also made one of the fastest terminal emulators (refterm), purely out of spite, to show Microsoft they were full of shit in their answer to an issue he posted to the Windows Terminal github. This directly led to performance improvements in Windows Terminal.
Bink was the cutscene video codec for a long time and it's still the most popular option by far. Kraken compression is so good Sony worked with AMD to make a hardware decoder for the PS5 and paid for a general license that lets games use it for free. They are probably stronger than ever in terms of games using their tech and collected license fees.
True, though they've been acquired by Epic since 2021. They keep the RAD Game Tools branding, but aren't their own separate company anymore.
Oodle/Kraken is mostly the work of Charles Bloom and Fabian Giesen, a couple of other "legendary game programmers" at RAD, but who aren't quite as visibly prominent as Casey has been in the modern software developer mediasphere.
And I never left Python, the slowest language around.
AI is only going to make the problem worse unless you know what to look for.
this shouldn't be right, wtf
Also I learned to program in Turbo C and Turbo Pascal with its debugger and nothing comes faster (at least nothing single threaded)
When did you start using computers? I'm getting older now, and some people weren't alive when I first used a computer.
It's because the older programming approach was to just tell the computer how to do the thing - and the modern approach is to write a framework inside which you write a framework inside which you write an inefficient version of the thing because it's constrained to the tools available in the framework stack.
Old-school implementation of "scroll up": copy pixels then redraw the pixels at the bottom.
New-school implementation of "scroll up": delete virtualized rows from the beginning of a DOM table and add more to the end. Change the height properties on different parts of the scrollbar. Relayout the whole DOM, several times if you're not careful. Redraw the whole screen.
I don’t know the extent to which he contributed beyond that, but he is therefore by some reasonable definition _a_ game developer.
[1] the-witness.net/news/author/casey/
Like they can't comprehend the fact that these things only happen if there is an incentive for it
I think its because they work in games where there is a business incentive for performance, users care a lot if framerate suffers in a game
They are so used to working in games that they take it for granted that the business incentive for performance exists
As someone who has worked in both AAA games and big tech it just comes across as a bit naive to me
Yes it’s incentives - but why these incentives exist and not others is an interesting question to study.
In the 2010s we pushed everyone to learn how to code and maybe lost some of our higher standards. Maybe we’re doing that now with AI?
Maybe broader social and economic incentives have had a lot of short term thinking of late? It’s hard to organize society or a company to go in one direction very long before incentives completely change everything. Who knows, maybe that’s a feature, not a bug.
It’s worth asking though why we have this software industry and not another one. I’m not convinced it’s for legitimate reasons.
Then we can talk about how we all pay the increased cost in energy and materials, and so forth.
Optimizing code for max performance does take work. But just not doing incredibly dumb things and writing simple programs just takes education (and re-education).
I can’t count how many times I’ve replaced a distributed system with for loop.
It takes a lot more work to keep performance good in a large project than just letting it degrade.
I find Blow in particular a bit naive when it comes to the complexity of modern non-game software. Games are very complex, but Blow puts them on a pedestal separate from other software, as if a web backend can't be as complex. He calls software slow while simultaneously ignoring most of the functionality.
He's essentially never had to deal with distributed systems, or anything that allows multiple users to use the same system, with all the performance and anti-abuse issues that come with that.
His opinions about game development are insightful, and his thoughts on craft can be quite inspiring, but I think he could benefit from a stint on a (good) big tech server team to see some of the things he's currently missing.
In fact, Jon played a crucial role in the invention of the first MUF worm. We had just finished an extension to MUF that permitted triggering programs wherever a text string was expected. This gave a great flexibility and functionality to run MUF programs passively, by third parties, all kinds of new ways.
Unfortunately this also gave rise to a "worm" type of MUF that replicated itself whenever it was triggered passively. We went a little crazy when this was discovered, but it was all part and parcel of a multi-user game system, where there were permissions to consider, and multiple read/write/execute access to objects that should be owned by their respective player characters.
Jon Blow also inspired a lot of my exploits as I tested other systems of MUD programmability. In that time, TinyMUSH was also experimenting with a language that was more akin to scripting, and boy was it vulnerable. TinyMOO was also coming out, with a rather revolutionary programming paradigm. So in the mid-1990s, just in the insular TinyMU* world, there were at least 3 branches of programmable MUDs being actively developed.
Hopefully Jon could recall my greatest contribution to the corpus of early MUF programs. On Atlantis MUCK, I programmed a simulation of Conway's Game of Life. Of course, there was no animation possible and there was no multithreaded execution possible, either. So my program was given a starting board of Life, and then computed each frame in sequence, output it into the text stream of the game, and terminated after N iterations. Meanwhile, the entire game server locked up, because it could only run one MUF program at a time. Everyone else's commands and input would be held until my MUF program finished running.
This was a real issue, of course, and Jon and I worked to mitigate the problems caused. I think, ultimately, we just limited the number of iterations or the number of instructions that could be executed by any one MUF program at a time.
But Jon and I collaborated on a tutorial for player-programmers, and he helped us find and extinguish many bugs. And these were very complex systems, with 64 or even 256 simultaneous players.
Perhaps Jon has forgotten some of those lessons, but perhaps not.
That said, and this is coming from someone who (sadly) missed the era of computing you're describing, I think there's still a difference between dealing with a 64-256 player multiplayer system, and scaling a server application across dozens? thousands? of machines, across the world.
My limited research suggests that MUDs run almost exclusively on a single machine (?), or even just having a small number in one location drastically simplifies the problem space.
It doesn't really contain anything new or earthshattering, but if you find diving into the history of programming as a discipline interesting, it might be for you.
I would recommend it to anyone who liked the rambling style of "The big OOPs", but imo that one was better. I still liked it.
The best thing you could take away from this lecture is something a reasonable person should take away from the original "root of all evil" saying anyway. Measure. Measure. Measure. If you aren't measuring that's not optimization it's masturbation.
Ironically in the process of measuring for a third time yesterday I tripped a bug in Bill's language for which I opened an issue. This is not the goal of measuring three times but merely a happy accident.
Along the way Casey discovers (?) that Structured Programming means just what we today call programming†, that software was a lot smaller in the days when 4096 bytes of RAM was a good entry level option and that loads of these famous people from 1970s computer science knew each other.
† And knowing about this is one reason the Structured Concurrency people want that everywhere. Very possibly there's a future where it seems silly that people once wrote programs which did not use structured concurrency.
There is some space for this "Estimate the theoretical maximum" as a high level insight, if the software needs to do a thing which your estimate says is impossible that's important to address up front but I think for practical software engineering you're much more likely to need profile-fix-repeat
Actually measuring is crucial and an estimate is not a measurement. I think one of the reasons many in the Handmade Community don't like measuring is that it too often ruins their "estimate" of how the crap they've written is achieving a "theoretical maximum" in favour of a boring fact that it's much worse than alternatives.
That is you know the data is some size, and memory throughput is some rate, and clock cycles is some rate etc...
You are entirely correct about the line of thinking if your assumptions are down a different line of thinking such as algorithmic estimates though, where you might miss some better way of doing things.
This can be applied roughly to anything. I find it very handy when thinking about CPU/GPU performance as well. For instance knowing that at 5GHz you have 5 billion clock cycles per second, which depending on instructions and pipe lining etc... can be roughly 5-30 billion operations per second per core tells you how long some process should take if you know roughly how many operations are required and the data size you are operating on.
Obviously as you drill down things get much more complicated, but they give you some rough idea about how fast things could be under some set of assumptions. Very similar to how a physicist works with models and assumptions.
1. Your hottest loop is spilling registers which only shows up as non-local cache thrashing (i.e. some other random code becomes slow) or randomly slow instructions i.e. "why is this xorps to initialise this float suddenly slow" due to pipeline stalls
2. Your code stops fitting into cache due to the code bloat, there's no one method which is slow, everything is slowed down by a percentage factor
3. A lot of useless work being done like temporary strings being copied everywhere
4. Your code is "I/O bound" because all the data you're accessing is scattered all across memory, leading to completely predictable TLB stalls
It's very easy to make a large program, quite a bit harder to make a small one...
I'm on AMD and uProf isn't that good (it often fails to match PDB to source, it's slow / crashy, stuff like that) so I mostly use Superluminal myself (has godly UX) and only drop down to uProf when I have a suspicion and want to gather hardware stats.
The problem is knowing what to measure. There's another saying
"When a measure becomes a target, it ceases to be a good measure."
As an example from memory, there was a game dev company that celebrated they had maxed out the cores on the PS3. That didn't mean anything though, anyone can max out the cores by filing them with bad code. But hey, their "measurement" told them they had maxed out the machine
This can be a problem, but much less so because so often we're doing "easy mode" where we don't need a proxy. The "it ceases to be a good measure" is because you're measuring a proxy. You wanted to deliver happiness, you measured wealth because it was easier to measure but seemed correlated and now you've got rich miserable people, oops. But software engineers can often measure the actual thing they want to improve directly, not a proxy and so it cannot cease to be a good measure.
Latency and hitching are annoying to measure, so for a long time, they were pretty much ignored. That has improved and hopefully will continue to improve. There is the bufferbloat initiative, gaming magazines take frame time histograms now, input devices and screens commonly have their latency measured. But latency is still under-measured - for GPUs / GPU drivers, for all software, for games and particularly for websites.
If your goal is fast software it can be measured quantively and you’re likely to improve the actual thing using those metrics.
Why not measure performance for which there are many good metrics.