Today I went to see Nouvelle Vague (2025). It depicts the filming of Breathless (orig: À bout de souffle), an important example of French New Wave cinema, and does so in the style of the time. Director Richard Linklater has said that he intended the film to seem as if it could have plausibly been shot by another director in the movement working at the same time.

I’ll here continue the ongoing tradition of breaking my own rule of not reviewing the movies I watch by saying it was good! I liked it!

In the movie, long-time film critic Jean-Luc Godard takes on the task of directing his first movie, which is to serve as “the greatest criticism” by depicting the way he believes a movie should be. His distinct approach to filmmaking is jarring to the industry veterans that he works with, putting him at loggerheads with both the producer and star. In the end he succeeds at working the way he wants to and releases the movie to acclaim.

That’s a very familiar shape of story. I’ve seen a whole passel of movies and books starring the brilliant underdog who knows just how to do the thing right, if they can just overcome the stick-in-the-mud academy/teacher/boss who stands in their way. For some examples, see Skyward, Amadeus, Harry Potter and the Methods of Rationality, and 17-year-old Zane in his fantasies. And yet despite fitting that shape narratively, Nouvelle Vague carries none of the righteousness that I would have thought was necessary to the trope. It does not feel like Godard is a genius and the producer a bozo, and there is no lasting enmity or redemptive moment of “you were right Mr Godard, we see the light now”. What’s left is a very healthy depiction of normal people having differences, getting frustrated, and showing up to work anyhow.

Godard has a strong vision for what he will film and how his crew will make it happen, and he is rarely willing to compromise with his coworkers. At the same time, he does not need to be the smartest person in the room, or be seen as having all the answers. If he has no ideas to film on the day, he cancels or cuts short production. If his tooth hurts and he can’t think of anything to put to film, he sends everyone home. He’s perfectly content to sit in the café with his notebook and papers trying to come up with what comes next, while the cast and crew sit around drinking, chatting, and dancing. That would embarrass me to tears! I flush just thinking about it. I’ve cut Dungeons and Dragons sessions short for the same reason, and it always feels like failure. Something to learn from Godard, then.

A model behavior too, is the way that the members of the movement discuss each other’s films. In the opening scene Godard and friends watch the new movie from Georges de Beauregard. At the party afterward, Beauregard asks Godard what he thinks, and Godard tells him the film is terrible. Beauregard replies cheerily that Godard is his favorite film critic, and the two have a drink. This resilience in the face of criticism is the domain of a man comfortable in his own worth. Quiet confidence: not the fragile arrogance that everything you make is golden, but the acceptance that you are not your work, and a failed act does not a failed man make.

That same confidence, focused in attention but detached from shame, is I think what gives this Godard his power to accept his flubs and sail through chaos to the finish line. I’m not sure how one disentangles those shame-fibers that have grown through their mind-cloth. An ongoing project, in my case.

Today’s graveyard section is short, just Good Luck, Have Fun, Don’t Die. Eh? Meh? Not a lot interesting to say about it. Someone said once that a movie should pass the “refrigerator test”: that is, if you’re enthralled by the magic moving picture at the theater, and don’t think of a plothole or objection to the movie until you’re at home staring into your fridge’s cavernous receptacle of sandwich meats, then the movie has succeeded. Fridge thoughts don’t count. GLHFDD did not pass the fridge test. Both predictable and incoherent. Still, a fun modern B-movie if you just want some spectacle that touches on contemporary themes.

That’s all for today! May you find the self-assuredness of the New Wave in your own world.

Zane

Yesterday I went to see the new Project Hail Mary adaptation. It was good! And I don’t have much more to say about it. It was an effective adaptation of the book, and a fun and visually striking movie in its own right. Ryan Gosling brings a lot of charisma and humanity to his role as Dr. Grace, the grade-school science teacher trapped in space. Apart from one scene in the middle climax where the drama wasn’t justified (why could they remote-trigger the ball drop, but not the winch rewind?) all the pieces fit together cleanly and tell a tight story. Like my friend said afterward, it’s two and a half hours, but it really doesn’t feel like it.

As much as I enjoyed it though, I was left afterward with very little to ponder or talk about. Why is that? I don’t think that it’s that the movie is bad, or unenjoyable: when we went to see The Bride, which I think doesn’t quite hold together as a movie despite doing a bunch of things well, we were left for days afterwards picking over its bones and thinking about what it did right and wrong. Nor do I think it’s that Hail Mary is too good: Marty Supreme, The Testament of Ann Lee, and Silent Friend are all great movies from this year that left me with connections aplenty to think about and talk about.

I’m a bit stumped here. I find myself tempted to say things like “well, maybe it’s because it didn’t have a Big Concept”, but I think this is just me hiding my incomprehension in words that I can’t define, or picking another synonym for the thing that I’m failing to explain.

And perhaps I should caveat that this could just be a me thing. Ars Technica put out a pretty extensive article discussing the linguistic issues of first contact and how they’re portrayed or ignored in the movie. So clearly someone else found some meat to chew on! Maybe I didn’t have the right receptors or thought-connections for this movie to light my sparklers.

Still I think that some movies are just talking about more than other movies, or talking in a way that is itself novel. That novelty, or at least novelty-for-me, is a big part of what I’m looking for in all media. A fun game is good, but a game that is fun in a way that I haven’t seen before is way better.

What do you think? What kinds of movies get you talking?

Here below the fold I lay out a graveyard of movies that I watched, and might have had something to say about, but did not get around to blogging about. Please take my silence on these movies as my own failure to sit down and write, and not a criticism on the movies. Most were great!

• Winter in Sokcho: A girl in Sokcho, Korea has never known her French father, who left before she was born. A French painter moves in to the guesthouse she works at, and becomes the canvas for her feelings.

• Un Poeta: A poet, alcoholic, and former professor struggles to put his life back together and bring a budding young talent to light.

• Hoppers: Avatar, but for beavers.

• The Bride: Frankenstein’s monster brings a woman to life to be his Bride. Little does he know she’s also haunted by the ghost of Mary Shelley.

• Silent Friend: Do plants speak? Could we possibly understand each other?

• Dust Bunny: Mads Mikkelsen, high-end hitman, refuses to take on the monster under your bed.

• Rental Family: Brendan Fraser will pretend to be whoever you need him to be. Can false pretenses create real connection?

Oh, in other news I read Akira Kurosawa’s Something Like an Autobiography. It was good! Thanks to Carl for the recommendation — if you’re looking for skilled designers to help you with your design issues, give Frog House a ring. Kurosawa recounts scenes from his life from earliest memories in the 1910s through to his directing Rashomon in 1950. And what a life! Walking hours in the morning to take kendo lessons from master swordsmen, witnessing the horror of the Great Kanto Earthquake, working as a painter in the silent film era. He saw and felt a lot, and lays it out for the reader with the sensitivity and awe for natural phenomena that would become characteristic of his directorial style.

That’s all I have for today. Until next time!

Just watched Wuthering Heights (2026). Holy cow is that movie horny. In retrospect, that should have been no surprise from the director of Saltburn! Emerald Fennel, chill out, man.

In the movie, Catherine lives with her father, who is both a gambler and a drunk, in their house on the moors. One day he adopts a poor boy of her age, who she names Heathcliff. The two grow up together and fall in love, but just as they think to do something about it she marries the new neighbor, the wealthy Mr Linton. Heathcliff runs away, returns five years later rich, and the two have an affair that brings the movie to its climax. A bit like Great Gatsby, but a century earlier.

I was struck by the visual contrast between scenes filmed out on the wild moors, in fog and rain and some tepid sun, and those shot in the controlled lighting of a soundstage. The mansions of Catherine’s father and Mr Linton show off a fairytale absurdity: strawberries the size of your head; wallpaper painted to look like skin; a fireplace lintel made of stucco hands; a pile of liquor bottles ten feet high. Sometimes they play this for a laugh, but mostly it shows the grotesquery, idiocy, and wretched taste of the hyper-rich. The stark wild beauty of the moors stands in contrast as the site of Catherine and Heathcliff’s many trysts: the truth of one’s desire is plain, and harsh, and unforgiving.

There’s something going on with the maximalist aesthetic these last few years. Poor Things (2023) features similiarly over-the-top decor, and especially the design of the mad scientist.

Bugonia (2025), another Lanthamos joint, also shows bulbous flesh/cloth sets in the last chunk of the movie. I’ll say no more about that one. The earliest I can remember seeing this style was in the hotdog-fingered wackiness of Everything Everywhere All at Once (2022).

It’s hard to put my finger on what this aesthetic is. Visually I think it tends to be round, fleshy, and saturated. It’s deployed to different effect in different movies, but it always seems to convey a sense of overstuffed absurdity -- a feeling which I also got from the opening scene of No Other Choice (2025). This scene introduces our main character as the family man, husband and father to two, having a typical Sunday barbeque. Their dogs, a pair of golden retrievers, patiently wait for scraps. His wife opens her birthday gift: a pair of golden dance shoes. The family comes together for a group hug, awash in the literal golden light of this picture-perfect moment. Though aesthetically it couldn’t be more different from the fleshy world of the other movies described, something rhymes in the way it layers elements of its ideal one over another until the whole thing feels like it must crash down like a jenga tower made of jello.

Is there a term for this style? I wonder if they have named themselves. Until they do, I dub them henceforth: fleshglorp. Long live the fleshglorp.

The Testament of Ann Lee (2025)

Gorgeous! Compelling! Human!

The story is a little thread of americana woven tightly into the dense tapestry of northeastern American history. These — the Shakers — were always a small group: nine made the crossing to New York, at peak they numbered two to four thousand, and today they are just three. And yet they live on in the many clever and beautiful things they made.

In high school we sang Simple Gifts, one of thousands of Shaker hymns. My parents have what I now believe to be a set of Shaker chairs, or at least chairs made in the Shaker style. They’re light, comfortable, sturdy, and seem to be endlessly repairable.

The cultural legacy of this small, hardworking group is remarkable. I’m thankful to Mona Fastvold and her cast and crew for giving me a peg on which to hang these details of my life.

Obex (2026)

This is an odd one. It felt very much to me like two different movies. The first half is an instrospection into a life of isolation that felt unsettlingly familiar to me. There are real emotions here, and it gives you the time to feel your way through them, while also building up to the midpoint switch.

The second half is a simple adventure movie. Everything and everyone exists to support the plot. Emotions are simple and stated aloud. All threads resolve by the end. The writing here had only as much depth as the ancient Mac games it was emulating.

Maybe there’s something in that contrast? Our main character is a shut-in who won’t even open the door to talk to the woman who goes grocery shopping for him. His dreams are nightmarish reverberations of some childhood trauma. His only means of dealing with the challenging emotions of everyday life is to drown them out in media overload. The two-part structure could be an intentional reflection of this life: the overwhelming trials of the everyday set against the straightforward formula of the media that we use to escape it.

I dunno, man. I’d rather sit with the complex emotions for the whole 90 minutes. For a movie that does that, check out Still Walking (2008). A man, his wife, and their son visit his parents to commemorate the anniversary of his brother’s death.

Alternatively if you just want to see some wacky shit happen in black and white, try Hundreds of Beavers (2022).

The Last Viking (2025)

Mads Mikkelsen in a heartfelt role = instant recommendation every time. The man is in a lot of Hollywood shovelware — like seriously, did anyone expect a hit from Indiana Jones and the Dial of Destiny? — but when given a role with depth he fills it admirably.

Not really sure what to say that would add to this movie, so I think I’ll leave you today with two other Mads Mikkelsen movies that really show what he can do.

In Another Round (2020), he plays a high school teacher who agrees with a few of his colleagues to chase the Ballmer peak: maintain a blood alcohol level of 0.5% during work hours, under they hypothesis that it will improve their lives.

In The Hunt (2012), he plays a divorced kindergarten teacher who is falsely accused of sexually abusing a child, with disastrous consequences.

That’s all we’ve got today! I will try to go see some more movies this weekend to catch up with what’s coming out here, and workshop some more how to fit a trip to the theater into my work week.

To all you popsicle-fingered folks in the US, I hope you are staying warm and melty indoors.

See you soon!

Short one today. Did you know rabies takes one to three months from infection to first symptoms? The virus spends that time hiking up your nerves from the site of infection to the brain.

In countries where rabies is common in dogs, the vast majority of cases in humans come from dog bites. Thankfully in Europe and the United States there is no longer a stable circulating population of rabies in dogs. Score one for vaccination!

One last odd rabies story before I go.

In 1978, a hiker walking through the Rhone River valley in the Swiss Alps might have been surprised to stumble over a chicken head. Chicken heads, laced with live rabies vaccine, had been spread across the valley in an attempt to halt an oncoming tide of rabies. Scientists hoped that foxes would eat the bait, thus immunizing themselves and forming a living barrier against the disease. The plan worked, and hope for a new solution to the rabies problem grew.

Chicken heads are messy and hard to get in massive numbers, so scientists developed a sort of fox dessert wafer made of fishmeal, bonemeal and fat, with an imbedded plastic packet of rabies vaccine. The wafer was a hit with foxes, and since 1983 over 5.2 million baits have been distributed. Over 70 per cent of foxes in the target areas take the bait and become immunized. This means that there are no longer enough susceptible animals in these areas to continue the chain of transmission.

Today Switzerland is virtually rabies-free.

Taken from The World of the Fox by Rebecca L. Grambo.

Hey, what the hell, are those vampire teeth?!

Maybe? I’m still figuring it out.

It’s clear from the outset that the movie is About Something, but the continually unrolling mystery resists total analysis. You can catch fragments of symbolism and themes that fit together like a collage of mirrors, for me never quite resolving into a unified image.

This movie-going experience can be uncomfortable. It’s relaxing to see a movie that speaks in clear metaphors, like Nausicaä, say. And it can be satisfying to watch one that initially appears blurry before snapping into focus in the final moments, giving a feeling akin to the final scenes of a puzzle-box mystery like Last Night in Soho. If you’re expecting an experience like either of those, the ambiguity that a movie like Resurrection immerses you in can make you feel stupid, like you’re just not getting the point. It’s a little like buying a jigsaw puzzle, but getting a box of watercolors. Like what is this bullshit! What’s the intended way to put these together?

But if you release the movie from the obligation of communicating a single clear point, these movies can provide some of the richest experiences out there, extending off the screen into conversations afterward as you try to piece together not what the movie meant, but what it meant to you.

For that reason, I loved watching The Boy and the Heron when it came out two years ago. It’s a movie overflowing with themes and things that could symbolize emotions or events in the real world, but each angle of analysis I threw at it could only account for some different fraction of the events of the movie. Nothing I could think of would unify the many bits of strange synthetic folklore that Miyazaki balled up into the movie. In the end, my tin-foil hat theory is that the movie’s resistance to analysis reflects the unexplainability of the horror that is war.

I’ve never seen anyone else talk about this theory, and I doubt that it was Miyazaki’s true motivation in making the work what it is. But I had great fun grappling with what I saw and heard and felt and trying to anneal it into a crystal of thought.

---

Best wishes to you in this second week of the second quarter of our century. May you find your peace with, what the heck, another immortal vampire born in 1601, where in the world are these things coming from, Artemis did you leave the back door open ---

Today I watched It Was Just an Accident, in which a guy thinks he’s found the intelligence agent who tortured him in jail, now years after his release.

I’ve now told you the whole plot of the first movie, but if you’re gonna like it I don’t think that matters. And yet I don’t dare tell you anything more about what happens in the second for fear of ruining it for you. What the heck? How can two excellent movies be so different in this way? What’s the difference between movies you can spoil and movies you can’t?

I think that it has something to do with what’s interesting about the movie. A fractional list of ways a movie can be interesting: It can have shots that convey some symbolic meaning, like the crime boss in Tazza standing at Jesus’ place in The Last Supper and looking over at Judas. It can have shots that are simply wordlessly moving, like gosh, any shot of the sea in The Outrun. It can make you itch to know something, like a good whodunit. It can make you curious to learn more about a mysterious world. It can make you laugh. It can take you to a place that you don’t want to leave, or let you spend time with characters you think are clever, or cool, or hot.

Some of these kinds of fun are wholly reliant on surprise, like the whodunit itch, and some are totally separate, like the moving landscape shots. Others land somewhere in the middle. A joke is less funny the second time -- with some rare exceptions -- but I think you’d be hard pressed to tell me any of the jokes in the Naked Gun in such a way as to spoil them. You could describe them, yeah, but they’re still going to be funny when I see them in the movie.

I would divide them into two categories: experiential and analytical. Here by experiential, I mean things like feeling an emotion, looking at a richly textured image, or watching an expression play out on an actor’s face. These are complex experiences with a ton of detail that’s hard to put into words. They stand in contrast to the analytical: the events in a story, the meaning of symbols, or a fact about the world. These may be complicated, but if we sat down and talked through them we could get all the information across. For whatever reason, experiential fun is harder to talk about, unless both people have a shared experience that they can point to. If you’re both lying there looking at the clouds, you can just say “that one looks like a nose” and the other person will get to have the same perception as you; but go home to tell your partner about the cloud that looked like a nose and, try as you might, they won’t really get it.

Because experiential fun is so hard to convey in words, a movie that relies mostly on it is unspoilable. The point of watching Hamnet is how the pictures, gestures, and expressions build up an arc of feeling in you bit by bit over the course of the movie. Know what you may about the plot, these elements will still have their impact. Analytical fun, though, is all about words! Symbols, meaning, and plot are all tied up in and made out of language, and so by using language we can defang their fun. In It Was Just an Accident the crux of the fun of the movie is the tension of what will come next. Did our main character get the right guy? Will he run from him? Capture him? Kill him? The open question of the plot creates tension that would fall apart if I told you what comes next.

Most movies incorporate fun from both sides of the spectrum. Even the examples I’ve chosen do this: Hamnet’s second half is probably more impactful if you don’t know quite how it plays out, and It Was Just an Accident has plenty of beautiful shots that can delight even if you know where the story is going.

So take some comfort the next time you see a trailer that shows too much. The story might be spent for you, but the telling is still to come.

The film follows Marty Mauser, a world-class ping pong player desperate to be the greatest. The story centers on his conflict with Endo Koto, the player for Japan, and Marty’s struggles to be able to attend the 1952 British Open, and later the World Championships. In character, Mauser’s narcissism and monomaniacal obsession with his sport recall Chalamet’s 2024 depiction of a young Bob Dylan in A Complete Unknown. In structure and emotional tone, the movie reminded me a lot 2019’s Uncut Gems, the panic attack of a movie starring Adam Sandler as a gambling-obsessed gem dealer trying to recover a lost jewel. Now that I look at the credits, that makes a ton of sense: Uncut gems was directed by the Safdie brothers Josh and Benny, and cowritten by them and Ronald Bronstein; Marty Supreme has the same credentials, minus the involvement of Benny Safdie.

I was really struck by how historically charged the atmosphere of the movie was. 1952 was just seven years after the end of World War II, and many of the characters wear scars of the time: tattoos from Auschwitz, hearing loss from the Tokyo air raids, the painful memory of a lost son. My picture of the “postwar period” is dominated by the Korean war -- the product of a childhood spent rewatching M*A*S*H -- so I found novel this picture of a world of damaged people heaving a collective sigh and going back to work.

The dramatic final match takes place in Japan in 1952. I didn’t know much about this period of Japanese history, so I did some reading up. After Japan’s surrender on September 2nd 1945, the nation entered a seven-year period of American occupation. During this time the civil government still operated separately from the occupying army, but underwent dramatic changes at their demand including the institution of a new American-written constitution. Among other changes, Article 9 of this constitution banned the country from maintaining an army of its own. In the first years of occupation a primary goal was to demilitarize the country by removing 200,000 wartime officials from government posts and restructuring finance and industry to make it less able to support a military. Rising Cold War tensions and American concerns over the spread of communism in the far east led to the reversal of much of that work.

In 1952 the US-Japan Security Treaty went into effect, which along with the Treaty of San Francisco ended the occupation, reestablished Japan as a sovereign nation, and created the military alliance between the US and Japan that lasts to this day. These were hotly disputed among the members of government and the people of Japan, who variously believed that Japan: should not continue to be occupied by 260,000 US troops; should not risk being drawn into US-Soviet conflicts; and should not create a standing army. Widespread protests occurred throughout the country, including the Bloody May Day Incident. In 1950 criticism from Joseph Stalin had caused the Japanese Communist Party to reverse their policy of peaceful protest and attempt to foment revolution. Members and supporters of the party infiltrated the peaceful union May Day protest and incited a violent confrontation between protestors and police. In the ensuing chaos 2300 people were injured, and 2 were killed.

It is into this uncertain world that Marty inserts himself in the movie’s third act. While the plot doesn’t delve into the local politics, the tension between the people of Japan and the American GIs is on full display. If you’re interested in seeing a contemporary Japanese perspective on this period, the 1960 film Dry Lake (aka Youth in Fury) follows a university student entangled in the mass protests against the revised security treaty. The director, Masahiro Shinoda, is one of the greats of Japanese New Wave cinema. My two favorites of his are Demon Pond, an adaptation of a kabuki play by the same name, and Silence, a movie following two Jesuit priests who illegally enter 16th century Japan to spread the word of Christianity.

If you like the ping pong angle but want something a bit more heartfelt and less cynical, I highly recommend the incredible Ping Pong the Animation.

That’s all for this first Something. I wish you all a happy new year, and may you see many movies that make you say “I’m an immortal vampire born in 1601.”

The idea came to me earlier this year when I was trying to work out how big the Korean movie industry is. This is a surprisingly hard thing to do! There are some articles on the internet with fuzzy estimates that disagree with each other, in the neighborhood of 100-500 movies.

Part of the problem is getting the data: to my knowledge there’s no single publicly available database which lists all movies that have released in a theater. IMDB presumably has much of that information, but surfaces it in ways that (1) obscure the answer and (2) make me nervous that they might be missing a large chunk of releases.

Another part of the problem is definitions. What’s a movie? Do documentaries count? What about recordings of live performances? Does it have to be 90 minutes? Does a 20-minute short by Wes Anderson count? What about Leo Dicaprio’s 4-hour epic of the year? What does it mean to come out in a theater? Does showing just once in a private screening count? If not, then how many times does it need to show? In how many theaters? For that matter, what constitutes a real theater? Home theaters obviously don’t count; what about museums? What about private clubs? The questions go on and on, with every one dialing the movie count up or down. Trying to answer all of these seriously seems like great bait for pedants, but I’m not much interested in really getting an answer. Instead, the question has become a jumping-off point for my project of 2026.

Let’s set some ground rules. For me to count it in the challenge, a movie has to be a new release showing in one of the 14 Munich movie theaters1. I won’t be including documentaries. Nothing against a good documentary, but I’m most interested in exploring fiction movies this year. Biopics and historical dramas are fine, though. The new releases rule is partly to avoid blowing up the movie count, and partly so that what I watch can be an even covering sample of what’s released with minimal editorial influence from any source of recommendations. I expect that there will be some edge cases around foreign (ie. non-German) movies that first released abroad in a recent year, and are now being released in Germany for the first time in 2026. I’ll deal with these individually as they come up. The spirit of the law here is to see them if they’re new, and I don’t want to get bogged down in planning for every eventuality.

For each movie that I go see I’ll write something here on this blog. I’d like each one to include something interesting I learned from the movie, whether that be about the form of movies, the subject matter, or whatever else the experience makes me think of. I will definitely include my subjective experience, but these are not meant to be reviews in any traditional sense. And lastly, I’ll do my best to avoid spoilers and flag them when they come up.

Some questions I’ve gotten when pitching this to friends:

Oh god, why?

I love a crazy project. When I graduated from college I cycled the length of the US Pacific coast, from the Mexican border in San Diego all the way to Vancouver, Canada. 2000 miles, and 6 weeks on the road. Now, this is arguably not crazy: lots of people do it every year, and the route is popular enough to be covered by multiple guidebooks, including the excellent Bicycling the Pacific Coast. While I was on the road I met up with a friend of my dad’s who had just completed an even longer cycling trip from New York City to Oregon, some 3000-odd miles, and was killing time by riding down the coast to San Francisco. But still, the thing has a romance to it and, as I found out when I told people what I was planning, a real shock value!

The crazy project serves as a nice reminder of what’s possible when you put your mind to it. Completed, it gives you a good reason to believe that you can do the next crazy thing. Uncompleted, it stands as a half-finished story or a fond memory. Either way, I find I learn something about myself.

And I definitely have something of a defiant streak. The more people tell me something’s impossible, the more I want to try it :-)

Okay, but why this project?

How do you choose what to watch?

For me, movies on my watchlist come from some mix of personal recommendations from friends, suggestions and mentions from internet people I follow, posters I see around the city, and sometimes whatever’s at the top of Critierion Channel. This works well to help me find movies I like: Trakt tells me I watched 54 movies in 2025, and looking back through the list there’s only one or two that I didn’t like.

At the same time, I do worry about the influence of advertising. In the most utopian view, advertising is a way to pair good products with people who would like them. In the more dystopian view, it’s an attempt by corporations to persuade consumers to buy something they don’t need or want. In the extreme, an algorithmic feed has 100% control over what you watch within the feed. If TikTok doesn’t want to show you some content, you will never see it unless linked to it from outside. Surrendering control over your attention like this is fun, but concerning: a meaningful chunk of who we are comes from what we watch, read, and think about. Giving up your attention to the algorithm means giving up part of your ability to decide who you are. On the internet you can regain control by subscribing to RSS feeds and newsletters from individual people and brands. What then can we do about movies?

Now, obviously the situation at the theater is not at so dire as with infinite feeds. We don’t sit down in the movie theater with a blank ticket and watch whatever they think we’d like (mostly -- some theaters have tickets like this for advance screenings) (also hmm isn’t this how a lot of people watch Netflix?), we think about it, talk with our friends, and then consciously decide to go to a specific movie. That choice is influenced by advertising, yes, but also by recommendations from real human beings.

Still, I wonder: what’s out there that I’ve never heard about?

By watching every movie that comes out I can step through the mystifying veil and get a real, total understanding of what’s being made, and hopefully find a lot of interesting stuff that no one I know is talking about.

What about direct-to-streaming movies?

1. Too expensive to get subscriptions for all the platforms.

2. Too hard to understand what’s being released.

3. Good grief, isn’t 100-500 movies enough?

Man, you’re gonna watch a lot of baaad movies.

Ain’t that the truth.

My hope is that even a bad movie can inspire an interesting thought. A movie with flat characters can still have an interesting premise, or teach me something about story structure, or have a neat shot or series of cuts.

As well, what I think is bad may be someone else’s favorite movie. Sometimes that’s because of irreconcilable personality differences, but sometimes it’s just because I haven’t figured out how to enjoy the thing. For a long time I found roguelike games boring. I would play them for hours until I got frustrated enough to quit, and never pick them up again. When Hades came out in 2020, I tried a different approach: each morning I played two runs, maybe an hour of game time, and then I put the game down and went about my day. Short sessions helped me avoid burning out on the game, and daily play helped me build up skill. Over the course of the next couple months I put in >40 hours, and completed the story. Having fun is a wide skill. You might be good at it in some situations, and bad at it in others. I intend to try to have fun, even with movies that I initially don’t like.

I do reserve the right to skip kids’ movies if I get tired of them though.

But what about the cost?

A few theaters in Munich offer monthly or yearly subscriptions. The Yorck theaters offer a year of unlimited movies for ~240EUR. That pays off if you watch two movies a month. Neo Kinos has a deal with the same price, but for 8 viewings a month. I’ll probably use one or the other, and pay for an individual ticket if there’s a movie that I can only see at a different theater.

Isn’t it just too many movies?

Well, it is a lot of movies. At the upper end of 500 movies a year, that’s more than one every day and about 10 times as many as I saw the previous year. At an average 2hrs per movie that comes out to 1000 hours over the course of the year, or about half of a full-time job. It will take up a big chunk of my free time, and is guaranteed to restructure my social life (such as it is).

Too many? I guess we’ll have to wait and see.

Follow along here on the blog. First something will be coming new years’ day.

I’ll soon be starting a new job at Bedrock Energy, where I’ll be working on accelerating some physical simulations that we use to design an optimal field of boreholes for geothermal heat pumps. These sorts of simulations, using the Finite Element Method (FEM), essentially come down to solving the equation Au=b, where A is a known matrix, b is a known vector, and u is the vector you’re trying to solve for. You could solve this by finding the inverse of A (assuming it’s invertible) and computing u=A^(-1)*b, but for an n x n matrix inversion is an O(n^3) operation, and the matrices we deal with are quite large. So practical techniques involve an optimization process: start with a guess for u, compute Au and compare to the known target b, and adjust your guess to produce a better guess. That process involves a lot of repeated matmuls, so it’s important to be able to do them fast.

An aside: in practice the matrix A is usually very sparse, with most entries being 0. For that scenario the optimal storage and matmul algorithms end up being quite different than the general case. In this blog post we’ll ignore that, and just look at dense matrices. I may look at sparse matrices in a future post.

And one last thing before we get into the meat of the optimization. When I heard “matrix multiplication”, my first instinct was “can we do this on a GPU?” GPUs are linear-algebra crunching machines, thanks to ~35 years of optimization for games and ML workloads. The issue in our case, though, is the high cost of data transfer from RAM to GPU memory. In the optimization cycle described above, each time we try a guess for u we need to send that vector over to the GPU. It’s my impression, although I haven’t yet done a careful evaluation, that the time taken by data transfer would eat up the gain from faster matrix multiplications.

So I’m left wondering: how fast can we get matrix multiplications on modern CPUs? Let’s find out. To give a goalpost to aim for, we’ll benchmark against Eigen, which is in my experience the linear algebra library that everyone uses.

1. A naive implementation

We’ll start with the simplest implementation we can think of. We’ll store matrices at rest in row-major order. This means that the values are in one big NxM array, walking through the matrix from left to right and top to bottom, in the same order that your eyes are reading through this text.

struct RowMatrix {
  size_t rows, height;
  f32 *values;

  RowMatrix(size_t width, size_t height)
    : width(width), height(height)
  {
    values = (f32*)aligned_alloc(128, sizeof(f32) * width * height);
  }
  ~RowMatrix() {
    free(values);
  }

  inline void set(size_t row, size_t col, f32 value) {
    values[row * width + col] = value;
  }
  inline f32 get(size_t row, size_t col) {
    return values[row * width + col];
  }
};

The matrix multiplication will also be as simple as possible. Matrix multiplication is defined such that if C = A*B, then C_ij = sum_k(A_ik * B_kj). We’ll implement that directly with a triply nested for-loop.

void naive_matmul(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      f32 acc = 0.0f;
      for (size_t k = 0; k < A.cols; k++) {
        acc += A.get(i, k) * B.get(k, j);
      }
      C.set(i, j, acc);
    }
  }
}

Done! How’d we do? On a 512 x 512 matrix of f32s, our naive implementation was 0.03x as fast as Eigen. Oof!. I’ll notate this as a speed of 0.03E, where 1E is the speed of Eigen, and 2E is twice as fast.

2. Transpose first

So how can we improve on that? Let’s think about memory first. When you issue a load instruction, the CPU will first check the L1 cache for the data before repeating with the L2, then L3, and finally issuing a fetch from RAM. The caches are split into cache lines, which are usually 64 bytes these days. When the fetch misses the cache and falls through to RAM, it will pull up a whole cache line’s worth of consecutive memory at a time, replacing some line which was already in the cache. This helps to keep the number of lines for the CPU to check small, and also helps make the ~400-cycle long fetch worth it.

This is relevant to the inner loop of our naive implementation. In that k-loop we’re iterating over a row of A and a column of B. Since our matrices are in row-major format, the elements of that row of A are consecutive in memory. When we load the first 4-byte element, we actually pull 64B/4B=8 elements at once into the cache, making the next iterations of the loop quicker to run. The elements of the column of B, however, are spaced out in memory, so each fetch from RAM can only pull one into cache at a time.

Under the hypothesis that the extra fetches are affecting our runtime, we can try first transposing B so that when we’re in our hot loop those fetches can run smoothly.

void naive_matmul_transp(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  RowMatrix BB(B.cols, B.rows);
  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      BB.set(j, i, B.get(i, j));
    }
  }

  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      f32 acc = 0.0f;
      for (size_t k = 0; k < A.cols; k++) {
        acc += A.get(i, k) * BB.get(j, k);
      }
      C.set(i, j, acc);
    }
  }
}

This speeds up our matmul by 1.5x, bringing it to 0.04E. Instrumentation shows that the transpose is just 1% of the runtime, and the multiply 99%.

3. SIMD!

Next let’s try some SIMD. I’m on an Apple M1 chip, which is ARM, so we’re looking at the Neon instruction set. Neon allows us to do loads, adds, and multiplies of four f32s with one instruction, as well as a bunch of other stuff that we won’t get into here1. As long as the chip has enough floating point hardware, this should allow us to speed up our matmuls by a factor of 4.

To understand the code sample, let’s go through some Neon basics. A float32x4_t is Neon’s C type for a 128-bit vector of four f32s. We can put them in variables as normal, and the compiler will make sure to use the vector registers instead of the normal ones. vmulq_f32 and vaddq_f32 each take two float32x4_ts and produce the elementwise product and sum of the two vectors, respectively. So vmulq_f32({1,2,3,4}, {5,6,7,8}) → {5,12,21,32}, and vaddq_f32({1,2,3,4}, {5,6,7,8}) → {6,8,10,12}. vaddvq_f32 takes a float32x4_t and adds the component f32s together, producing a single f32 sum.

#include <arm_neon.h>

void matmul_1(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  RowMatrix BB(B.cols, B.rows);
  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      BB.set(j, i, B.get(i, j));
    }
  }

  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      float32x4_t acc = {};
      for (size_t k = 0; k < A.cols; k += 4) {
        float32x4_t *ap = (float32x4_t *)(A.values  + A.cols  * i + k);
        float32x4_t *bp = (float32x4_t *)(BB.values + BB.cols * j + k);
        float32x4_t product = vmulq_f32(*ap, *bp);
        acc = vaddq_f32(acc, product);
      }
      C.set(i, j, vaddvq_f32(acc));
    }
  }
}

I’ll note here that this implementation assumes the dimensions of your matrices are divisible by 4. It’s not hard to extend the matrix implementation to make sure that this is always true internally by padding the edges of the matrix with zeros. For brevity, I won’t bother to do that in this post.

Using Neon gives us a 4.5x speedup, bringing us to 0.18E. I’m not sure how we do better than 4x there. If you have an idea, I’d be eager to hear about it in the comments.

4. Loop unrolling

I tried a few things next, none of which improved the performance: moving the pointer computation outside the k-loop; incrementing the pointer instead of issuing a p[k]; using fused multiply-add in the form of vfmaq_f32. The next improvement came from a little loop unrolling:

void matmul_6(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  RowMatrix BB(B.cols, B.rows);
  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      BB.set(j, i, B.get(i, j));
    }
  }

  for (size_t i = 0; i < C.rows; i++) {
    for (size_t j = 0; j < C.cols; j++) {
      float32x4_t acc_1 = {};
      float32x4_t acc_2 = {};
      float32x4_t acc_3 = {};
      float32x4_t acc_4 = {};
      float32x4_t *ap = (float32x4_t *)(A.values + A.cols * i);
      float32x4_t *bp = (float32x4_t *)(BB.values + BB.cols * j);
      for (size_t k = 0; k < A.cols; k += 16) {
        acc_1 = vaddq_f32(acc_1, vmulq_f32(*(ap++), *(bp++)));
        acc_2 = vaddq_f32(acc_2, vmulq_f32(*(ap++), *(bp++)));
        acc_3 = vaddq_f32(acc_3, vmulq_f32(*(ap++), *(bp++)));
        acc_4 = vaddq_f32(acc_4, vmulq_f32(*(ap++), *(bp++)));
      }
      C.set(
        i, j,
        vaddvq_f32(acc_1) + vaddvq_f32(acc_2) +
        vaddvq_f32(acc_3) + vaddvq_f32(acc_4)
      );
    }
  }
}

This bumped us up by 1.8x, landing us at 0.32E.

Loop unrolling like this generally helps improve performance by reducing the number of loop instructions to be executed and allowing more loop iterations to run in parallel. Exactly how many times to unroll is a priori a mystery; profile on your target platform to find the right number. For my machine, 4 hit the sweet spot; 2 or 8 were both slower.

Also slower was using f32 accumulators instead of the float32x4 accumulators. I guess the vector reduce vaddvq_f32 is slower than the vector add vaddq_f32.

5. Apple Silicon AMX

The next big idea is to use a newer, bigger form of SIMD. Apple Silicon has an undocumented instruction set known as AMX, and brilliantly reverse-engineered by Peter Cawley (corsix@). I’ll be using his header file to get access to AMX intrinsics throughout this post. AMX gives you enormous 512-bit vector registers for operands and lets you drive a huge amount of floating point hardware to do a 16x16 vector outer product with a single instruction. Bram Wasti has some great visuals to illustrate how bonkers huge these registers are compared to Neon in his post on jott.live.

The idea then is to rearrange our matrix multiplication to be a bunch of outer products. This takes a couple of insights. The first one is that a sub-block C(a:b, c:d) = (AB)(a:b, c:d) = A(a:b, :) * B(:, c:d). The second is that any matrix multiplication AB can be factored as the sum of the outer products of matching rows of A and columns of B. Combining these two insights, we can compute a 16x16 block of our result matrix C by finding the right 16 columns of A and rows of B, and then iterating along the k-axis computing outer products of pairs of 16-float vectors.

#include "amx/aarch64.h";

inline void transpose(RowMatrix &dst, RowMatrix &src) {
  for (size_t row = 0; row < src.rows; row++) {
    for (size_t col = 0; col < src.cols; col++) {
      dst.set(col, row, src.get(row, col));
    }
  }
}

void matmul_amx_1(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  RowMatrix AA(A.cols, A.rows);
  transpose(AA, A);
  RowMatrix CC(C.cols, C.rows);

  // Enable AMX instructions.
  // Have to do this or the program will crash.
  AMX_SET();
  // Loop over each 16x16 block in C to compute it.
  for (size_t block_row = 0; block_row < C.rows; block_row += 16) {
    for (size_t block_col = 0; block_col < C.cols; block_col += 16) {
      // Only set for k == 0
      // to reset the matrix-accumulator Z.
      u64 reset_z = 1ull << 27;

      for (size_t k = 0; k < A.cols; k++) {
        // Load 16 floats from AA into X register 0.
        AMX_LDX((u64)(AA.values + (k+q)*AA.cols + block_row));
        // Load 16 floats from B  into Y register 0.
        AMX_LDY((u64)(B.values  + (k+q)*B.cols  + block_col));

        // if (reset_z) Z = 0;
        // Z += outer_product(X, Y);
        AMX_FMA32(reset_z);

        reset_z = 0;
      }

      // Read the 16x16 floats from the various Z registers
      // into the correct block of CC.
      for (u64 i = 0; i < 16; i++) {
        u64 z_reg = (i * 4ull) << 56;
        AMX_STZ(z_reg | (u64)(CC.values + (block_col + i)*CC.cols + block_row));
      }
    }
  }
  // Disable AMX instructions.
  // Have to do this because AMX_SET() is not reentrant.
  AMX_CLR();

  transpose(C, CC);
}

This gives us a huge 10x improvement, bringing us to 3.27E.

6. Bigger blocks, fewer loads

We can further improve our utilization of the hardware by doing a 32x32 block of C at once. The advantage here comes from improving the ratio of loads to outer products: previously we did 2 loads to compute 1 outer product, whereas now we do 4 loads to compute 4 outer products. This cuts our program’s total loads in half, saving memory bandwidth and cutting the runtime.

void matmul_amx_2(RowMatrix &C, RowMatrix &A, RowMatrix &B) {
  RowMatrix AA(A.cols, A.rows);
  transpose(AA, A);
  RowMatrix CC(C.cols, C.rows);

  AMX_SET();
  size_t BLOCK_SIZE = 32;
  for (size_t block_row = 0; block_row < C.rows; block_row += BLOCK_SIZE) {
    for (size_t block_col = 0; block_col < C.cols; block_col += BLOCK_SIZE) {
      u64 reset_z = 1ull << 27; // only set for k == 0

      for (size_t k = 0; k < A.cols; k++) {
        // Load 32 floats into X and Y registers 0 and 1.
        f32 *aa_addr = AA.values + k*AA.cols + block_row;
        AMX_LDX((0ull << 56) | (u64)aa_addr);
        AMX_LDX((1ull << 56) | (u64)aa_addr + 16);
        f32 *b_addr  = B.values  + k*B.cols  + block_col;
        AMX_LDY((0ull << 56) | (u64)b_addr);
        AMX_LDY((1ull << 56) | (u64)b_addr + 16);

        // Do a 32x32 outer product as 4 16x16 outer products:
        // [ X0@Y0  X1@Y0 ]
        // [ X0@Y1  X1@Y1 ]
        AMX_FMA32(reset_z | (0ull << 20) | (0ull  << 10) | 0ull);
        AMX_FMA32(reset_z | (1ull << 20) | (64ull << 10) | 0ull);
        AMX_FMA32(reset_z | (2ull << 20) | (0ull  << 10) | 64ull);
        AMX_FMA32(reset_z | (3ull << 20) | (64ull << 10) | 64ull);
        reset_z = 0;
      }

      // Read the 32x32 outer product out of the Z registers.
      for (u64 i = 0; i < 16; i++) {
        u64 reg = i*4ull;
        AMX_STZ(
          load_store_2
          | (reg + 0) << 56
          | (u64)(CC.values + (block_col + i +  0)*CC.cols + block_row)
        );
        AMX_STZ(
          load_store_2
          | (reg + 0) << 56
          | (u64)(CC.values + (block_col + i + 16)*CC.cols + block_row)
        );
      }
    }
  }
  AMX_CLR();

  transpose(C, CC);
}

This gives another ~2x improvement, for a total of 6.19E.

7. Faster transposes with SIMD

When we started we justified ignoring the matrix transpose because it took just 1% of the runtime, whereas the naive multiply took 99%. Over the course of this post we’ve cut down that multiply by a factor of >200x, so it’s probably time to look at the transpose. Profiling now confirms: our two transposes together are 66% of the runtime.

We can do a 4x4 transpose with some creative use of Neon byte-juggling instructions.

inline void transpose_4x4_block(f32 *dst, f32 *src, size_t rows, size_t cols) {
  float32x4_t row0 = vld1q_f32(src + 0*cols); // 0 1 2 3
  float32x4_t row1 = vld1q_f32(src + 1*cols); // 4 5 6 7
  float32x4_t row2 = vld1q_f32(src + 2*cols); // 8 9 a b
  float32x4_t row3 = vld1q_f32(src + 3*cols); // c d e f

  float32x4x2_t mix01 = vtrnq_f32(row0, row1);
  // 0 4 2 6
  // 1 5 3 7
  float32x4x2_t mix23 = vtrnq_f32(row2, row3);
  // 8 c a e
  // 9 d b f

  float32x4_t out0 = vcombine_f32(vget_low_f32(mix01.val[0]), vget_low_f32(mix23.val[0]));
  vst1q_f32(dst + 0*rows, out0); // 0 4 8 c
  float32x4_t out1 = vcombine_f32(vget_low_f32(mix01.val[1]), vget_low_f32(mix23.val[1]));
  vst1q_f32(dst + 1*rows, out1); // 1 5 9 d
  float32x4_t out2 = vcombine_f32(vget_high_f32(mix01.val[0]), vget_high_f32(mix23.val[0]));
  vst1q_f32(dst + 2*rows, out2); // 2 6 a e
  float32x4_t out3 = vcombine_f32(vget_high_f32(mix01.val[1]), vget_high_f32(mix23.val[1]));
  vst1q_f32(dst + 3*rows, out3); // 3 7 b f
}

Using this as a primitive, we can build up a bigger transpose by walking over the matrix and doing 4x4 block transposes.

// B = A.transpose()
inline void transpose_4x4(RowMatrix &B, RowMatrix &A) {
  f32 *dst = B.values;
  f32 *src = A.values;
  size_t rows = A.rows;
  size_t cols = A.cols;
  for (size_t j = 0; j < cols; j += 4) {
    for (size_t i = 0; i < rows; i += 4) {
      transpose_4x4_block(dst + j*rows + i, src + i*cols + j, rows, cols);
    }
  }
}

This speeds up the transposes by a factor of 3x, so that they’re now just 35% of the runtime of the matmul, giving us a 1.8x overall speedup and putting us at 11.06E.

8. Transposes with AMX

And, one more idea: let’s do bigger transposes at once by using AMX. The AMX_EXTRV instruction treats the Z registers collectively as a square matrix of cells, each holding a few values, and moves a column from Z into a single Y register. By loading rows into Z, moving columns into Y, and then reading those columns out to memory as rows, we can transpose a 16x16 block of f32s with 48 instructions: 16 loads, 16 moves, and 16 stores.

inline void transpose_16x16_block(f32 *dst, f32 *src, size_t rows, size_t cols) {
  AMX_SET(); {
    for (u64 row = 0; row < 16; row++) {
      u64 z_reg = row*4;
      AMX_LDZ((z_reg << 56) | (u64)(src + row*cols));
    }

    for (size_t row = 0; row < 16; row++) {
      size_t z_column = row * 4;
      AMX_EXTRY(
        AMX_EXTRV_PLAIN_ZTOY
        | AMX_EXTRV_LANEWIDTH_32BIT
        | (z_column << 20)
      );
      AMX_STY((u64)(dst + row*cols));
    }
  } AMX_CLR();
}

// transpose_16x16() is then very similar to transpose_4x4(),
// just moving over 16x16 blocks and calling our new kernel.

This chops our transpose time down by a factor of 4.2x, so that it’s now just 12% of our total matmul time, improving our total time by 1.36x, and leaving us at a final 15.03E.

Over the course of this article we took an initial naive implementation of matrix multiplication and incrementally improved its runtime by 500x by improving cache coherency and making use of all the intrinsics available. Our final implementation was 15x faster than Eigen. That was a nice surprise! Eigen makes use of SIMD, but evidently it does not try to use Apple Silicon’s AMX extensions. It would be interesting to see what it would take to patch Eigen to make use of these extensions when they’re available.

I set out on this exploration to learn how fast matmuls work and to try using SIMD intrinsics. I’m happy to report that I learned a great deal. One more piece of evidence in favor of Matthias Endler’s recommendation to reinvent the wheel.

A great thing about this project has been how at every point there are several more threads to pull on to learn more. Some that I’m left with here at the end:

• patch Eigen with to use Apple Silicon AMX

• repeat the optimization process for sparse matrices

• translate to x86 SIMD, and explore Intel’s AMX

• adjust the naive code to convince the compiler to auto-vectorize it

• figure out why that SIMD change gave a 4.5x speedup instead of just 4x

• do more careful profiling with hardware performance counters to verify or disprove our guesses about why each step helped

• get hard numbers for the cost + benefit of doing the matmuls on the GPU.

  • when does it start making sense? when the matrices are large? when you’ll reuse them some number of times?

• explore some of the further optimizations listed in the Algorithmica HPC post on matmuls

Not sure which of these I’ll try. Until next time!

Resources

• simd.info

• How to Get 1.5 TFlops of FP32 Performance on a Single M1 CPU Core

• @corsix/amx

• Algorithmica HPC - Matrix Multiplication

• What happens inside Eigen, on a simple example

But first! Since we’re talking about performance optimization I feel obligated to get up on my soapbox and say: Don’t try to optimize your code until you’ve looked at a profiler! Performance optimization is a sharp tool. Don’t play with it when the lights are out. Okay, down off my soapbox now; let me show you some cool stuff.

My goal with this work was to improve the load time of our dashboards. That’s the time between when you hit enter in browser’s URL bar, all the way until all the graphs on the page had all the data drawn to the screen.

After doing some profiling, I’d discovered that parsing the timeseries data we were getting from the backends took 40% of the total page load time. In one case that parsing was taking 20ms per chart, or ~1000ms for the whole page. The total amount of time would vary depending on the size of the dashboard and amount of data being loaded, but we’ll use those numbers as a reference for later improvements.

Depending on your background and your standards for your tools that might sound like a lot of time, or nothing at all. How should we feel about 20ms? To quote Dick Sites:

Someone walks into my office and says “My program is too slow.” After a pause, I ask “How slow should it be?”

Let’s do the math. One of our charts is showing 50 timeseries, each of which has 60 points. Each point consists of an 8-byte floating point value and an 8-byte timestamp. In total, then, a chart’s data must be at least 50*60*16B ~= 47KiB. If our wire format were perfectly arranged such that we could just move the data from the response into an array, we would expect to be limited by memory bandwidth. The famous Latency Numbers Every Engineer Should Know by Jeff Dean and Peter Norvig quote 1MB of sequential memory reads as 250µs, so we’re looking at a best case of 250µs / 1MB * 47KiB ~= 10µs. Our real parsing takes 20,000µs instead! That gives us 2000x room for improvement.

So what the heck are we trying to parse, then? A look at the network tab shows that a single point is encoded as this lovely JSON blob:

[{
  "3": 3.141592653589793116,
  "1": [{
    "1": "1745311200"
  }]
}]

Some digging around in the codebase turns up this protobuf message definition:

message Point {
  Timestamp timestamp = 1;
  string string_value = 2;
  float64 float_value = 3;
}

message Timestamp {
  int64 seconds = 1;
  int32 nanos = 2;
}

Another (internal) tool called JSPB is then producing JSON serializers from these messages. Parsing is handled by a JSON.parse(), followed by the JSPB-generated parser walking the JSON tree to produce an object tree that looks more like the proto messages, followed by our code doing another copy of the data into a third format. There’s a lot to dislike here!

• The point is 53B (minimized), instead of the optimal 16B.

• Each message is encoded as an array with an object inside of it.

• There are two nested messages, instead of just one flat one.

• The int64 seconds field is encoded as a string. This is unfortunately necessary if you’re putting it in JSON, because the JS number type is a 64-bit float, which can only represent up to 53-bit integers accurately. JSON doesn’t support bigints.

• The floating point value is encoded in 20 bytes of ASCII decimal, because JSON, instead of 8 bytes of binary.

We could keep ragging on it, but at this point it’s clear: we can do better!

Let’s try the simplest possible encoding. As described above each point is an 8B timestamp and an 8B floating point value. We’ll encode these directly on the wire back-to-back in little endian, so that our “parser” can just read them directly. In the header we’ll put a 2B version number for the format so that we can upgrade it later, a 2B point type to choose between double- or string-valued points, and a 4B count of the number of points in the message. Together that looks like this:

+------------+------------+-------------------------+
| 2B version | 2B type    | 4B num points           |
+------------+------------+-------------------------+
| 8B timestamp                                      |
| 8B float value                                    |
| ... N times                                       |
+---------------------------------------------------+

We’ll also support a version with string-valued points, where the 8B float value is replaced with a 4B string length and 4B string offset into a data block that lives at the end of the message. Including error checking and comments, a JS parser that handles both versions is about 60 lines of code.

So how did we do? A run in prod shows this is 10x faster when both are compiled and minified, and 30x faster when running in a debug build. Great!

…but we can do better.

We’re encoding an 8B timestamp for every point. Our timeseries backend guarantees that the data it produces is evenly spaced in time, as long as there are no gaps. We can take advantage of this by encoding only the timestamp of the first point, and the time between consecutive points. We can indicate a gap by inserting points with a value of NaN. Our new format looks like this:

+------------+------------+-------------------------+
| 2B version | 2B type    | 4B num points           |
+------------+------------+-------------------------+
| 8B start timestamp                                |
| 8B time between points                            |
+---------------------------------------------------+
| 8B float value                                    |
| ... N times                                       |
+---------------------------------------------------+

This packed format is more efficient for data without gaps. The sparse format is better for data with a lot of gaps. We can get the best of both worlds by having the serializer check on the fly which version would be faster to decode, and encoding to the faster-to-parse format.

Measuring again, this is now 16x faster than the original when both are compiled and minified, and 50x faster in debug builds.

There’s certainly more we could do to improve parsing: NaN-boxing a number of gap points; run-length-encoding for timeseries with constant values; adjusting the rest of our frontend code so that we can reduce the number of allocations for our list of points; etc. But we should stop here.

Our original goal was not to optimize parsing, but to optimize the page load. Now that we’ve improved parsing by 16x in prod, it’s no longer a big fraction of the total page load time. We’re at the point of diminishing returns, and our optimization effort is better spent elsewhere in the system.

So let’s celebrate!

• Our original page that took 1000ms to parse now takes just 62ms.

• Parsing the points now takes less time than parsing the rest of the metadata about the timeseries.

• Where parsing took 40% of the page load before, it now takes just 4%.

Why is the new format faster? Are there insights that we could apply more generally?

I have some hypotheses as to why. Each of these is a true statement about the code that would affect the performance, but I’m not sure what the effect size is in each case.

• The new code copies the data just once, instead of three times.

• The new code does just one allocation per point instead of 7+.

  • again, would be great to also dodge this allocation

• The new code is MUCH easier to branch predict.

  • The old code has to do JSON parsing plus a tree walk. The JSON parser has at its core a switch statement that checks what the next token is. That’s inherently unpredictable for the CPU!

  • The new code in contrast should give <= 3 branch prediction failures for the whole timeseries, outside of allocations. Those would be when checking whether to parse string values or float values, and at the start and end of the per-point for loop.

• The new wire format is smaller, and so may have better cache locality.

Those are good technical reasons. But I think there’s a more general reason why the new code performs better: It does exactly what we need, and nothing more.

Protobuf and JSPB are great tools. They solve a general, hard problem well:

• they auto-generate serializing and parsing code

• for complex, deeply nested messages

• where the schema changes often

• and the serializer and parser might be using different schema versions.

But each of these features has a cost, and in this case we just don’t need them.

• Writing serializing and parsing code once didn’t cost that much: just ~80loc;

• points are simple, and don’t nest;

• the schema hasn’t changed in 4-5 years;

• and given that cadence, we can manage release of new versions manually.

More broadly, it’s important to recognize that convergence, ie. applying a common tool to your problem, is a double-edged sword. It has some great, and well-written-about benefits:

• many tools will support the common thing

• there’s a team working just on that thing

• etc.

But it has series downsides that are underappreciated in my corner of the tech world:

• it rarely does exactly what you need, and adapting it has a cost

• it often does something else too that you still have to pay for

• you have to learn how to use it

• changing its behavior is hard, because you can’t afford to break other users

• and your mileage may vary, but often the team working on the tool won’t fix your bugs and may not accept your patches because you’re just one user, likely using it in a strange way they didn’t predict.

Neither approach is a perfect fit for every situation. Like most things in software buy versus build is a choice with tradeoffs that you need to critically evaluate yourself. But please don’t assume that because someone else wrote it, it’s the best fit for your problem. Don’t be afraid to make something new.