Monthly Update #52🎃

Happy Halloween, Sleepyheads! It’s been quite a busy month with lots going on. And we had a week of the most beautiful, picture perfect autumn weather, but I unfortunately missed it all due to illness. 😥 Okay, plenty to talk about today, so I’ll get right into it…

Non-Porting Updates 📰

Aside from the work on the Switch port, here’s a quick update on the goings on this month:

RetroGameCon Follow-Up

We kicked the month off by showing off Little Nemo at RetroGameCon. It all went really well with loads of people stopping by to try out the game, buy a “physical” copy of the Steam game (keycode on a keychain), or just say hello.

A photo of me standing in our RetroGameCon booth

As you can see, we had a nice set up with custom printed banners, two demo stations, a book of the original comics to show off to those unfamiliar with Little Nemo, and loads of keychains to sell.

A photo of our desk at the booth

Thank you to everyone that stopped by! We had a blast and it was great to meet all of you that were able to attend.

Personal Life Stuff

I also had a wedding to attend and did some traveling two weeks later. It was great to get to see friends, but unfortunately, by the time I got back home, I was pretty sick 🤧. I think this was the first time throughout working on Little Nemo that I got properly sick. I’m actually still in the process of getting through it, and it was bad enough that I couldn’t work at all last week.

So ultimately I didn’t get to work on the Switch port this month as much as would be ideal because I spent a lot of time pre-occupied and dealing with illness. But with that said, I did get a good chunk of work done this month, and what I worked on was very informative in terms of the Switch port timeline. So let’s get into that:

Switch Port Timeline 📅

Last month when I talked about the porting process, I tried to break down a lot of the optimization work that I’ve been doing into general categories. But most of that work was very CPU-centric. This month I’ve been tackling the GPU optimizations, and there’s a lot to dig into here, but the thing that it’s made clear is that it’s going to be more difficult and take more time than anticipated.

I also talked about the possibility of pushing the Switch release into early next year, and this month’s work made it clear that the Switch port won’t be ready by year’s end. So I’m going to try to find an appropriate window to release early next year. Once again, I’m terribly sorry this is taking so long, but hopefully it’ll start to make more sense as you read on about what I’ve been up to:

GPU Optimizations 🛠️

A quick bit of game dev context for those of you unfamiliar: modern consoles and computers typically have two types of processors inside, a CPU and a GPU. The CPU handles your game’s logic while the GPU does the work of actually drawing everything into a buffer that will eventually wind up on screen. These two processors operate mostly independently, but we need both to finish their workloads in under 16.6ms to hit a steady 60fps.

So in last month’s update when I talked about various things I was doing to make the game more performant, that was mostly to get the CPU under control. But this month I’ve been focused on the GPU performance, and I thought it would be nice to give you a close look at what I’ve been doing since it’s a whole different type of work:

Background Performance

The backgrounds in Little Nemo tend to be quite visually complex, formed from many parallaxing layers of sprites. Because we have so many layers, all with varying degrees of opacity, the Switch’s GPU starts to struggle to draw all of these sprites in.

Early on when developing these backgrounds, my focus for performance was mostly on sprite batching (that is, arranging the sprites to get drawn all together in one draw call). But that doesn’t mean the GPU isn’t still doing a lot of work to actually draw them all in. And we wind up with a symptom that is called “overdraw”, which means for a given pixel on the screen, there might actually be several sprites all contributing (or potentially contributing) to the end result for that pixel. That is to say more simply: pixels where lots of sprites overlap become a lot more expensive to render.

To give you an idea of scale, just the background alone for Dreamswept Plains (the sky, some clouds, the Palace in the distance, the mountains, and the rolling hills) was taking over 30ms of GPU time each frame (and remember, we need to get below 16.6ms to render everything). Since everything is already batched into one or two draw calls, we’re just hitting a limit of how long it takes for Unity to draw its Sprite Renderers on the Switch’s GPU.

So there were no easy optimizations to be had here. And I really don’t want to sacrifice the visual quality of the game by reducing the number of sprites and parallaxing layers, so I had to come up with a fairly drastic solution.

Uber Background Shader
The solution is to bypass Sprite Renderers entirely by creating a single shader that renders the entire background all together. I can just apply this shader to quads (effectively just rectangles) that go in each room.

A Unity editor look at the background drawn as a shader, the two overlapping quads are both using the same shader

As you can see in the video above, the quads with this shader applied work effectively like “windows” looking into the background, which is exactly how we want them to function (with the old Sprite Renderer approach we actually had to use Sprite Masks to limit the visibility like that so that you couldn’t see adjacent domains’ backgrounds).

So how exactly does this all get done in a shader? I don’t want to get too lost in the weeds on this, as it does get fairly complicated, but I’ll try to give a good high-level overview of what exactly is happening.

Material Manager
First off, because this shader is fairly complicated, we’re going to need a manager that runs logic (which runs on the CPU of course) and prepares all of the data that the shader will need. This manager will also be a good place for us to put all of the data we need for the background. Since we won’t be using a prefab, we’ll need to create our own data structure that will describe where each instance of a sprite should be located, how much parallax depth it should have, and how frequently it should repeat, along with other details.

The Inspector Panel where you can see the details the Manager needs so the shader can get all the parameters needed

Each frame, we have some logic that needs to run to calculate things like converting a parallax depth and sprite position, to the apparent position. It’s important that we do as many of these calculations as possible here on the CPU so that they only happen once per frame. Any work we do in our shader will happen once per pixel.

One of the important details that our manager will pass into the shader for each sprite is the sprite’s Atlas Rect. This is the rectangular portion of the Sprite Atlas texture within which the sprite appears.

A screenshot preview of the Sprite Atlas used for the Dreamswept Plains background

Since all of the sprites are all jammed together into one texture that the shader will sample from, we have to have a rectangle that describes how to “find” each sprite. That’s what our Atlas Rects are for.

The Shader
Once our shader gets the details needed about each sprite we want rendered, it needs to do some math so that we can effectively answer the question of: “what color does pixel x, y correspond to on our Sprite Atlas for sprites a, b, and c.” How exactly we do this is beyond the scope of this update, and it’s hard to succinctly explain, but it’s not as much math as you might think. For repeating sprites (which is most of them), we can do a little extra math to account for repetition at the desired interval, and for rotating sprites like the rolling hills, there’s also a little extra math involved.

Here’s the Shader Graph for the Dreamswept Plains background shader

If you take a look at the background shader’s Shader Graph (although the core of this shader’s logic is written in high-level shader language, I like to use Shader Graph to get everything wired up), you can see how we have a node for each of the 27 sprite instances that make up our background.

These nodes are all wired up front-to-back (sprites nearest to the camera first) because it’s important that we skip as much work as possible, and if a given sprite fully paints in one of our pixels, we know that we don’t have to sample anything else behind/after it.

So ultimately, the flow for the shader looks something like:

For each pixel we need to render, let’s start with a completely clear RGBA color (value of 0 for each of the four channels, resulting in a transparent black). Now pass this color into the first of our 27 sprite nodes, give that node an opportunity to update the color, and then pass it on to the next, until we’ve gone through all of them.

At each node, first we’ll check if the alpha channel is fully saturated (a value of 1, or close enough to it), and if it is, don’t do any other work. But if it’s not fully saturated yet, we need to check if this sprite needs to contribute anything to it, so let’s convert our sprite’s apparent position and size into UV coordinates and see if this pixel falls within those coordinates. If so, we’ll sample the texture accordingly and add that sample to this pixel’s color.

It’s as simple as that, but importantly, we’re giving ourselves as many opportunities as possible to avoid doing as much work as possible by doing the easiest stuff first. Checking alpha saturation is easy, but getting UV coordinates for a sprite is a bit of work, and finally sampling the texture is a lot of work.

The Results?
So after building the background with a shader, and doing lots of optimizations to it from there, we got this background from over 30ms down to about 8ms. That’s still pretty hefty, but that’s a test scenario where it’s covering the entire screen, and most importantly, fits into our 16.6ms budget.

So I was able to prove that I can render my backgrounds much more cheaply on the GPU by using custom shaders. By doing this one for Dreamswept Plains, I’ve done most of the hard work, but I’ll need to convert the backgrounds for each of the other domains as well (all while profiling to ensure they each remain under budget).

Other Shader Optimizations

I’ve also been taking a look at some of my other shaders to find performance optimizations. The Oblivion shader, for instance, was very expensive on the Switch. I found a screen full of Oblivion could take over 20ms to render. Even though it’s typically only covering a portion of the screen, that’s way too expensive, so I started digging into why. After some trial and error, I discovered almost all of the expense was from using Shader Graph’s “Gradient Noise” nodes. Noise is an important way to add random variation to things, such as the gentle undulation the Oblivion does. By simply pre-generating noise textures and using those instead, I was able to get that same full screen of Oblivion from 20ms down to under 5ms.

Here’s a pre-baked Perlin noise texture I’m using now instead of procedurally generated noise, this texture can be used to create some surprisingly interesting effects

I likely have similar optimizations that need doing in the other shaders throughout the game. Luckily I don’t actually have too many different shaders (most sprites stick to one of a handful of relatively simple ones), but things which are plenty fast on non-mobile hardware can easily become huge bottlenecks on the Switch’s GPU.

If this stuff is at all interesting to you, make sure to come join our Discord, there’s a #game-dev channel in there and it’s nice when y’all come in there with questions. I’m happy to get into some of the finer details there if anyone is interested. Come chat me up!

Kickstarter Friends 💚

At RetroGameCon earlier this month, our booth neighbor was Tyler, who is making an action-adventure game for the Game Boy Color called Essencelord. There were even Game Boy Colors on-hand so I could give it a go. They’ve already reached their funding goal for the Kickstarter, but there are less than 24 hours to go, so take a peek and see if it’s something you’d be interested in supporting (there’s even a link to the demo you can try out for yourself).

Essencelord is a loot-driven action adventure game for the Game Boy Color. You’ll travel across the diverse world of Oren to uncover the mystery of Essence all while fighting monsters and solving puzzles.

That’s It For This Month 👋

Okay, that’ll do it for this month. I hope everyone reading has a wonderful Halloween, whether that means donning a costume and going out, just staying in with some candy and a horror movie, or something else entirely. Enjoy our one last day before we get to November and Spooky Season comes to a close! 🎃👻 Thanks for reading, Sleepyheads. See you next month!

-Dave