Welcome to a new Dev Log everyone! A couple of important things have happened since the last Dev Log and, of course, lots and lots of development as well. So let's dig in ...

New Team Member: Francisco Pereira

Before we dive into the code I just wanted to announce that I'm very pleased to welcome Francisco Pereira as a new full-time member of the Phaser team. If you're on the Phaser Discord you may know him better as gammafp and he's long been helping spread the joy of Phaser into Spanish speaking climes. He joins the team thanks to a fantastic new sponsor (that I can't announce details of just yet).

His roles will be many and varied. At the moment he has been busy making sure that new issues on GitHub are correctly tagged, that forum posts are approved in a timely manner, he's been tweeting out a new Phaser showcase game every day from the official Twitter account and, most importantly of all, he has been going through every single Phaser example, building up a spreadsheet of which ones work and fail under Phaser 3.50 and then going through fixing them one by one. This is extremely important work and means when we release 3.50 we can be 100% sure that all the examples will run properly. Over time, Francisco will also start helping with Phaser 4 development, but right now both of us are heads-down and full steam ahead in getting 3.50 finished.

W3C Web Games Community Talk

On October 20th I was pleased to be invited to take part in the first W3C Web Gaming Community talk. The group, run by Tom Greenaway from Google and Noel Meudec from Facebook, aims to give a voice to web game developers so their issues and queries may be raised to W3C, the World Wide Web Consortium. The idea being we can work together to ensure that both browser vendors and standards committees are working together on the same page in order to advance what's possible for web gaming.

As part of the first meeting, I gave a talk about some of the trials game developers face when working on the web. From distribution to monetization, to the browser landscape. Although the talk was given live, I did pre-record a version of it which you're welcome to watch here. It's only 10 minutes long, so won't eat up much of your time. You can also find my slides and the minutes of the meeting at the same link.

The next meeting is in December, so I'll be sure to report back on any updates from it.

3.50 Development - Pipelines Overhaul

Since the first version, Phaser 3 has used an internal system known as 'pipelines' to handle rendering under WebGL. The idea was that different pipelines could handle different types of effects. For example, the Bitmap Mask Pipeline did, as the name clearly implies, handle bitmap masking. There was also a pipeline for lights and the main one, the Texture Tint Pipeline, handled all sprite and graphics rendering.

These pipelines feature methods such as 'batchSprite' or 'drawFillRect', which the Game Objects call in order to render themselves.

I wrote about some of the new pipeline features back in the August Dev Log, so it's worth skimming over that before carrying on.

Every Game Object in Phaser uses a pipeline to render with. By default, most texture based Game Objects use the Texture Tint Pipeline, which is now called the Multi Pipeline in 3.50. Shapes and Graphics Game Objects use the new Graphics Pipeline. You'll see if you look at the Phaser source that each Game Object makes a call in its constructor such as:

With no parameters given, the call says 'Use the Multi Pipeline for this Game Object'. Otherwise, you can specify the pipeline to be used based on any of the defaults available.

As well as the built-in pipelines, it was also possible to create your own. You could then create your own Game Objects that, instead of using a default pipeline, rendered via your own pipeline instead.

While quite flexible under the right hands, this approach was also severely limited.

At least until now :)

Pipeline Manager

New in 3.50 is the Pipeline Manager. This is a new class that exists as part of the WebGL Renderer. Its responsibility is to handle the creation, activation, updating, and deletion of pipelines, both custom and standard. When the Phaser Game instance boots, in turn booting the renderer, it wakes the Pipeline Manager which goes through the motions of installing the default pipelines and any custom ones you may have configured in the Game Config.

Previously, you had to add pipelines as part of a Scene, but you can now bundle them into your config instead:

The above makes the "Hue Rotation Pipeline" available to any Scene, right from the start, as it gets installed as part of the boot process.

Should you need to, you can still add pipelines within a Scene, but the config approach definitely makes for easier code, especially when bundling a pipeline into your final package.

WebGLPipeline Class

The base class, from which all pipelines extend, is the WebGLPipeline class. This has existed since 3.0.0 but has been given a massive upgrade for 3.50. In short, it's now a lot more flexible than it ever was before.

In previous versions, a pipeline could only have one single shader. If you only needed to add a single effect to your game, this wasn't too much of an issue. However, for more visually complex games it was a real limitation. Every pipeline you created would, in turn, create its own vertex array as well. These take-up precious memory. How much was based on the size of the batch, but a typical batch of 2048 quads with 8 vertex attributes would easily eat a MB of RAM. It doesn't sound like too much, but if you multiply that per shader, or (very likely) have a much larger batch size, it starts to eat away very quickly at what little resources you have, especially on mobile.

This needed to change. So when I refactored the core pipeline class, I also built in the ability for a single pipeline to have as many shaders as you needed, which all shared the same vertex buffer. I also made the way in which you define them a lot simpler.

Pipelines are now created via a new configuration object. One of the properties is the new 'shaders' array, where you can define multiple shaders:

Here, we've added a Grey Scale and Hue Shader into a single pipeline. Each one has its own fragment shader. They could also have their own vertex shaders if needed, but in this case, they're using the Multi Pipeline vertex shader.

Internally, as part of the boot process, a new WebGLShader instance is created for each shader in a pipeline. This is a new class available in 3.50 and it encapsulates a pipeline shader and the methods needed to manipulate it. It will automatically create and set both the shader attributes and uniforms, caching their locations and values. It also provides a whole raft of handy methods for setting all possible shader values, such as `set1f` or `setMatrix4fv`. Prior to 3.50 methods similar to these were available on the WebGL Renderer class itself. This just didn't feel like a logical place to keep them, so I have moved them into the Shader class and provided proxy methods in the Pipeline class as well.

The new Shader class instances are stored in the 'shaders' array within the pipeline. You can access them either via the array directly, or use the handy `getShaderByName` method.

The WebGLPipeline class has a huge number of new render hooks, which gives you much more control and power than previously. When the pipeline is booted, the `onBoot` hook is called. For this example pipeline I'm setting a couple of handy aliases and also two uniforms:

This hook is only called once per pipeline, so is a good place to set properties or uniform values that never change.

Each frame, the `onPreRender` hook is called. Here, you can set any uniform values that change on a per-frame basis. For example:

Here, we set the current gray value into the Gray Shader and the time and speed into the Hue Rotation Shader. This method is only called once per frame and, more importantly, there's a new uniform cache in play as well.

Prior to 3.50, if you were to set a uniform value it would do several things. First, it would set the shader program as being currently. Then, it would perform a uniform location look-up for the given uniform (i.e. 'uTime'). Using this location, it would then set the value you gave it and finally clear the program again. All of this would happen for every single uniform. It was a lot of mostly redundant WebGL operations because uniform locations do not change once the shader has been created.

As part of 3.50, I implemented a new uniform and attribute cache.

Now, if you call a function such as `set1f`, it will first check to see if the value is any different from that already set in the uniform. It does this by caching the value locally. If the value is the same then it just silently bails out, spending no WebGL operations at all in the process.

If the value is different, it uses the cached uniform location in order to set it, again cutting down on the number of operations required compared to before. This means you can now comfortably set uniform values in any of the shaders your pipeline has, at any point, without worrying about it messing with the active program or wasting WebGL operations in the process.

There are lots of hooks you can now use:

This gives you complete control over every aspect of the rendering process and makes creating your own custom pipelines much easier than ever.

Another very important new feature is the ability for Game Objects to now be able to set 'pipeline data'.

When a pipeline is set, you can provide an object with it that can contain information the pipelines can use. Here's an example:

Above, you can see we've set an 'effect' property. We can react to this within the pipeline code:

The `onBind` hook is called for every Game Object that asks to use the pipeline. In the code above I simply read to see which effect the Game Object should use. 0 for the Gray Scale effect or 1 for the Hue Rotate. You can see the results here:

We can go further, though. How about allowing you to set the grayscale factor or the speed of the hue rotation? We can expose those via the pipeline data as well:

And our updated pipeline code:

Which gives us fine-grained control over the effect being applied to the sprites:

You can run this demo here.

The pipeline source code is in the Phaser 3 Examples repo.

The above approach works fine if you don't have a large number of Game Objects that require unique values. However, if each Game Object needs, for example, a different hue speed or grayscale factor, then it becomes expensive for the GPU to keep setting uniforms like this. This is because WebGL is unable to batch together these sprites because it has to draw them between each call, as it needs to reset the uniform values.

Thankfully, this isn't the only way you can do this.

You can now also easily set the shader attributes as part of the configuration. The new pipeline and shader classes will now calculate all of the sizes for you. All you have to do is say which attributes exist, what type they are, and how many there are. Here's an example config:

We've got 4 attributes. The first three are standard ones from the Multi Pipeline for the vertex position and texture. The final one, 'inSpeed' is a new attribute used exclusively by this pipeline. By specifying the attributes here, the vertex buffer is created automatically based on them, so you no longer need to pass sizes or buffer values to the pipeline. We can then use these attributes in our shaders:

I know the code is quite hard to see in the screenshot above, so you can view it on GitHub here.

With our new attribute all that is left is to override the 'batchQuad' and 'batchVert' methods to include them in the vertex buffer:

And voila! Our pipeline is complete. To prove how effective it is, let's create a basic Scene:

Running it, we can see our 128 sprites, all using the new Hue Rotate pipeline:

You can run the demo here.

What's most important of all, though, is our WebGL debug view. Have a look at the capture of a single frame from GL Spector:

Just 7 gl operations for the whole Scene and 1 draw call. Now that's a decent performance! Had we used the 'uniform' approach it would have been 128 draw calls and I dread to think how many gl ops in total.

But, depending on what you need to achieve, you've both options open to you. If you only want an effect running on a handful of sprites, then the much easier uniform approach is available. If you have the need for speed, attributes are one of the ways forward.

However, they're not the only one as you'll soon see.

Post-Render FX and RenderTargets

The final, extremely important new feature added to pipelines in 3.50 is the ability to create both render targets and post-render pipelines.

These are brand new and super-powerful. Let's cover them in more detail.

A normal pipeline takes control of the rendering of a Game Object. For example, in our Hue Pipeline above, instead of rendering the sprites with their normal texture colors, the colors were blended and mixed, giving a hue rotation effect. But what if you wanted an effect that worked by manipulating a Game Object once it has been rendered? Such as a Blur shader for example? This is where post pipelines come into play.

A post-effect pipeline works by creating one or more render targets. These are instances of the new RenderTarget class and they're created automatically by simply adding a 'renderTargets' property to your pipeline config. A Render Target is the combination of a WebGL Texture with a Framebuffer. When a pipeline is defined as being a post-pipeline two important things happen during rendering.

If a Game Object has a post-pipeline set, then it's activated right before that Game Object renders (with whatever default pipeline it is using for rendering). Then, after the render is over, the post-pipeline is automatically invoked again. In the default set-up, a post pipeline will redirect rendering to its own Render Target. Then the Game Object will render, using whatever pipeline it has bound. Finally, the post pipeline will then draw its Render Target to the game canvas and is finished.

This allows for some really smart effects. For example, effects such as blur, distortion, vignettes, CRT overlays, outlines, glows, pixelation, god rays or wave shaders can all take place as post-pipelines. Let's create a simple example from scratch, so you can see how it functions in practice.

First, we'll create our pipeline. This will consist of a vertex and fragment shader:

Our pipeline has just two attributes and we override the 'batchVert' method as before:

And that's it. You can see in the configuration object that it will create a single Render Target. We'll save our pipeline as 'BarrelPostFX' so we can import it into our example:

It should be quite clear what's going on here. We create 3 sprites using our 'catstick.png' and the one in the middle has our barrel pipeline set as its 'Post Pipeline'. Notice the call to 'setPostPipeline' instead of 'setPipeline'. That is all you need to do in order to give a Game Object a post-pipeline.

Running the demo, we get this:

The tween is updating the 'barrel' value, causing the sprite to bulge in and out every couple of seconds.

This is the kind of effect where you need to be able to overdraw the sprite's original texture frame, which is why it works well as a post-pipeline. You have a few different Render Target settings you can play with. The first is the scale. By changing the scale you can control how large the framebuffer is. Some effects can be quite expensive for the GPU to process, so it's often better for them to run in a lower-resolution, so they complete quicker (and have fewer fragments to process). You can tweak the scale property to control this. For example, setting it to 0.5 would make a framebuffer half the size of the renderer dimensions.

The minFilter property allows you to set the WebGL minFilter Texture property. By default it's 0, which is 'linear', but you can set it to 1 for 'nearest', or any other valid GLenum. Combine a small scale with the nearest min filter gives you a nice chunky pixel effect:

Finally, you can set 'autoClear' to true or false. By default, you may want the framebuffer to be cleared between frames, but if you're trying to create a trail effect or similar, it may be better to leave the previous contents rendered and blur it out over time. This property allows you to do this.

While it's really great to apply a post-pipeline to a Sprite, I also spent a good chunk of time building a frame buffer stack into the renderer, allowing you to use post-pipelines on Containers, as well.

This is where things start to get really powerful. The Container will render all of its children to its post-pipeline framebuffer. Then it draws those children back to the game via whatever shader you like. Using our barrel pipeline, we can apply it to a Container and get some fun results:

This is as simple as creating a Container, populating it with some sprites and then calling:

Because of the way the post-pipelines work when all of those sprites are rendered, they're still fully batched. They're just batched to a different framebuffer, that's all. We can see the above example running in Spector GL:

Just 2 draw calls, one for all the sprites to the framebuffer and one for the framebuffer to the game canvas. Using a post-pipeline on a Container doesn't have any negative impact on the rendering performance of the children. Of course, if you have a really complex post-fx shader, then it will take time to render that, but on the whole, this is a really neat way to throw lots of effects into your game quickly, with minimum impact.

What's more - post-pipelines work with nested Containers, as well :)

You can see this demo running on YouTube.

What you can see in the video above are 2 Containers. One contains all of the spinning diamonds. This has our Hue Rotation pipeline running on it, that we created earlier. Then we've got the 3 cat stick sprites. These are in another container, along with the diamonds container, and that has our barrel pipeline running on it:

As you can see, having a Container with a post-pipeline, inside of another Container, can lead to some powerful combinations of effects! Add to this the ability for a single pipeline to have multiple shaders, pipeline Game Object data, and multiple Render Targets and you've finally got a whole suite of tools with which to create whatever effect you care to think of.

Oh, and did I mention, you can now apply post-pipelines to Tilemap Layers, too? :)

You can run this demo for yourself (use WASD to move the car)

Before I forget, someone asked me on Discord the other day if you can apply shaders to the new Mesh Game Object, which I covered in detail in the previous Dev Log.

The answer is, of course, yes :)

Watch this YouTube video to see it in action!

I'm really happy to have finally landed these new pipeline features into Phaser 3. It's been important for quite a while to me, so I'm pleased to say it's available right now in the Beta 10 release and I'd urge you all to have a good play with it.

Download v3.50 Beta 10

As you can see, Beta 10 yet again contains lots more changes than the previous beta. There's lots more than I've covered above! I could really do with your help testing it, though. The more eyes-on it gets, the more solid the final release will be.

The new v3.50 Beta 10 is available from both npm and GitHub.

You can get it from npm using the beta tag:

Or download it from here: https://github.com/photonstorm/phaser/releases/tag/v3.50.0-beta.10

You'll find pre-built bundles to download in the dist folder, or you can check out the master branch and build yourself.

Note that it does not have updated TypeScript defs yet, so if you want to use those, please pull down the repo and use `npm run tsgen` to build new ones locally.

If you find an issue report it to me either on Discord, or (even better) open it as an issue on GitHub and tag it "3.50 Beta 10".

With the new pipeline system in place, I can now finish off what I needed to do with the Lights system. This is the final part of the jigsaw before I can start packaging the 3.50 release. I hate giving deadlines, but if I had to, I would say that my current objective is for 3.50 to release at the end of November, and then I'll spend all of December creating lots of cool examples for the Patreon Backers Packs, to finish off 2020 with a real bang.