Autotiles, instancing and object clusters

new approach: using instancing
New approach: using instancing
attempted approach: using autotiling
Attempted approach: using autotiling
original approach: a sprite per cell
Original approach: using nothing

The problem

Some of the blocking tiles in the game are things like a tree, a cactus, etc. Occasionally, I want to use these tiles to represent blocked cells in an outdoors map. But, if I just put one such sprite per cell, the result looks poor (see bottom image above, “original approach”). So I thought, “ok, let’s try to create an autotile version of the trees”.

In the meantime, I’ve developed some helper tool to assist with creating autotiles (rug/fence/blob) from a selection of input tiles:

Autotile tool: blob

… So I hacked a bit of that code away, to automatically place sprites that respect the edge restrictions, so effectively automatically creating the autotile blob from any single sprite. Example output:

Autotile tool: blob, automatic placement based on edges

While I was super happy initially, I soon realized that it would only work under very specific circumstances (symmetric sprites, placed appropriately at particular spots), and in order to cover all scenarios , I would need to automatically create a lot more sprites. So, after seeing a lot of restrictions, I wanted to go for plan B, and reuse some code that I already have for the overworld. That code uses Poisson disk sampling to create instances of things to populate the overworld.

Sprite shader refactoring

The problem was that that shader was restricted for the overworld vegetation, so I needed to generalise. I took a hard look of the miscellaneous shaders that I’m using for sprites (anything that uses texture atlases) and I noticed ones for the following:

  • GUI
  • Static objects
  • Moving objects
  • Moving object shadows
  • Moving objects occluded areas
  • Vegetation normal
  • Vegetation shadows
  • Vegetation decal normal
  • [Future] static object decals
  • [Future] moving object decals

So, lots of combinations. So I delved in Unity’s multi_compile pragma and custom, manual shader variants, and I came up with the following scheme, to have 3 different shader variant axes for sprites:

  • Orientation: Standing or decal
  • Sprite type: Static, moving or “splat”
  • Render type: Regular, shadow or occluded

GUI is still its own thing, but all the rest can be expressed with one value per “axis” above. While Unity nicely allows keywords to configure the multi_compile option, such configuration cannot change blend settings, z settings and core things like that. So, variants based on Render type (regular, shadow, occluded) are all different shader files, that define some defines and include the common shader code. The rest of the variants are just expressed with #ifdef. Here’s how the “Regular” render type variant shader looks like:

Shader "Sprite/TextureAtlasSpriteRegular"
{
	Properties
	{
		g_TextureAtlasSprites("TextureAtlasSprites", 2DArray) = "white" {}
		g_TextureAtlasConstants("TextureAtlasConstants", Vector) = (32,32,1,0)
		g_RealTime("Real time", int) = 0
		g_RenderingMoveSpeed("Rendering move speed", float) = 1
	}
		SubShader
		{
			Tags { "Queue" = "AlphaTest" "RenderType" = "Opaque" }
			LOD 100

			AlphaToMask On

			Pass
			{
				CGPROGRAM
				#pragma vertex vert
				#pragma fragment frag
				#pragma target 4.5

				#define VARIANT_REGULAR
				#pragma multi_compile_local VARIANT_ORIENTATION_DECAL VARIANT_ORIENTATION_STANDING
				#pragma multi_compile_local VARIANT_SPRITETYPE_STATIC VARIANT_SPRITETYPE_MOVING VARIANT_SPRITETYPE_SPLAT

				#pragma multi_compile_instancing
				//#pragma instancing_options procedural:setup

				#include "UnityCG.cginc"

				#include "Assets/Shaders/common.cginc"
				#include "Assets/Shaders/sprite.cginc"
				#include "Assets/Shaders/noise/random.cginc"

				// We don't need this, as we don't have gameobjects and materials for each
				UNITY_INSTANCING_BUFFER_START(Props)
				UNITY_INSTANCING_BUFFER_END(Props)

				
				#include "Assets/Shaders/Sprite/TextureAtlasSprite_common.cginc"

				ENDCG
			}
		}
}

So, now all the sprite code for all the variants is in a single source file, which is super convenient for editing. This approach now allows easy proper shadows for any object (static or moving) among other things.

Benefits of the new system: everything has proper shadows! fountain, chest, character, door.

As this was a hell of a tangent, to solve the original problem, I wrote a pseudo-autotile algorithm class called “Splat” where, if I’ve specified it, instead of autotiling it creates an instance buffer and renders that with the Splat render variant (which includes shadows). This results in the first image shown on the page, where we have nice randomized trees including shadows. And, even though I’m not showing it here, we can use a variety of tree types, which is very, very convenient (with autotiling that would be near impossible).

Spritesheet to Unity

I’ve made a few posts already about spritesheets, atlases, etc, as I can’t seem to make up my mind. Especially, as my sprite needs change constantly, as I don’t really have complete art and I’m trying to get away with a bit of DCSS tiles and a bit of Oryx tiles at the moment.

Originally, I used a 2D texture atlas + a JSON file with the description of what sprites are where. It was a nightmare to edit. Also, in runtime, filtering was difficult, as Unity provides only so much freedom with sampling, as with texture atlases care needs to be taken at the edges to prevent bleeding and do correct filtering. So, difficult to edit, and difficult to render. Booh.

Then I thought “Ok, let’s use texture arrays in Unity”. So atlas+JSON as source data, then conversion to a texture array in Unity for runtime. Rendering is now easy, without any filtering issues. I do have a limit of a maximum of 2048 sprites per atlas, which is not great, but my 32-bit sprite instance data has now 8 whole bits free as a result, as I need only 11 bits to represent the texture index. On the minus side, editing was still hard.

The last few weeks, I had the sudden realisation that the atlas+JSON format as source data is very, very pointless, as I’m converting to arrays in Unity anyway. So, I went back to the basic form, which is files-and-folders. One file per sprite, some special naming format for animations, folders and subfolders for grouping and … magic! Now the spritesheet is very, very easily editable. Tiles can be previewed directly in explorer, I can change sprite names at will, add/remove tiles, and do some more stuff (more next few weeks), and it’s all very, very easy. When I’m done with editing, I run some Unity script that converts that to an array (still limit of 2048 max per atlas applies), and that’s it. I feel like I’ve been making my life more difficult with the 2D texture atlas format. So, the new atlas format will be the final (barring minor mods), as there’s no problem point really.

With such a simple “loose” format, it’s quite easy to write python scripts/tools to process the spritesheets, e.g. rename sprites or mass-rename animations, create distance fields per sprite, do some autotiling work, etc.