Let's Talk Procedural Content

Procedural content has been a buzz word for a while now. The concept isn’t new but people seem to be using it a bit more than in the past.  Plenty of games have used procedural techniques in different areas.  For example in TA we used a tool called Bryce to generate landscapes.  Diablo and Minecraft used it for worlds.  Many other examples exist.

When this is brought up, people invariably think that if you are using procedural tools that your entire game has to be procedural. Forget about trying to do that. What I want to talk about is using procedural tools as part of the asset creation process to enhance the power of the tools that we already use.  Think about it as poly modeling with really advanced tools.

The end goal here is to allow artists to create things quickly and efficiently, customize them and save them off for use in the game.  This doesn’t imply that the procedural part has to be done at game runtime; these tools purely exist as a way for the artists to express themselves.  This has a few benefits including fitting into current pipelines and being able to use procedural techniques that are extremely processor intensive.  Pixar doesn’t send out the data they use to render their movies, they send out final rendered frames.  Same idea.  Sure being able to do it at runtime has some nice things going for it but it’s not always necessary in every case. 

There are a bunch of issues with creating the procedural models themselves and I’ll talk about that some other time.  Let’s just assume for a minute that we do have some procedural models that we want to use.  By models I mean something that takes a set of inputs and gives a processed output.  For example a procedural forest model would have things like type of tree, density of trees, overall canopy height etc and would generate trees, bushes etc in some area you’ve defined within your world.  A procedural terrain model would allow you to specify what the ground was made of, how hilly it is etc.

So let’s start out with tessellated terrain that’s slightly hilly.  In your head basically just picture some slightly rolling hills that are about a square kilometer.  Now let’s select that terrain and create a procedural forest model that overlaps it.  We set the canopy to about 50ish feet, set the density fairly high and bam; we have a forest covering our terrain.  I’m sure someone’s engine already does this so I’m not really taking a leap here.

Ok, the next thing is to find a nice flat area because I want to put down a building.  If we don’t have a flat area we go back and fix that at the terrain generation stage.  The forest will automatically regen to conform to the terrain because we can always “re-run” the procedural simulation if we store them off (duh).

We have our flat area covered by forest because we selected the entire terrain.  Simply select the blank out area or whatever you want to call it tool and start erasing part of the forest model to create a clearing.  All of this erasing is itself a procedural model that’s being applied as a construction step.  It’s storing off the idea that the area should be cleared so that if we re-generate with slightly different parameters we still get the same basic result, a clearing in the forest.  Of course if we modify the initial forest parameters we can easily change the type, density etc. of the forest.

Now that we have a hilly forest with a flat clearing in the middle we can start to add interesting game-play stuff.  The forest itself is really a fancy backdrop but we can customize any particular element of to our specific needs.  So the next thing I want to add is a concrete bunker that has some shiny new metal parts on it including a door, various panels, bolts, pipes etc.  This bunker could itself be a form of procedural model (e.g. a specialization of procedural buildings) or it could be a bespoke item created for game play purposes.  Either way picture us plunking down a concrete bunker here.

Now we have a dense forest on rolling hills with a concrete bunker in the middle of a clearing. Let’s get started on the actual interesting stuff! The next level of procedural generation past this is to start doing serious simulation on the world.  This is going to require a format that is flexible and allows both the textures and geometry of the world to be extensively modified.  There are a lot of different formats that could be tried and as long as we can convert between them we can come up with different representations for different algorithms.  Some sort of voxel type format might make a lot of sense for all of the models but I don’t want to make any particular assumption about the best way to do it.

First off let’s make it rain.  We set the procedural weather model to be rainy (skycube to match!).  We literally simulate water coming down from the sky.  We see where it bounces; we potentially let it erode dirt.  We let it form channels and see where it flows to.  We can run this simulation for a little bit of time, or a lot.  We can have a lot of rain, or a little.  The longer we run it the more extreme effects it has.  For example it could start to rust and streak any metal.  Picture our shiny bunker getting long streaks of rust running down from the panels and bolts.  Picture the rain causing erosion as it goes down the hills and potentially forming lakes, puddles and streams.   At any point we can “lock in” the simulation to capture how full the streams and lakes are etc.  The point being we’ve taken a very process intensive process and applied it to try and get a lot of bespoke looking stuff without having to model these processes in the head of an artist.

Now let’s go back to our forest.  We started with a simple model that generated a dense forest to make life easy.  But is there any reason that a forest aging model couldn’t be applied to the forest at the same time we are simulating the rain?  For example trees that are near water could grow larger.  Areas with sparse water could clear and eventually switch to other types of vegetation.  We could simulate sunlight and plant growth patterns.  We know how to do shadows pretty well, let’s simulate the sun moving across the sky and see which patches get more sunlight. Picture our streaked bunked with grass starting to grow around it, vines snaking up the side, bushes growing up beside it and having their shape affected by the structure.  If you ran it long enough maybe the forest itself would start to intrude on the structure and possibly even crack the concrete structure over time to create a ruin.  As we improve our physics model we can certainly do things like finite element analysis to figure out and simulate breakage.   

Are you starting to understand the power of this kind of modeling?  We don’t need to be perfect, just create models that have interesting effects and combine them.  This could be really cool even when simply done. Keep in mind I’m not taking any real leaps here, most of this is based on stuff I’ve seen happening already.  I’ve seen water erosion in real-time, I’ve seen procedural forests, I’ve seen rust streaking.  What we really need is a structure to build all of this stuff on top of.  The majority of the actual work is going to be defining all of the procedural models. But that’s something that can be centrally done.  Once people have created those models they can be re-used and upgraded in quality over time.

If we could define an extensible procedural model format that could plug into multiple tools we would really have something. This is an ongoing area of research, for example GML. A common net repository of models that anyone could contribute too that had standard (and actually usable) license terms would mean we could all benefit. You can still build whatever bespoke stuff you need for your game but the time wasting work that’s the same every time could be made easier.

All that I’ve said here is a straightforward application of stuff we can already do.  There are numerous products out there headed in this direction right now.  CityEngine and SpeedTree are just some of the examples among many.

Obviously there is a lot of research to be done on this stuff going forward.  I foresee more research on how best to represent these models, better simulation tools and lots of specialty models that do a domain specific job.

Speaking of specialty models human characters are already widely created using procedural models.  A lot of game companies have a custom solution to this and they all have different strengths and limitations.  This could go away if we have a good commercial package that was a high quality implementation.  It would create humans out of the box but the algorithms could be customized and extended (or downloaded from someone else who has done that).   There is simply no reason that this needs to be reimplemented by every game company on earth.  It’s an active area of research already obviously.  Of course creating the geometry is hard, getting it to animate is harder.  But we’ll crack it eventually.

There are plenty of examples where specialized models would come in handy. This would enable creation of all kinds of animals and plants, specific types of rock etc.   Hopefully the standard modeling language would be expressive enough to do most normal things.  Domain specific ways of thinking about particular models will probably be necessary in some (a lot?) of cases.  A long as we have a well defined environment for these models to interact in I think we can get away with a lot of hacky stuff at first.

If you are going to have a procedural forest it would make sense to have it include animals.  Deer, bears, cattle, birds or whatever could be modeled and then given time based behavioral simulation.  There is already precedent for actual mating of models so it’s possible we could simulate multiple generations and actually evolve new populations.  This level of modeling is getting pretty sophisticated but I don’t see any intrinsic limitation.

Once you have a forest populated with procedural animals you can consider extending these algorithms ever further.  For example you could simulate the animal populations and their effects on the world so that over time you converge to some interesting results.  That natural cave now has a bear living in it.  Lakes and streams have fish that have evolved to use particular spawning grounds.   Areas trampled often by animals could have less vegetation.  Grazing areas and other types of foliage could be eaten or even spread by the animal population moving seeds about.  Most of the behaviors I’m describing could be modeled using simple techniques that would still probably produce neat results.  For example to model foliage destruction you could use a heat map of where the simulated animals walked.  Look for seeds sources and spreads them across the heat map graph using high traffic areas as connections.  Then have those seeds germinate over time based on all of the other factors we’ve talked about.

Now I’ve used a forest as an example here but that’s just one type of operator.  There will be meta operators at every level here that feed into and effect one another.  For example you could do a planetary generator that runs a meta simulation to decide where to place forest operators.  Oceans in tropical areas near shorelines could generate coral reefs operators.   The weather operator combined with elevation would define temperature which could feed into other processes like snow level.  I can see people creating a ton of custom operators that when combined together do fascinating things.  Volcanoes that shape the landscape.  Craters that are created through simulated bombardment and then eroded.  Plate tectonics to shape continents?  The ideas are endless if we have a framework to play around in.

Pie in the sky?  Let’s check back in 2040 and laugh at the primitive ideas presented here.  Or maybe our AI assistant will laugh for us.

Just wait until I start talking about augmented reality modeling tools.  But that’s another story…