DevBlog: Camera Control

Camera

I’ve been working on some usability issues for Edict over the last couple of weeks. Perhaps the most prominent is the camera control system.

Free-look

Up to this point we’ve had a free-look camera with no constraints on orientation or position. It’s a reasonable choice if you’re only concerned about development because it’s straightforward to implement, and it lets you view arbitrary parts of the scene. Want to check that backface culling is working on your terrain? Just fly underground. Want to view the sun position? Just pan skyward.

However, this lets the user move to locations that should be inaccessible for gameplay or performance reasons (eg, underground, or viewing the entire map); orientation changes tends to accumulate a lot of unintended camera roll; and, reframing the view is unnecessarily finicky for players.

Overhead

I’ve always intended to lock the player’s view into something like the overhead style that’s common to many strategy games. Perhaps not directly above the terrain with a pure orthographic projection like in "RimWorld"[1], but closer to the isometric view of games like "Age of Empires".

While I do want to reduce some freedoms (like roll and pitch) it’s worthwhile retaining enough camera parameters to display some amount of visual depth in the world.

I’ve implemented a more complex variant of a system from a previous engine I developed. It takes some cues from physical camera systems that I’ve used over the years.

Parameters

We define the camera in terms of two parameters which are applied at a given height.

height

The elevation at which these parameters apply
fov

The vertical field of view angle.
slope

The vertical angle of the camera. ie, how much it tilts down.

The FoV parameter aims to emulate the feel of a zoom lens on a camera. Longer focal lengths feel flatter and more compressed, while shorter focal lengths feel more expansive and draw you into the scene as a participant.

The slope parameter will be used to angle the camera very close to straight down when at maximum height so that it gives an appearance closer to that of a map, and to reduce any terrain or world obstructions. When the camera is zoomed in we move the camera a little closer to horizontal so that it feels more like they’re viewing a scene in the world rather than planning movements on a map.

These parameters are specified for the minimum and maximum permitted camera heights and linearly interpolated.[2]

angles
Figure 1. Camera Parameters

The diagram above shows a camera viewing a target \(T\) at distance \(D\), across heights \(H_0 \dots H_1\), with slopes \(S_0 \dots S_1\).

Re-centering

Testing the system as described shows that it’s functional, but quickly becomes frustrating: the camera does not remain pointed at the same world position after we update any camera parameters.

If we simply update the camera parameters then the camera will almost certainly be pointed at a different region of the world. Even a simple height increase will result in the camera target moving further away.

The player has pointed it at some area of interest and we should not disrupt this without a good reason.

I found it valuable to reframe the problem away from the camera position, and focus more on the target position. The problem then becomes finding a camera world-position such that, when the camera parameters are updated, the camera looks directly at original target world-position.

This tends to result in a camera following an arc as the user zooms out (noted in the camera parameters diagram). ie, to increase the height of the camera and look at the same target we may need to move the camera slightly away from the target position.

Using the model

The player primarily uses the scroll wheel on their mouse as a "zoom" control. This directly controls the height of the camera.

When the height is adjusted we linearly interpolate these parameters, solve for the world position, and copy the values into our camera object.

Camera movement in practice
Movement speed

When I was attempting to capture some animation footage it was immediately obvious that I’d forgotten to scale the panning speed of the camera. I found myself either slowly crawling around the terrain when zoomed out, or whizzing by a zombie/human fight when zoomed in.

The solution I’ve found useful here was to specify the panning speed as a fraction of the visible world. We know the view target, and we know the FoV angle, so we can calculate the extent of the visible terrain. [3] So if you want to spend 3 seconds panning across the visible scene then we divide that width by 3 for the panning speed.

Configuration

There’s not really a great deal of configuration we want to directly expose to the player. Some parameters will be dictated by performance constraints, others by our gameplay intentions.

However if you happen to know where to look you’ll see "yet-another-JSON file" that looks like the following.

interface/overhead/default.json
{
    "max" : {
        "fov"    :  45,
        "height" : 200,
        "slope"  : -80
    },
    "min" : {
        "fov"    :  60,
        "height" :   5,
        "slope"  : -35
    }
}

Height vs FoV

There are interesting second order effects that appear when the FoV parameter approaches zero.

My formulation of the camera system uses camera height as a proxy for camera zoom. That is: when you move the camera further away from the ground then objects tend to appear smaller; move the camera closer to the ground and objects appear larger.

However, you can achieve a similar outcome by decreasing the FoV of the camera. If the camera observes a smaller fraction of the scene, but the screen it’s presented on remains the same size, then objects will look larger.

This is what happens when we approach the maximum camera height in our system.

If we set an FoV range of 30..60 degrees then we start to see the scene scaling down as we initially increase the camera height, but then scaling back up as FoV effects start to dominate.

This should be something we can solve by some straightforward adjustments to the camera model. But at this point we will largely ignore the issue as the parameter values that feel best in Edict are outside the problematic range.

TODO

There’s still some work to be done before we hand out some binaries for pre-alpha testing; most important is probably sanding off the rougher edges from the initial GUI prototype. But it’s getting fairly close now. I’ll attend to some of these tasks over the coming week.

While largely uninteresting from a user perspective I’ve also spent parts of the last week or two integrating the new entity-component system into the engine. It should (eventually) lead to a modest performance gain, but perhaps more importantly it will simplify the implementation of systems like visualising building placement.

This work seems to have largely stabilised now and I should have some form of write-up on it next week.

1. I did try a quick orthographic test. I think this system would work fine if you designed for it from the start, but repositioning the camera felt quite disconcerting in Edict when there were no depth cues. In particular I never really got used to the lack of forward/backward controls.
2. There’s no reason, aside from implementation time, that this couldn’t be extended to an arbitrary number of heights or blending routines. There are probably some nice things you could do here with splines.
3. Or rather, we have a decent estimate. We don’t take into account the orientientation of the ground plane, nor the terrain geometry. But it’s a useful approximation and you have probably calculated most of the required values anyway.

DevBlog: Multi-file Animations

Edict

Now that we have the technical capability for multi-file animations we need to integrate this functionality into our engine.

Our current convention is to use files with the name 'model.fbx' as geometry sources, and '[email protected]' as animation sources. [1] And up until this point we’ve gotten away with directly specifying that uninfected agents used 'manModel.fbx' and infected agents used 'Zombie.fbx' within the simulation code (I know, I know…​).

Loading

For FBX file used in Edict we create a corresponding JSON configuration file. These are not provided to players, but drive the compilation parameters for each asset before it is shipped.

This setup has a number of useful properties:

  • JSON is a very flexible way to specify these parameters.

  • It keeps the configuration directly adjacent to the asset rather than deeply within the buildsystem.

  • It allows us to request alterations to a file’s contents without actually baking these changes into the original file. It is better to keep the original files pristene.

Geometry
manModel.json
{ "scale": 0.1687558009806587 }

All FBX assets support the 'scale' attribute which requests resize of the entire model/skeleton.

I try to keep units within the engine in metres. But sometimes this isn’t convenient for an animator, or we need to use models from outside the project which can’t be expected to conform to all our local conventions.

Animation
[email protected]
{
  "animation": {
    "rules": [
      { "rule": "rename",   "name": "Take 001", "target": "walk" },
      { "rule": "velocity", "name": "walk",     "value": 1.8 }
    ]
  },
}

FBX assets that contain animations have access to a collection of post-processing rules, specified as an ordered list of updates. This allows us to conform the contents to our asset conventions, and specify paramater values which aren’t otherwise available (eg, target velocity of an animation).

Some of the supported rules include:

rename

Change the name of the animation. It allows us to rename animations that would otherwise clash, or to update names that aren’t suitably descriptive.

delete

Remove the animation from further processing. Some external assets include animations that we don’t (yet) support. It’s often easier to just remove them.

velocity

Set the ground speed at which an animation expects the model is moving. This is required so that we know how fast to play back the animation to avoid skating.

At the end of the model/animation compilation process we end up with a file with a name like [email protected]. It contains graphics data formatted in a way that’s trivial to load, and with the post-processing rules baked in.

When the game starts the engine will scan the installed directory that contains the compiled resources. Each file will be installed into a cache in memory that is indexed by keys consisting of the filename (to simplify referencing the asset from other configuration files), and a runtime generated integer (for internal use where efficiency is paramount).

Linking

Now that the raw data is accessible to the engine we need a way to describe what model and animation each agent type should use. To this end we have one more configuration file; anything residing in the directory render/agent will name the resources that are used to render each agent.

zombie.json
{
    "model": "manModel",
    "animations": {
        "run":  "zombieRun",
        "walk": "zombieWalk",
        "idle": "zombieIdle"
    }
}

The files are named after the infection types that the simulation uses and are automatically associated with agents based on their current infection status. [2]

The configuration file specifies the key for the model, and the key for each known animation type (currently 'idle', 'walk', 'run'). [3] When we load this at runtime we combine the two keys so that, for example, 'zombieRun' becomes '\manModel@zombieRun'. This keeps the names we use for our asset storage and the engine cache consistent, and should reduce the chance of referencing the wrong file.

When it comes time to actually render the agent it’s mostly a question of indexing a few arrays. We look up the infection status, then the bundle of model/animation keys; lookup the current animation, and upload the interpolated bone transforms; finally dispatch a request to render the model key.

Testing

I had a few false starts when specifying this system. One option allowed arbitrary animation names but this flexibility complicated the implementation. Another option allowed much more flexibility in the way asset keys were specified, but that would have resulted in two methods of referring to assets and made it easier to specify mismatched models and animations.

The last experiment was a lot easier to implement than I was expecting, and was the most robust. While there are a few issues with the exact values we use for zombie animation speed and velocities, the underlying features seem to meet our current needs.

TODO

For at least the coming week I’ll be focusing on features and fixes that make the current iteration of Edict substantially easier to interact with. We want to get binaries in the hands of testers soon, and keep the build up-to-date for rapid feedback and gameplay iteration.

Most immediately this will involve the implementation of a fixed camera system, visual feedback on the purpose of regions that the user has placed, and smoothing out the rough edges on the current UI system.

1. We also have the ability to extract any animations present whatever the expected intention was so that we can use external assets more easily. For example the excellent packs from Quaternius and Kenny tend to come as single files. However we prefer the split-file approach where possible.
2. The engine has a concept of 'uninfected', 'latent', 'symptomatic', 'zombie', and 'immune'. However 'zombie' and 'uninfected' are of primary importance to the current iteration of the simulation.
3. I’ve got a note to allow arbitrary animation names at some point in the future, but fixed size arrays tend to be a safe and efficient way to start with.

DevBlog: FBX Loader

Edict

This week my efforts have been focussed on finalising the initial functionality of our new FBX loading library, followed by the first efforts at loading multi-file FBX model data.

FBX

The FBX file format is quite common in the industry. It is a flexible way to store geometry, animation, material, and other visual data.

Most content pipelines can export and/or ingest FBX data in some way. We use FBX as an export format from Maya (and occasionally Blender), and an import format for our model processing and compilation tools.

I’ve been focussed on finishing the first pass at our animation system so that James can start seeing more of his work in engine. As part of this we identified one capability that we’d really like to see: the separation of geometry and material data from the animation data. That is to say, we would like to:

  • Apply multiple animations to one mesh. eg, 'model.fbx' stores the geometry, '[email protected]' stores the walk animation, '[email protected]' stores the run animation.

  • Apply one animation to multiple meshes. eg, 'human0.fbx' stores one human mesh, 'human1.fbx' stores a second human mesh, and '[email protected]' stores the common walk animation.

I’ve used libassimp to handle the loading of static geometry data for a few years and it’s been pretty capable overall (though the build system has caused intermittent problems with cross compilation). [1]

Unfortunately we hit a bit of a wall over December.

Of nodes and transforms

An FBX file contains — in part — a tree of named transforms I’ll refer to as 'nodes' that generally correspond to joints in a model. The animator controls the movement of nodes by defining a set of 'curves' which define their translation, rotation, and scaling over time.

Visualisation of manModel’s node

Each node effects its children. eg, Moving 'R_armA_shoulder' will also move 'R_armA_elbow', which will also move 'R_arma_wrist', and so forth.

The movement of these nodes defines how the mesh will change over time. Each vertex in the mesh calculates its position by combining the transforms of some of these nodes as they move about the world. eg, the position of the upper arm will be driven by a combination of the transforms for 'R_armA_shoulder' and 'R_armA_elbow'.

We can end up with a reasonably broad and deep tree of nodes if we want to animate an object in detail. Our 'manModel' contains nearly 50 nodes.

node hierarchy
Figure 1. manModel node hierarchy

One area where we can gain a bit of graphics performance is by reducing the number of these nodes where they don’t have (much of) a visual impact. The fewer transforms we need to apply at runtime, the more time and memory we can dedicate to other tasks.

This is hugely important for FBX because it tends to result in far more nodes than are reasonable due to the flexibility of the format. Specifically, it allows the author to record not only 'translate', 'rotate', and 'scale' transforms at each node over time (which are the raw controls that animators tend to work with), but also 'pre', 'post', 'pivot', and 'offset' forms of these (which aren’t always directly exposed).

The full transform sequence required amounts to the following:

\[t . r_\mathrm{offset} . r_\mathrm{pivot} . r_\mathrm{pre} . r . r_\mathrm{post} . r_\mathrm{pivot}^{-1} . s_\mathrm{offset} . s_\mathrm{pivot} . s . s_\mathrm{pivot}^{-1}\]

Note that there are terms between \(r\), \(s\), and \(t\). It may be possible to remove the pre, post, and offset transforms, but only if they are an identity transform. Otherwise, this doubles your node count. The \(r\), \(s\), and \(t\) terms can only be removed if no animation curve references them.

We were easily reaching over three hundred nodes on our character models. A great deal of these nodes are unnecessary.

Using libassimp

libassimp is definitely capable of representing these complex transforms. It stores transforms like \(r_\mathrm{pre}\) as virtual nodes within the tree; updating their names with suffixes like $AssimpFbx$_PreRotation.

Collapsing unnecessary nodes is a fairly straightforward operation. We can walk the node tree, check if each node has been referenced, and if not we fold it into the parent node. libassimp can be instructed to do this with the option AI_CONFIG_IMPORT_FBX_PRESERVE_PIVOTS=false. However experiments with this option showed a number of cases where transforms were lost or applied incorrectly.

Importantly, this approach only works if:

  • You consider every related file together. Just because the contents of 'model.fbx' don’t reference the 'elbow' node doesn’t mean that '[email protected]' won’t.

  • Every file is loaded with exactly the same options. The exact counts and values of many structures must match between files. If one node is missing then every subsequent transform may be incorrect.

I need a way to extract the data from all related files, construct a list of all referenced nodes, analyse all the associated transforms to check if any can be collapsed, and then perform the modifications. If we’re even slightly off in the data matching we end up with missing nodes, and animations with misplaced limbs.

libassimp does store the required values in the virtual nodes, but accessing the data proved fragile and it was not 100% reliable when it removed unnecessary nodes. It also failed to create identical node hierarchies between files that were based on the same skeleton; there were inconsistencies when just animation curves, or just geometry, were present. This led to situations where limbs further from the root node were progressively more distorted or detached.

Missing bones after libassimp FBX import

I could have modified the library to include the necessary functionality, but it would have been an involved process and I’ve had ongoing issues maintaining libassimp and it’s dependencies. [2]

libcruft-mesh

So I expanded our mesh processing library to include FBX loading support over the last couple of months. This was simplified somewhat because we do not need to support all the functionality FBX provides. Only the subset which Maya and Blender exercise.

The extensions to libcruft-mesh give direct access to each of the node’s internal transforms so a more complete decision can be made about which nodes are required.

And we take great pains to ensure that every file is loaded without a loss of information, regardless of whether the data appears to be used, so that we increase the accuracy of cross-file comparisons.

As a side effect this is prompting me to more closely integrate mesh processing algorithsm I’ve developed in libcruft-mesh, such as auto-LOD generation and cache optimising index reordering.

Documentation

Coming back from holidays, reading some recently committed code, and feedback from James' perusal of the (woefully out of date) user manual during testing I’ve realised I need to work on all forms of documentation this year.

So part of this week was spent adding some documentation to libcruft-mesh, updating the system manual (so that I don’t forget how all the above systems work), and reinitialising this blog.

I’ve thrown together a fairly minimal install of the nikola static site generator. It seems to do exactly what it says on the tin, and largely stay out of my way. I’m pretty happy with it so far.

TODO

There’s still work to be done automatically linking 'model' and 'animation' files within our build system but the functionality is there if you hold the system’s hand. Finishing the implicit linking and configuration driven linking of '.fbx' files will be a job for the coming week.

1. I use pkg-config to identify library paths and flags for our dependencies. But many packages fail to pass at least one of the flags that the toolchain requires for cross-compilation; if you’re using CMake you need to be using IMPORTED_TARGET with pkg_check_modules. Header and library paths are particularly problematic as they can lead to the inadvertent use of libraries that are part of my local system.
2. zlib doesn’t even maintain a consistent library name across target systems; Windows is zlib1.dll and Linux is libz.so.1.