Koeky 3D and Animations

Whenever you want your game to be more than just static triangle meshes moving around you'll have to use animations. In 3D games a common way to implement animations is to make use of skeletal animations.

Skeletal animations make use of (like the name suggests) skeletons. A skeleton is build from bones, every bone can be linked to a parent and to several children. This forms a hierarchy. This is a powerful principle, it means than when you move the shoulder bone the hand bone will also move (the hand bone is a child of the shoulder bone in this example).

Vertices in this case are the flesh of the model. Every vertex is linked to a bone (or more if you want to). So whenever a bone moves, all vertices that are attached to it will also move. This animates the triangle mesh.

Koeky 3D supports skeletal animations. If you can supply the layout of the skeleton and an animation, Koeky 3D can animate your triangle mesh (the source code contains an example showing an animated dwarf).

Creating a skeleton and animation

There are several classes of interest if you want to implement skeletal animations. First off is the Bone class. The Bone class represents 1 bone in the skeleton. To create a Bone object you need to do the following:
    int index = 1; // The index of this bone in the skeleton hierarchy
    int parentIndex = 0; // The index of the parent bone (-1 means no parent)
    Matrix4 localTransform = ....; // Local transform comes here. 

    // Create the bone
    Bone bone = new Bone(index, parentIndex, localTransform);

By creating several of these Bone objects and linking them (using the index and parent index variables) you can create a hierarchy of bones (a skeleton!).
The local transform describes the initial transform of this bone, usually you make your animations in a modeling program and then read the animations from a file (for example a milkshape model). The local transform is also stored there, so it is only a matter of passing it to the bone class.

Next we need to store the bones in a Skeleton object.
    Bone[] bones = .... // bones are defined here
    Skeleton skeleton = new Skeleton(bones); // Create the skeleton

An array of bones can be re used in as many Skeleton objects as you want. So you could cache the bones and when you create a new model that needs to be animated you only need to create a new Skeleton object.

Now for some more functional code. The example below uses the Milkshape model loader I wrote (available when you download the source code) to define an array of Bones and finally create a Skeleton.
    // Load the milkshape model
    MilkshapeLoader loader = new MilkshapeLoader();
    MilkshapeModel milkModel;
    MilkshapeLoadResult result = loader.LoadModel("Data/Models/dwarf.ms3d", 
                                                   out milkModel);
    if (result != MilkshapeLoadResult.ModelLoaded)
    {
        // Model failed to load, show a message and quit
        MessageBox.Show("Error: " + result.ToString());
        Environment.Exit(0);
    }

    // The MilkshapeModel class does not contain data which can
    // be used as Bone data immediatly, so we need to convert it.

    Bone[] bones = new Bone[milkModel.bones.Length];
    for (int i = 0; i < bones.Length; i++)
    {
        MilkshapeBone milkBone = milkModel.bones[i];

        // Define the index and parent index
        int index = milkBone.index;
        int parentIndex = milkBone.parentBone == null ? -1 : milkBone.parentBone.index;

        // Define the local transform. We use the GLConversion class to 
        // easily create a transform
        Matrix4 localMatrix = GLConversion.CreateTransformMatrix(milkBone.initPosition, 
                                           milkBone.initRotation);

        // Create and store the bone
        bones[i] = new Bone(index, parentIndex, localMatrix);
    }

    // Create the Skeleton object
    Skeleton skeleton = new Skeleton(bones);


The above example is aimed at the milkshape model loader. But any form of bone data will do, as long as you convert it first. Koeky 3D Framework is not aimed at Milkshape. I just chose it because it is a simple format.

Right now we got a skeleton. We now need to have an animation to animate it with. Animations are stored in the Animation class. To create one we need to have key frame data. Keyframes is the format in which the animation is defined. You can think of a keyframe as a snapshot of a pose at a certain time. For example: imagine the following two key frames.
    Vector2 firstKeyframe = new Vector2(0.0f, 0.0f);
    float firstKeyFrameTime = 0.0f;
    
    Vector2 secondKeyFrame = new Vector2(10.0f, 0.0f);
    float secondKeyFrameTime = 10.0f;

at time 0.0 (the time of the first key frame) the position is (0.0f, 0.0f). This is a snapshot I previously described. At time = 10.0 (the time of the second key frame) the position is (10.0f, 0.0f). This is the second snapshot.
Animations work with this kind of data. The animation has a current tim and based on two keyframes which lie between that time a value is calculated. In the above example that would mean at time 5.0 we are between keyframe 1 and keyframe 2. And because keyframe 1 is at (0.0f, 0.0f) and keyframe 2 is at (10.0f, 0.0f) we must be at (5.0f, 0.0f).

In Koeky 3D you got two kinds of key frames. You got keyframes which translate (move) a bone and you got keyframes which rotate a bone. The keyframes for one bone are stored in a BoneKeyframes object.
If we use the milkshape model we loaded previously creating the keyframes for every bone will look like this:
// We store the key frames per bone in an array
BoneKeyframes[] animationKeyFrames = new BoneKeyframes[bones.Length];
for (int i = 0; i < bones.Length; i++)
{
    MilkshapeBone milkBone = milkModel.bones[i];

    // Create all key frames which translate a bone
    KeyframePosition[] translationKeyFrames = 
                   new KeyframePosition[milkBone.translations.Length];
    for (int j = 0; j < translationKeyFrames.Length; j++)
    {
        MilkshapeMovement translation = milkBone.translations[j];

        translationKeyFrames[j] = new KeyframePosition
                                               (
                                                  translation.time, 
                                                  translation.movement.X,
                                                  translation.movement.Y, 
                                                  translation.movement.Z
                                                );
    }

    // Create all key frames which rotate a bone
    KeyframeRotation[] rotationKeyFrames = 
               new KeyframeRotation[milkBone.rotations.Length];
    for (int j = 0; j < rotationKeyFrames.Length; j++)
    {
        MilkshapeMovement rotation = milkBone.rotations[j];

        rotationKeyFrames[j] = new KeyframeRotation
                                     (
                                       rotation.time, 
                                       rotation.movement.X, 
                                       rotation.movement.Y, 
                                       rotation.movement.Z
                                      );
    }

    // Store the translation and rotation key frames in a BoneKeyFrames object
    animationKeyFrames[i] = new BoneKeyframes(translationKeyFrames, rotationKeyFrames);
    animationKeyFrames[i].weight = 1.0f;
}

We just copy the data from a milkshape model into a new class and store it. It is important to note that every bone in a skeleton has its own list of keyframes. Keyframes are not for an entire skeleton but for every bone seperately. The way Koeky 3D reads this array of BoneKeyframes is that the first element animates the first bone in the skeleton, the second animates the second bone and so on.

Another point of interest is the last line:
animationKeyFrames[i].weight = 1.0f;

A weight describes how much the keyframes influence the bone it applies to. Setting it to 0 means it will not influence it, 1 means it influences it completely. This can be used to blend several animations together (more about that later).

Now we can finally create the Animation class.
Animation animation = new Animation(animationKeyFrames);


Using this Animation class we can call the following method on the Skeleton class:
skeleton.StartAnimation(animation, AnimationType.ForwardLoop, 1.0f);

We start the previously defined animation on the previously defined skeleton. We tell it that it runs forward (from start to end) and when it the animation ends we start again at the start (looping). The speed is 1.0 meaning it runs at the default speed. If we would say speed = 2.0 we would go twice as slow and with 0.5 we would go twice as fast.

Next we need to call the following method in (for example) our onUpdateFrame method:
protected override void OnUpdateFrame(FrameEventArgs e)
{
    this.skeleton.Update((float)e.Time);
}

This will update our animation by the given timestep. The given timestep is added to the current time of the skeleton.

That's it: we have defined a skeleton (using an array of bones), next we defined a set of translation and rotation keyframes and used these to create an animation. Lastly we started and updated an animation. But we are not quite there yet...

Rendering an animated model

All the above code really is not related to OpenGL, we are not drawing anything here! To render an animated model we need to have a shader which transforms every vertex according to the bone it is attached to. So we somehow need to know how every vertex is linked to a certain bone.

The way this usually is done is by defining the attached bone per vertex as a vertex attribute. So we need a vertex position, a vertex normal, a vertex texture coordinate and a vertex bone index to define an animated model.
Let's define a vertex array object which contains all this data, we are once again using the milkshape model we already loaded:
// Create the vertex buffers
VertexBuffer verticesBuffer = new VertexBuffer(BufferUsageHint.StaticDraw, 
                                      (int)BufferAttribute.Vertex, 
                                      milkModel.vertices);
VertexBuffer texCoordBuffer = new VertexBuffer(BufferUsageHint.StaticDraw, 
                      (int)BufferAttribute.TexCoord, 
                      milkModel.texCoords);
VertexBuffer normalBuffer = new VertexBuffer(BufferUsageHint.StaticDraw, 
                    (int)BufferAttribute.Normal, 
                     milkModel.normals);

// The interesting part: here we create a buffer to keep track which vertex is linked to 
// which bone.
VertexBuffer boneBuffer = new VertexBuffer(BufferUsageHint.StaticDraw, (int)BufferAttribute.BoneIndex, 
                milkModel.boneIndices);

// Create the vertex array object
VertexArray vertexArray = new VertexArray(verticesBuffer, texCoordBuffer, normalBuffer, boneBuffer);

The milkModel.boneIndices variable contains Vector4 structs. This is because you can have more than one bone linked to one vertex. In this tutorial, however, I will focus on animating a model with only one bone per vertex.

I am using the Model class from the tutorial Koeky 3D Framework: Loading Models because it will cover everything we need. We do however need a new render Technique and a new vertex shader to render this model. I will use the class ModelTechnique as the render Technique (techniques are explained in the tutorial Koeky 3D Framework: Shaders), which is provided by default. As for the vertex shader I use the code below:
#version 140
in vec3 in_Vertex;
in vec2 in_TexCoord;
in vec3 in_Normal;
// Here is our bone index
in vec4 in_BoneIndex;
                    
out vec2 out_TexCoord;
out vec3 out_Normal;

uniform mat4x4 projection, world, view;
uniform mat4x4[64] joints;
uniform mat4x4[64] invJoints;

void main()
{
    out_TexCoord = in_TexCoord;
	
    // cast the bone index to a integer.
    int boneIndex = int(in_BoneIndex.x);
                
    // To properly transform the vertex it must first be transformed
    // back to its origin. Therefore we transform it with the inverse
    // of the absolute transform.
    // Next we can transform it with the current joint (bone) transformation.
    vec4 newVertex = vec4(in_Vertex, 1.0f) * invJoints[boneIndex] * joints[boneIndex];
    // We must also transform the normal, ofcourse.
    vec3 newNormal = in_Normal * mat3x3(joints[boneIndex]);
		
    // Same as in_Normal * gl_TextureMatrix
    out_Normal = mat3x3(transpose(inverse(view * world))) * newNormal;
		
    // Transform the vertex to get the final position
    gl_Position = projection * view * world * newVertex;
}

The fragment shader is no different than the one used in the model rendering tutorial.

The OnRenderFrame method looks like this:
protected override void OnRenderFrame(FrameEventArgs e)
{
    this.glManager.ClearScreen(ClearBufferMask.DepthBufferBit | 
                    ClearBufferMask.ColorBufferBit);

    // Bind the model technique
    this.glManager.BindTechnique(this.modelTechnique);

    // Uploads the joints (or bones) transform to the shader.
    // This is an array with 4x4 matrices containing one transform per bone.
    this.modelTechnique.Joints = this.skeleton.finalJointMatrices;

    // Uploads the inverse of every absolute transform. 
    // Before multiplying a vertex by it's bone transform 
    // we must first center it around zero again.
    // We do this by multiplying with the inverse of the default transform of the bone.
    // We could also do this transformation while loading the model, 
    // this may speed things up since we would not 
    // need to upload them to the shader anymore.
    this.modelTechnique.InvJoints = this.skeleton.invJointMatrices;

    this.glManager.Projection = this.renderOptions.Projection;
    this.glManager.View = Matrix4.CreateTranslation(0.0f, -30.0f, -150.0f);

    // Render the model
    this.model.Draw(this.glManager);

    base.SwapBuffers();
}


The complete code is available when you make a checkout of the source code. The project AnimationExample shows the complete code.

Final words

At this point we know how to create a skeleton and an animation. To compliment this we also know how to render a model which uses the animation data from the skeleton class.
Not every functionality of animations are explained in this tutorial. It is possible to run several animations at the same time on the same model, by using the weight attribute in the BoneKeyframes class you can determine what should happen. I will probably write a seperate tutorial for this tough.

Last edited Aug 13, 2012 at 3:47 PM by Mathyn, version 1

Comments

No comments yet.