I've finally gotten around to implementing proper skeletal animation for the 3D models of people in 3DWorld's cities and buildings. This involved completing many different steps along the way. I'll discuss my process here. This will be a relatively short post with lots of technical details, so fee free to play the videos at the end without reading the text. I'll add links where I can to articles that explain these concepts and give more info.
First, I had to write a wrapper for importing models using Assimp because my built in loaders only support the OBJ and 3DS file formats, and these don't store animations. This includes adding support for reading embedded textures as found in the FBX file format. The basic OpenGL/Assimp tutorial was useful for this, but only explained how to import simple static model geometry. If anyone is curious, my model import code can be found here, and the model rendering side of the code can be found here.
I originally had no idea how to load and use model animations, so I found a GitHub/YouTube tutorial series and another blog post that showed how to implement the matrix transforms with GLM. This took many attempts to get right because the code from those tutorials wasn't easy to integrate into 3DWorld's existing rendering system. I also ran into issues with the coordinate system used (I use Z as up rather than Y) and bounding cubes of animated geometry. Getting these things right required quite a bit of trial-and-error, including generating text logs of coordinates and transforms that I had to manually review. I guess that's to be expected when trying to learn multiple new things at the same time.
I used 3DWorld's model viewer scene config for working with Assimp model loading and experimenting with animations. This view shows a single model with nothing else other than the sky background. It's much faster to load than the city/buildings scene. I was able to compare my results with the 3D Viewer application that comes with Windows, which supports loading/drawing embedded textures and animations. In addition, I took advantage of 3DWorld's hot reload of OpenGL shader programs while debugging to get the GPU side of this correct.
The models and animations I used for this were found on Adobe Mixamo, which is free but requires creating an account. While my code worked fine on the md5mesh model used in the tutorial, all of the Mixamo models looked very wrong when animations were enabled, with curvy legs and arms bent in the wrong direction. For example:
 |
| Is it my imagination, or is something wrong with the animations here? For some reason, this model doesn't look like the one on the website. |
It turns out that the problem was due to a known bug in Assimp. This took me many hours to figure out. What's particularly annoying is that the version of Assimp I have on linux doesn't have this bug, it's only present on Windows. This is why I never came across it while working with the tutorial project on linux.
I replaced the four static models I was using for people in 3DWorld with five new animated people models. (Well, technically I was animating the original models in the vertex shader, but only their legs moved.) Here's a YouTube video showing the final results with five different models using walking animations.
That looks pretty good for regular people. But what about gameplay mode? We need to add some zombies, and I can finally get proper zombie models and animations. They definitely make building gameplay mode much more frightening!
That's all I have for today. The next step is to store/use multiple animations per model. I need at least an idle standing animation for people. They currently stop mid-walk with one leg up in the air, which doesn't look very natural. I can't revert to the non-animated version when stopped because these models all begin in a T-pose with their arms straight out to their sides.
Great to see you can use standard armature animations now! Opens up a lot of possibilities! Are you considering implementing in-engine IK so the characters can properly interact with environment objects?
ReplyDeleteOn the canned animation side, I imagine Mixamo has plenty of idle animations for you to choose from, but if you need something tweaked, or anything custom, I should be able to help you out.
Thanks! I haven't done anything with IK and I'm not really familiar with that area. I need to look for a good tutorial on this when I have time. Most of the tutorials are for doing it in UE or Unity rather than a custom engine. I assume this would improve the animations when people and walking up and down stairs, which is currently not very good.
ReplyDeleteI definitely need some sort of idle animation to avoid people stopping with one foot raised. The problem is that I can't find a way to load multiple animations from a single model. Some of those Mixamo models that come with multiple animations end up with a single animation that's all the individual ones concatenated together into a single large animation. At least that's what Assimp returns - one animation channel with no obvious way to find the start or end of an individual sequence. I'm not sure if it's a limitation of Mixamo, the model file format I'm using (typically FBX), or Assimp itself.
The FBX format only supports a single animation per armature per file. The multiple animations strung together probably intend you to differentiate by frame range. Otherwise, you'll need to import the animations from multiple different files, though the base mesh shouldn't need to be included, just the armature.
DeleteIt should also be possible to use the same animation data on multiple armatures, as long as they share a bone hierarchy and naming convention, which I believe is what using a standardized platform like Mixamo should ensure.
DeleteWhat 3D file formats support multiple animations in a single file? The importer and animation system I wrote expect a single file with both the model/armature and multiple animations together. This is how the tutorial I started from worked. I don't think it would support reading only one or the other, at least not in the way it's implemented. The config file system that specifies the scene objects also expects everything in a single filename. The system I have in place isn't only used for people but is also used for cars, helicopters, and room objects, where I have plans to animate these as well at some point. For example, the helicopter blades rotate.
DeleteI wish I had known about this limitation earlier. Maybe I would have done this differently. Thanks.
You're going to want a way to transfer animation data across different compatible armatures, at the very least to run different animations (sitting, standing, walking, running, etc) on the same character, as well as smoothly blending between them. From there, importing just animation data and adding it to the animation library should be pretty straightforward.
DeleteAnimating mechanical things could be done with armatures too if you want. You're certainly going to need to get under the hood there though, as just playing one animation for the car wheels turning, and a different one for steering, and then trying to overlay them, is a sure path to madness. Much better to drive the transforms parametrically.
I'll have to try loading just the animation from a file and see what happens. It probably won't work and will need to be fixed. My internet has been down for 3 days due to the storms, so I can't work on that now anyway.
DeleteI'm procedurally animating the mechanical objects and small animals such as rats, snakes, and spiders. My point was that I'm using the same model loading system for both animated and non-animated/procedurally animated models, so I need to be careful I don't break it. It has to be able to load separate animations but in an optional way. For example, I need a syntax for assigning a filename for a particular animation of a particular model. I have to think the whole flow through before implementing anything. It's pretty clear now that what I already wrote isn't enough to make this work.
[It won't let me sign in to comment?] 3DWorld now supports attaching named animations from separate files and now there are "walk" and "idle" animations for all people and zombies. It doesn't blend smoothly though for various reasons. The walk animation is played at a variable speed that depends on the character and their state (faster when crossing streets and chasing the player, etc.), while the idle animation always plays at a constant rate. In addition, switching animations mid-step can cause some strange interpolation when the legs need to interpolate to very different posts.
DeleteThe other problem is that the state machine can't tell the rendering system ahead of time when a person/zombie is transitioning between states, it can only say that person started or stopped walking. So I can't interpolate correctly across the transition, the blending is delayed until the start of the new animation. So if the blend time is too long the person will slide at the beginning of their walk while their legs are stationary from the idle animation.
Well at least it's better than it was originally. I'll continue to think about how to improve this.
Also note that I tried to share animations across armatures. That seems to sometimes work, sometimes produce janky animation, and sometimes results in the model T-posing. So instead I had to export each animation for each armature. At least the files are much smaller than the models and only require one set of textures.
You're doing well to have got that far! Animation system jank is par for the course. Looking forward to seeing the new multi-animations in place!
DeleteIf you're looking for a place to get started with IK, I know Blender has a built-in system, and the project is open source.