Unstable results with Two-Bone IK

[jankrassnigg]

I am running into the very same issue as this one:
#172

However, I don't think that the proposed fix there is actually the right solution.

I'm playing a simple idle animation on a character, and then want to fix up a hand position with IK. However, the result keeps jerking back and forth between two extreme positions. This happens due to the underlying idle animation.

See the following video. The purple sphere is used to calculate the pole vector. The pole vector goes from the shoulder joint towards the sphere (and gets normalized).

ik-moving.mp4

When the sphere is at a "critical" location, the IK result oscilates quite a bit.

However, I can also reproduce this without the animation (or rather a paused animation), just by moving the target position slightly:

ik-not-moving.mp4

As you can see, there are two angles where this issue appears. It doesn't happen when the sphere is above or below the model.

This leads me to believe, that some calculation inside the two-bone job is sensitive to what axis the pole vector points at. Ie. in the ozz samples Y is the up vector, but our engine uses different conventions. Maybe some step assumes certain directions?

I've stared at the code a bit, but can't make much out of it. In ComputeStartJoint() there is this piece:

Could it be that the SplatY and SplatZ assume what the ground plane is??

I've now printed the values for rotate_plane_axis_flipped_ss and can confirm that at the point of the problem, it flips from 0.34 | 0.27 | -0.90 to -0.34 | -0.27 | 0.90.

I also noticed, that this only happens because the sampled animation is putting the character into a very different pose than the rest pose. If I use the rest pose instead, everything is fine, but the idle animation rotates the arm about 90 degrees down.

Ie here I've used the rest pose, and then the IK doesn't have this flip:

ik-restpose.mp4

I'm pretty much out of ideas here, whether this could be my fault somehow. I don't see how I could pass a different pole vector to get the desired result.

[guillaumeblanc]

Hi !

Thanks for reporting the issue.

Could it be that the SplatY and SplatZ assume what the ground plane is??
rsqrts contains 3 reciprocal square root, which are used to normalize ref_plane, joint_plane and start_target. So nothing to do with the ground plane.

in the ozz samples Y is the up vector, but our engine uses different conventions
It's true that I only tested for Y up. I don't remember writing something specific to that in 2 bone IK though.

I would investigate these paths:

1. Have you tried to deactivate SIMD math? There could be an implementation difference or mistake.

• By setting forcing Ref implementation from cmake
• Or defining OZZ_BUILD_SIMD_REF

2. The issue is probably around rotate_plane indeed. To help with debugging, could you display more information, like shoulder transform, joints before IK pass, pole vector... ?

Overall, it would be great to create a repro to be able to look at it, and display more debug info. If you can serialize the skeleton and the animation (or even only the pose that is creating the issue), then it shall be pretty quick to adapt sample_two_bone_ik and repro the issue. I can prepare the sample if it helps.

Hope it helps,
Guillaume

[jankrassnigg]

I'll see what I can do in the next few days. Though it might be easier to create a repro case directly with our engine.

[guillaumeblanc]

Hi,

I created a sample to help with the repro / debug: https://github.com/guillaumeblanc/ozz-animation/tree/hotfix/2bik/samples/two_bone_ik_dbg

If you can provide the skeleton, the animation, the pole and target positions, then I can look at it.

Hope it helps,
Guillaume

[jankrassnigg]

Thanks for the sample, that was extremely helpful! Here's a video showing the issue. If you place the pole target just right, you also get that case where it flips back and forth just from the animation playback.

ozz-ik.mp4

The changes are so trivial, I'll just attach them here.
Skeleton and animation files are in this zip file. I've also attached the .cc file, although the only change is the name of the bones.

media.zip

I didn't need to put in a custom pole vector, it's already reproduceable with just the UI.

The model is this one (free version):
https://quaternius.com/packs/universalanimationlibrary.html

It's just the "Idle_Loop" animation. However, I'm not using the gltf2ozz tool to convert, we have our own pipeline, but of course internally we use all your infrastructure, so that shouldn't make any difference.

[guillaumeblanc]

Hi,

Thanks for the repro!

The first thing I noticed and wanted to understand is that the hand joint does not reach the target (even though it's green, aka the job expects the target is reached). The reason for that is that the mid axis for the skeleton you provided is different. I got back to the doc, and found that the mid axis is (-1, 0, 0) for this skeleton.

Changes to the sample have been pushed.
Can you have a try and tell if it fixes your issue?

Also, I noticed instabilities around the softening algo when the shoulder is animated. I'll continue investigations to see if I can improve it.

Thanks also for pointing out to Quaternius assets, very useful.

Cheers,
Guillaume

[jankrassnigg]

A quick test seems to confirm, that this fixes the problem. However, now I'm even more confused about the options.

I think I used +Z, because that kept the hand rotated upwards. With -X the hand is rotated side-ways, so it's not a "grip" anymore. The goal was to use aim IK on the right arm, and then two-bone IK on the left one to have it follow the pistol hand.

So I tried to use the "twist_angle" option. However, as far as I can tell, that does exactly the same thing, as rotating the pole vector, or not? I thought I could use that to twist the end bone (here the hand), but it affects the entire chain, and I don't see when or why I would use that option, when I already have to specify a pole vector anyway. Here's a quick video:

pole-twist.mp4

Also I played some more with it with the aim IK and maybe this is expected, but when I rotate the pole vector 360 degree around the aim direction, there is one point where it's unstable:

aim-ik.mp4

Generally, I've always had a hard time understanding the comments on the IK options, because to someone without knowledge of this, they are a bit ambiguous.

• pole_vector - Pole vector, in model-space.

This "in model-space" was never fully clear to me. First of all, this is a direction vector, right? Originally I put a position in here, and it also seemed to work. But what IMO is a crucially missing point is, that this is always relative to the CURRENT bone position. That means you actually have to subtract the latest position from the target pole vector position. If you don't do that, then an animation that was applied right before the IK could completely mess up the pole vector direction.

• twist_angle - Twist_angle rotates joint around the target vector

As mentioned above, I don't see why this option exists, it seems redundant both on aim IK and 2-bone IK.

• soften - ...

I haven't figured out a case yet, where you'd want any other value than 1. Maybe this is more relevant for foot IK ? I haven't dabbled with that problem, yet.

Anyway, sorry for the information dump, this is a complicated topic and it's hard to debug.

[guillaumeblanc]

Hi,

Sorry for the delay.

I'll try to explain better the options. We'll then see if documentation can be updated.

I think I used +Z, because that kept the hand rotated upwards. With -X the hand is rotated side-ways, so it's not a "grip"

The middle joint axis is fixed for a skeleton, in this example it's the elbow. Its sign defines which side the rotation happens. See documentation.

On this screenshot, you can see that the elbow of this skeleton rotates around X axis (the red one). A positive rotation around X would close the arm, so you have to use -X.

I tried to use the "twist_angle" option. However, as far as I can tell, that does exactly the same thing, as rotating the pole vector, or not?

True, pole vector and twist angle both define the orientation of joint chain plane. The twist angle is used to correct situations where the pole vector can not be placed like you want due to the flipping.

Here's Autodesk documentation, my reference for the two bone IK implementation. Hope it explains it well.

Also I played some more with it with the aim IK and maybe this is expected, but when I rotate the pole vector 360 degree around the aim direction, there is one point where it's unstable:

I fixed an issue that causes instability: cf77472, could you test it?
The problem could also be that your pole vector is aligned with the target, like described above.

This "in model-space" was never fully clear to me. First of all, this is a direction vector, right? Originally I put a position in here, and it also seemed to work.

Model-space means relative to the root of the skeleton (aka the model). This is a direction, right. If you put a position, it will only work when the character is at 0,0.

But what IMO is a crucially missing point is, that this is always relative to the CURRENT bone position. That means you actually have to subtract the latest position from the target pole vector position. If you don't do that, then an animation that was applied right before the IK could completely mess up the pole vector direction.

No, it's not relative to the current bone position. Have you experimented something pointing you to that conclusion?

soften

The original two bone ik sample demonstrates that. It prevents a snapping visual effect when target becomes reachable. It's probably not useful in the use case you're targeting.

The goal was to use aim IK on the right arm, and then two-bone IK on the left one to have it follow the pistol hand.

Sounds doable :

• Aim IK on right arm shoulder
• 2 bone IK on left arm
• Then maybe you don't even need Aim IK on the right hand. You can compute the local-space orientation of the hand (based on the pistol anchor), and recompute model-space transforms for all hand child joints.

Would be fun to build a sample with that use case.

Hope it helps,
Guillaume

[jankrassnigg]

Thanks a lot for the help. I'm currently also busy with other things, and may only get back to this in a while.

I fixed an issue that causes instability: cf77472, could you test it?

I'll try to test it soon-ish, thanks for looking into it.

No, it's not relative to the current bone position. Have you experimented something pointing you to that conclusion?

I may have phrased that badly. It is a direction, in model-space, but if you use it like I do, ie. you have a target position that a bone should point towards, then this direction changes as an animation moves the bone around. Simply because the bone position changes and thus the direction towards that point changes.

During my tests I've actually come to the conclusion, that the pole vector has to dynamically change as well. For example, I was playing around with using IK to have the arm move in an arc from left to right (sweeping motion). While pointing your right arm forwards, the pole vector should point to the right, but when you point to the far right, the pole vector has to point backwards. So if you procedurally drive the IK target, you also have to procedurally adjust the pole vector all the time.

This might be obvious to others, but it was a bit of a revelation to me.

[guillaumeblanc]

During my tests I've actually come to the conclusion, that the pole vector has to dynamically change as well.

It's not obvious, thanks for the sharing. Animators usually animate the pole vector and the twist angle, it makes sense that you have to procedurally adjust it.

[jankrassnigg]

I have finally tested your fix. I had mentioned that I had this flip with aim IK, but your change was for two-bone IK. However, I think I now understand why my aim IK flips, so scratch that. Anyway, I did test your change with two-bone IK, but I don't see a difference in my cases.

I've added some debug visualization to my engine, which helped me understand some of the problems much better. Documentation is here, if you are interested:

• https://ezengine.net/pages/docs/animation/skeletal-animation/ik/aim-ik-component.html#debug-visualization
• https://ezengine.net/pages/docs/animation/skeletal-animation/ik/two-bone-ik-component.html#debug-visualization

Now I think I understand all of my problems with aim IK, but for two-bone IK I'm still a bit confused. Here's a video with debug visualization:

2b-ik-flip.mp4

The documentation says "IK chain orientation will flip when target vector and the pole vector are aligned/crossing each other." However, I don't see how this would be the case here. The start joint and target position are nearly orthogonal to the pole vector direction.

For reference, this is the start position of the arm before IK is applied:

The gizmo is at the position of the pole vector. During IK I calculate the pole vector direction as the direction from the mid-point between the start joint and the target position towards the pole vector position (the cyan arrow in the video).

I tried to understand the Maya docs, but so far no luck :D