Unable to load native library for native activity on Android 5.1

Running the example from the README, with this added to Cargo.toml:
```
[package.metadata.android.sdk]
min_sdk_version = 22
```

I get an exception on Android 5.1 arm64-v8a that it's unable to load the so file, no further clues.
I had the same issue with winit:
https://github.com/rust-windowing/glutin/issues/1644

Any ideas on how to proceed?