Current converters of ByteString and ByteArray to Natural in Cardano.Crypto.Utils first unnecessarily convert to Integer and the later convert it to Natural
There are exactly the same primitve operations available for Natural conversion in GHC.Num.Natural as there are for GHC.Num.Integer, thus such intermediate step can be avoided.
Additional property testing to make sure such switch is safe will be required.