Refactor to avoid *netip.Addr / *netip.Prefix

[louisroyer]

We don't need pointers, if the address is not initialized this is correctly detected by omitempty. Use IsValid() instead of == nil.