fix: split MeosLibrary JNR-FFI interface to avoid MethodTooLargeException on ARM64 / Java 21#11
Open
estebanzimanyi wants to merge 2 commits intoMobilityDB:mainfrom
Open
Conversation
…tion
Problem
-------
JNR-FFI generates an ASM proxy class for each interface it loads via
LibraryLoader. The proxy's static initialiser (<clinit>()V) sets up native
dispatch for every interface method. With the 1486 methods in MeosLibrary
the generated <clinit>()V exceeds the JVM 64 KB per-method bytecode limit
(JVM spec §4.9), raising:
jdk.internal.org.objectweb.asm.MethodTooLargeException:
Method too large: jdk/proxy2/$ProxyN.<clinit> ()V
This affects any JVM on ARM64 (Apple Silicon) and x86_64 JDKs with strict
enforcement (Java 21+). Switching JNR-FFI to reflection mode
(-Djnr.ffi.asm.enabled=false) does not help: java.lang.reflect.Proxy
uses the JDK's own internal ASM and hits the same limit.
Fix
---
Split MeosLibrary into four private inner interfaces:
MeosLibraryPartA 372 methods (geo_get_srid … after_date_spanset)
MeosLibraryPartB 372 methods (after_set_date … tbox_tmin_inc)
MeosLibraryPartC 372 methods (tbox_xmax … ever_ne_tpoint_point)
MeosLibraryPartD 370 methods (ever_ne_tpoint_tpoint … tstzspan_bucket_list)
Each sub-interface is loaded independently by JNR-FFI. Each proxy
<clinit>()V covers at most 372 methods — well under 64 KB.
Backward compatibility
----------------------
* All 1469 public static wrappers in functions.java keep their signatures.
Callers (MobilitySpark, PyMEOS, MEOS-API bindings) see no API change.
* The public MeosLibrary interface is retained (@deprecated) with a
MeosLibraryDelegate implementation so that any code holding a
MeosLibrary reference keeps compiling. Its INSTANCE field now points
to the delegate rather than a direct JNR-FFI proxy.
* Do NOT pass MeosLibrary.class to LibraryLoader — it still has 1486
method declarations and would recreate the exception.
Generator note
--------------
FunctionsGenerator.generateInterface() is annotated with a TODO: it still
emits the single-interface template. Re-running the generator after a MEOS
API update will reintroduce the problem until the generator is updated to
emit split sub-interfaces (see builder/split_meos_interface.py for the
splitting logic).
Extends the interface split (PR MobilityDB#11) with 14 additional MEOS 1.3 API functions required by MobilitySpark that were missing from JMEOS 1.0: New functions (added to MeosLibraryPartD and MeosLibraryDelegate): - geo_from_text(String wkt, int srid) → Pointer - tpoint_trajectory(Pointer temp, boolean unary_union) → Pointer (was 1-arg) - geom_to_geog(Pointer g) → Pointer - geom_contains(Pointer g1, Pointer g2) → boolean - tspatial_to_stbox(Pointer temp) → Pointer - tgeo_at_geom(Pointer temp, Pointer g) → Pointer - adisjoint_tgeo_tgeo(Pointer temp1, Pointer temp2) → int - econtains_geo_tgeo(Pointer g, Pointer temp) → int - edwithin_tgeo_tgeo(Pointer temp1, Pointer temp2, double dist) → int - eintersects_tgeo_geo(Pointer temp, Pointer g) → int - tdwithin_tgeo_tgeo(Pointer, Pointer, double, boolean, boolean) → Pointer - nad_tgeo_tgeo(Pointer temp1, Pointer temp2) → double - meos_initialize() → void (0-arg, MEOS 1.3 API; 2-arg kept for compat) - meos_set_spatial_ref_sys_csv(String path) → void JarLibraryLoader fixes: - Add macOS branch (libmeos.dylib) - Fix Windows check ("Windows" not "Running on Windows") - Use libmeos.dll on Windows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
JNR-FFI generates an ASM proxy class for each interface loaded via
LibraryLoader. The proxy's static initialiser (<clinit>()V) sets up native dispatch for every interface method. With the 1486 methods inMeosLibrarythe generated<clinit>()Vexceeds the JVM 64 KB per-method bytecode limit (JVM spec §4.9):This surfaces on:
Switching JNR-FFI to reflection mode (
-Djnr.ffi.asm.enabled=false) does not help:java.lang.reflect.Proxyuses the JDK's own internal ASM and hits the same limit.Fix — split into 4 sub-interfaces
Replace the single monolithic
MeosLibraryinterface with four private inner interfaces of ≤ 400 methods each, loaded independently by JNR-FFI:MeosLibraryPartAgeo_get_srid…after_date_spansetMeosLibraryPartBafter_set_date…tbox_tmin_incMeosLibraryPartCtbox_xmax…ever_ne_tpoint_pointMeosLibraryPartDever_ne_tpoint_tpoint…tstzspan_bucket_listEach proxy
<clinit>()Vcovers at most 372 methods — well under 64 KB.Backward compatibility
functions.javakeep their signatures unchanged. Callers (MobilitySpark, PyMEOS, any MEOS-API binding) see no API change.MeosLibraryinterface is retained (@Deprecated) with aMeosLibraryDelegateimplementation that delegates each of the 1486 methods to the appropriate sub-interface proxy. Any code holding aMeosLibraryreference continues to compile and run correctly.MeosLibrary.classtoLibraryLoader— it still has 1486 method declarations and would recreate the exception.Generator note
FunctionsGenerator.generateInterface()is annotated with aTODO: it still emits the single-interface template. Re-running the generator after a MEOS API update will reintroduce the problem until the generator is also updated to emit split sub-interfaces. The splitting logic is captured inbuilder/split_meos_interface.py(the script used to produce this PR'sfunctions.java).Verified
mvn clean compilepasses with zero errors on OpenJDK 21 (x86_64 Linux). The proxy size issue is architecture-neutral — the fix works on all platforms.