VFSClassLoader alternative #30

dlmarion · 2025-09-25T14:58:15Z

No description provided.

dlmarion · 2025-09-25T15:02:45Z

I had started working on a design for a VFSClassLoader replacement a few weeks ago. My design deviates from the requirements in #27 based on my prior work on the classloader implementations in Accumulo.

ddanielr

I understand this is a draft so this feedback may not end up being implemented.

...caching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/manifest/Manifest.java

...caching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/manifest/Resource.java

...src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java

...l-caching-classloader/src/main/java/org/apache/accumulo/classloaders/lcc/state/Contexts.java

...classloader/src/main/java/org/apache/accumulo/classloaders/lcc/state/ContextClassLoader.java

…umulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java Co-authored-by: Daniel Roberts <ddanielr@gmail.com>

…umulo/classloaders/lcc/state/Contexts.java Co-authored-by: Daniel Roberts <ddanielr@gmail.com>

…o-classloaders into new-classloader

...src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java

…anifest changes

...-classloader/src/main/java/org/apache/accumulo/classloader/lcc/state/ContextClassLoader.java

dlmarion · 2025-11-17T17:46:59Z

Build is failing due to no new-line in no-arg constructors for ContextDefinition.java and Resource.java. Not sure what's happening here with the formatting plugin. @ctubbsii - any ideas?

Update: I got it working by modifying the checkstyle rules in the pom to match the rules in the main Accumulo pom for the RightCurly rule.

...loader/src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoader.java

...ching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContext.java

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

...g-classloader/src/test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextTest.java

...hing-classloader/src/test/java/org/apache/accumulo/classloader/lcc/cache/CacheUtilsTest.java

modules/local-caching-classloader/README.md

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

ctubbsii · 2025-11-20T00:20:49Z

I noticed some things that weren't quite right here, with the use of URI vs. URL, and File vs. Path, with regard to the modernizer and other QA checks, and noticed that some of the QA tools weren't configured and running in this project, or weren't up-to-date. I've updated them in a separate PR, #32 that should be merged before we make any additional changes, to ensure that new additions to the repo comply with our QA checks.

keith-turner

This looks really good. Saw some things that could be potential follow ons and a few small little things that could be changed.

...ching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContext.java

keith-turner · 2025-11-20T21:53:46Z

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

+    final URL jarCJettyLocation = jetty.getURI().resolve("TestC.jar").toURL();
+
+    // ContextDefinition with all jars
+    ContextDefinition allJarsDef = createContextDef("all", jarAOrigLocation, jarBHdfsLocation,


This could be a follow on, would be helpful to create a little main class or script that a user could point at a dir and it spits out some json. That could be added to readme.

...s/local-caching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/Constants.java

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

...-caching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/cache/CacheUtils.java

...assloader/src/main/java/org/apache/accumulo/classloader/lcc/resolvers/LocalFileResolver.java

...src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java

keith-turner · 2025-11-20T23:21:18Z

...src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java

+          ContextDefinition def = parseContextDefinition(contextLocationUrl);
+          LocalCachingContext newCcl = new LocalCachingContext(def);
+          newCcl.initialize();
+          newlyCreated.set(true);


Why not call monitorContext here?

Initially the code did have the call to monitorContext here. It presented an issue because this is in a Function where the entry is not yet in the contexts Cache. It added some complexity to monitorContext when trying to look up the entry in contexts.

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

…umulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java Co-authored-by: Keith Turner <kturner@apache.org>

…umulo/classloader/lcc/cache/CacheUtils.java Co-authored-by: Keith Turner <kturner@apache.org>

keith-turner · 2025-11-21T20:14:20Z

Made a comment on a commit ea0501b#r171069412

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java

Updates from PR suggestions

dlmarion · 2025-11-24T21:16:08Z

Made a comment on a commit ea0501b#r171069412

Addressed this comment in a46d8e1

...test/java/org/apache/accumulo/classloader/lcc/MiniAccumuloClusterClassLoaderFactoryTest.java

Initial commit for VFSClassLoader replacement

59515ab

ddanielr reviewed Sep 25, 2025

View reviewed changes

dlmarion and others added 4 commits September 25, 2025 16:51

Update modules/local-caching-classloader/src/main/java/org/apache/acc…

afb6c2e

…umulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java Co-authored-by: Daniel Roberts <ddanielr@gmail.com>

Update modules/local-caching-classloader/src/main/java/org/apache/acc…

e973046

…umulo/classloaders/lcc/state/Contexts.java Co-authored-by: Daniel Roberts <ddanielr@gmail.com>

Minor updates

d6f1750

Merge branch 'new-classloader' of https://github.com/dlmarion/accumul…

fa9a5c0

…o-classloaders into new-classloader

dlmarion commented Sep 26, 2025

View reviewed changes

...src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java Outdated Show resolved Hide resolved

Removed independent monitoring of contexts, should only change when m…

ed03145

…anifest changes

jschmidt10 reviewed Oct 22, 2025

View reviewed changes

...-classloader/src/main/java/org/apache/accumulo/classloader/lcc/state/ContextClassLoader.java Outdated Show resolved Hide resolved

jschmidt10 reviewed Oct 22, 2025

View reviewed changes

...-classloader/src/main/java/org/apache/accumulo/classloader/lcc/state/ContextClassLoader.java Outdated Show resolved Hide resolved

dlmarion added 3 commits November 14, 2025 19:54

Updates, new tests, still wip

a13e28d

Modify pom to run tests in forked VM serially

c4f658e

more tests, cleanup, added readme

9b91a94

dlmarion marked this pull request as ready for review November 17, 2025 17:47

dlmarion requested review from ctubbsii, ddanielr, ivakegg, jschmidt10 and keith-turner and removed request for jschmidt10 November 17, 2025 17:47

dlmarion added 2 commits November 17, 2025 19:00

Fixed build issues

375114c

Fix javadoc

5890ecf

dlmarion changed the title ~~Initial commit for VFSClassLoader replacement~~ VFSClassLoader alternative Nov 18, 2025

jschmidt10 reviewed Nov 19, 2025

View reviewed changes

...loader/src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoader.java Outdated Show resolved Hide resolved

jschmidt10 reviewed Nov 19, 2025

View reviewed changes

...ching-classloader/src/main/java/org/apache/accumulo/classloader/lcc/LocalCachingContext.java Outdated Show resolved Hide resolved

Modified cache from weakValues to expireAfterAccess(24 hours)

c3adb17

keith-turner reviewed Nov 20, 2025

View reviewed changes

ctubbsii mentioned this pull request Nov 20, 2025

New HDFSContextClassLoaderFactory #28

Closed

dlmarion added 4 commits November 20, 2025 19:53

Addressed PR suggestions, refactored tests for better readability

f4387dd

Merge branch 'main' into new-classloader

70b906a

Changes from merging in main, fixing conflicts and new build issues

981fed8

Add new tests

303a1da

keith-turner approved these changes Nov 20, 2025

View reviewed changes

dlmarion and others added 5 commits November 21, 2025 06:47

Update modules/local-caching-classloader/src/main/java/org/apache/acc…

0067f8b

…umulo/classloader/lcc/LocalCachingContextClassLoaderFactory.java Co-authored-by: Keith Turner <kturner@apache.org>

Update modules/local-caching-classloader/src/main/java/org/apache/acc…

10677ec

…umulo/classloader/lcc/cache/CacheUtils.java Co-authored-by: Keith Turner <kturner@apache.org>

Applied PR suggestions, updated README

f2bd69e

Changed directory from system property to property passed via init

ea0501b

fix javadoc

e5b78cb

keith-turner reviewed Nov 24, 2025

View reviewed changes

...test/java/org/apache/accumulo/classloader/lcc/LocalCachingContextClassLoaderFactoryTest.java Show resolved Hide resolved

dlmarion added 2 commits November 24, 2025 19:23

Updates from PR suggestions

1d7a969

Merge pull request #1 from dlmarion/new-classloader-updates

a46d8e1

Updates from PR suggestions

dlmarion added 3 commits November 24, 2025 21:29

Updated README for new update grace period property

396d76f

Added test with Mini Accumulo and multiple tservers

b2ab523

Add missing minicluster dependency

4fa5aac

keith-turner reviewed Dec 2, 2025

View reviewed changes

Updates based on PR suggestions

2f63639

keith-turner reviewed Dec 2, 2025

View reviewed changes

...test/java/org/apache/accumulo/classloader/lcc/MiniAccumuloClusterClassLoaderFactoryTest.java Outdated Show resolved Hide resolved

dlmarion added 2 commits December 2, 2025 18:00

Updated with PR suggestion

911ac16

Rename test method

b167057

dlmarion merged commit 14eab97 into apache:main Dec 2, 2025
1 check passed

dlmarion deleted the new-classloader branch December 2, 2025 18:45

dlmarion mentioned this pull request Dec 4, 2025

Write an classloader that replaces VFS functionality. #27

Closed

VFSClassLoader alternative #30

VFSClassLoader alternative #30

Uh oh!

Conversation

dlmarion commented Sep 25, 2025

Uh oh!

dlmarion commented Sep 25, 2025

Uh oh!

ddanielr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dlmarion commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ctubbsii commented Nov 20, 2025

Uh oh!

keith-turner left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keith-turner Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keith-turner Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

dlmarion Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keith-turner commented Nov 21, 2025

Uh oh!

Uh oh!

dlmarion commented Nov 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dlmarion commented Nov 17, 2025 •

edited

Loading

keith-turner left a comment •

edited

Loading