Second fix to filepaths outside the folder root with the XAR system, plus testing

[bbimber]

This is related to: #7261.

More background is listed on that issue. In 25.11, additional checks were added to disallow relative filepaths. The problem is that the XAR/Experiment system creates relative filepaths on data import. PR #7261 addressed one problem. After fixing this, I hit the following. This error happens a few lines after the original error. The gist is that LabKey writes out a XAR file for the run, and this file still contains forced relative URLs, even if not under the container's file root.

My proposal here is to make the DataFileURL behavior consistent between ExperimentRun creation and XAR writing. The method "generatePathStringRelativeToRootIfUnderRoot" could certainly get a better name. I can add some code comments if you think this is a viable pathway.

If you dont like this, there's a second more involved option. The ExpDatas being written to XML store their original non-relative DataFileURL. The XAR code could read that and preferentially swap in that original URI. This is less invasive, but also raises the question of what's the point of writing a relative URI to that XML if we're just going to reverse it.

In addition to this fix, I created a contained test case that exercises the XarGenerator codepath. This can be used to illustrate this problem, and should be very easy to extend in the future as permissions are tightened. At the moment it exercises the situation of filepaths outside of the LK root; however, all the cross-folder situations I mentioned on #7261 would be easy to include.

Here is a stacktrace, from: https://prime-seq.ohsu.edu/Labs/Bimber/1976/pipeline-status-details.view?rowId=602940.

14 Dec 2025 07:58:31,535 ERROR: Path to parent not allowed: ../../../../1973/@files/sequenceOutputPipeline/SequenceOutput_2025-12-07_07-39-02/SingleCell.tca.ctd.PPG_Project2_Tissues_23230_T_NK.seurat.rds
java.nio.file.InvalidPathException: Path to parent not allowed: ../../../../1973/@files/sequenceOutputPipeline/SequenceOutput_2025-12-07_07-39-02/SingleCell.tca.ctd.PPG_Project2_Tissues_23230_T_NK.seurat.rds
	at org.labkey.api.exp.AbstractFileXarSource.canonicalizeDataFileURL(AbstractFileXarSource.java:126)
	at org.labkey.api.exp.XarSource.getCanonicalDataFileURL(XarSource.java:116)
	at org.labkey.experiment.xar.AutoFileLSIDReplacer.getReplacement(AutoFileLSIDReplacer.java:64)
	at org.labkey.api.exp.xar.Replacer$CompoundReplacer.getReplacement(Replacer.java:48)
	at org.labkey.api.exp.xar.LsidUtils.doReplacements(LsidUtils.java:154)
	at org.labkey.api.exp.xar.LsidUtils.resolveStringFromTemplate(LsidUtils.java:131)
	at org.labkey.api.exp.xar.LsidUtils.resolveLsidFromTemplate(LsidUtils.java:95)
	at org.labkey.api.exp.xar.LsidUtils.resolveLsidFromTemplate(LsidUtils.java:206)
	at org.labkey.experiment.XarReader.loadData(XarReader.java:1707)
	at org.labkey.experiment.XarReader.loadDoc(XarReader.java:418)
	at org.labkey.experiment.XarReader.parseAndLoad(XarReader.java:237)
	at org.labkey.experiment.api.ExperimentServiceImpl.importXar(ExperimentServiceImpl.java:2144)
	at org.labkey.experiment.api.ExperimentServiceImpl.importXar(ExperimentServiceImpl.java:2130)
	at org.labkey.experiment.pipeline.XarGeneratorTask.run(XarGeneratorTask.java:177)
	at org.labkey.api.pipeline.PipelineJob.runActiveTask(PipelineJob.java:831)
	at org.labkey.api.pipeline.PipelineJob.run(PipelineJob.java:1079)
	at org.labkey.pipeline.mule.PipelineJobRunner.run(PipelineJobRunner.java:40)
	at jdk.internal.reflect.GeneratedMethodAccessor634.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.mule.impl.model.resolvers.DynamicEntryPoint.invokeMethod(DynamicEntryPoint.java:312)
	at org.mule.impl.model.resolvers.DynamicEntryPoint.invoke(DynamicEntryPoint.java:259)
	at org.mule.impl.DefaultLifecycleAdapter.intercept(DefaultLifecycleAdapter.java:193)
	at org.mule.impl.InterceptorsInvoker.execute(InterceptorsInvoker.java:47)
	at org.mule.impl.model.DefaultMuleProxy.run(DefaultMuleProxy.java:470)
	at org.mule.impl.work.WorkerContext.run(WorkerContext.java:310)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1486)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:391)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:865)
	at org.mule.impl.work.ScheduleWorkExecutor.doExecute(ScheduleWorkExecutor.java:39)
	at org.mule.impl.work.MuleWorkManager.executeWork(MuleWorkManager.java:277)
	at org.mule.impl.work.MuleWorkManager.scheduleWork(MuleWorkManager.java:244)
	at org.mule.impl.model.seda.SedaComponent.run(SedaComponent.java:483)
	at org.mule.impl.work.WorkerContext.run(WorkerContext.java:310)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.base/java.lang.Thread.run(Thread.java:840)

[bbimber]

@labkey-jeckels and/or @XingY: this fully fixes the problem we're encountering, plus has a standalone test case if you want to reproduce it yourself. Would one of you be able to copy this FB to the LabKey side of the fork so TeamCity will run it?

You could run the XarTestPipelineJob integration test without the changes related to generatePathStringRelativeToRootIfUnderRoot() if you want to repro this. As noted in the main issue, I think XarTestPipelineJob is a platform we could use in the future to formally test file access situations, if you end up tightening permissions more.

[labkey-jeckels]

I mirrored the FB.

https://github.com/LabKey/platform/tree/25.11_fb_relPathsWithTest

I haven't reviewed yet. I may have time later today, or will definitely have time tomorrow.

[bbimber]

Thank you, and I don't expect anyone at labkey to review over the weekend. I appreciate the mirroring to at least get test data.

[bbimber]

Good morning - it seems like this is passing BVTs now: https://teamcity.labkey.org/buildConfiguration/LabKey_2511Release_Community_BvtAPostgres?branch=relPathsWithTest&buildTypeTab=overview. That includes the integration test I added, which provides direct coverage over this relative path LSID issue.

I think we can treat this as separate from the platform PR; however, I have some expanded coverage over cross-workbook pipeline jobs going here: BimberLab/DiscvrLabKeyModules#365.

Are there other test suites you'd like to see run?

[bbimber]

Note, the unrelated changes to sequence tests that exercise the cross-folder scenarios are now passing in this PR: https://teamcity.labkey.org/buildConfiguration/LabKey_2511Release_External_Discvr_ExternalModulesTestPostgres/3771811.

I am happy to make changes, but have no other plans to do anything further unless changes are requested.

[labkey-jeckels]

I reviewed the changes and I'm comfortable with the relative path handling. I pushed changes into the FB that streamline the use of FileLike, which should also make the merge into develop cleaner. @cnathe there will be conflicts with some method signature changes, but I think they should be pretty straightforward. Ping me if you have questions.

The new test passes locally for me. The commit should trigger the BVTs to rerun the new test. I queued the Sequence tests manually:

https://teamcity.labkey.org/buildConfiguration/LabKey_2511Release_External_Discvr_ExternalModulesTestPostgres/3772049

Questions for you, Ben:

1. Do we need to pass around the location in the test code? It seems like it's always WebServer. Could this be swapped for a simple static constant?
2. Of the scenarios you highlight at the top of XarTestPipelineJob for potential cross-folder references, which are concrete ones that are important for your use cases? They don't all need to be added here, but this can help guide when/if to add more coverage.
3. Do you care about XAR import/export scenarios with these kinds of files references, or just that the data imports cleanly as part of the job that did the analysis? In general, I'm OK with trusting Java code to reference the files it thinks are appropriate. We need to be much stricter for paths that could be (malicious) user-provided, like an uploaded XAR file.

[bbimber]

Hi @labkey-jeckels, see below:

1. Do we need to pass around the location in the test code? It seems like it's always WebServer. Could this be swapped for a simple static constant?

Yes, I see what you mean. XarTestPipelineJob was a quick fork from a different pipeline test class that needed to run on different locations. I'll remove this

2. Of the scenarios you highlight at the top of XarTestPipelineJob for potential cross-folder references, which are concrete ones that are important for your use cases? They don't all need to be added here, but this can help guide when/if to add more coverage.

All of those cases are from real scenarios as far as one container interacting with another. The specific scenarios are outlined in more detail here: https://www.labkey.org/ONPRC/Support%20Tickets/issues-details.view?issueId=54301 (search for 'Use Case 1'). Like I wrote there, this conceptually following Query/ContainerFilter rules.

3. Do you care about XAR import/export scenarios with these kinds of files references, or just that the data imports cleanly as part of the job that did the analysis? In general, I'm OK with trusting Java code to reference the files it thinks are appropriate. We need to be much stricter for paths that could be (malicious) user-provided, like an uploaded XAR file.

I dont ever import/export XAR. I barely do much with that piece of the pipeline codepath at all.

[labkey-jeckels]

@bbimber OK, thanks. Assuming TeamCity is happy with these changes, we can merge this tomorrow and separately consider the other scenarios to ensure compatibility with the additional changes already in develop.

[bbimber]

@labkey-jeckels, the bbimber<->labkey fork situation complicates this. I pushed a change, in addition to your FileLike change, that should be here: https://github.com/LabKey/platform/pull/7273/commits. Because TeamCity is running against the labkey form, we should really move to this PR, #7273 which is what's actually going to be committed. That branch is ahead of bbimber:25.11_fb_relPathsWithTest

[labkey-jeckels]

Closing in favor of #7273, which is now merged.

[bbimber]

excellent, thank you