Clarity requested on Media Reference's target_url format

[hww-clementpoulain]

Hi there,

I was wondering if any of those 2 pages were slightly out of date regarding how the Media Refence target_url should be formatted by write adapters.
Of if read adapters should be handling both cases?

The read the docs page here mentions it should have a file:// prefix.
Where the unit tests here mention it should be raw.

The confusion emerged from Adobe Premiere Beta 26.1 exporting OTIO files with media references with said file:// prefix.
And DaVinci Resolve 20.2 failing to import OTIO files formatted that way. (manually removing those prefixes let DaVinci Resolve import them just fine)
Which has been an issue reported a year ago in the Adobe community here , which can still be replicated today with above - recent - versions.

Best,
C

[reinecke]

Hi @hww-clementpoulain -

I'm going to preface this with one thing: My memory of how this all applies to Windows-style paths is weak, so most of what I say here is for POSIX style paths. There are some people in the community that may be able to provide more detail on the Windows path front if you're interested.

In OTIO we specify that URLs should be used - but these can be absolute or relative and technically be of various schemes (file://, https://, s3://, etc.).

URLs like /Volumes/My_Working_HD/media/sc1_take1.mov (forgive the bad faux filename) are still URLs, but relative anchored at the root. One way to think about this is with HTML links - many don't point to https:// full URLs - this allows them to behave the same when loading from a local filesystem or a remote URL. If you have an HTML file at https://mywebsite.com/about/index.html with a link like <a href="/index.html">Home</a> - this is a relative URL because the system infers the https://mywebsite.com part of it.

So, when loading an OTIO on a local machine both file:///Users/username/Movies/examplemedia.mxf and /Users/username/Movies/examplemedia.mxf should behave the same. There is a form of this URL that is file://Users/username/Movies/examplemedia.mxf (missing the leading / after the URL scheme) that is somewhat ambiguous (and even appears in our docs - whoops!). I think on POSIX systems in that case it is generally accepted that should resolve to a filepath anchored at the filesystem root - but this is technically a bad URL form.

If you want to go deep down the URL handling rabbit-hole, this utility module in OTIO python is built to cover a lot of the strange mixes of filesystem urls.

Trying the examples above we resolve:

>>> url_utils.filepath_from_url("file:///Users/username/Movies/examplemedia.mxf")
'/Users/username/Movies/examplemedia.mxf'
>>> url_utils.filepath_from_url("/Users/username/Movies/examplemedia.mxf")
'/Users/username/Movies/examplemedia.mxf'
>>> url_utils.filepath_from_url("file://Users/username/Movies/examplemedia.mxf")
'//Users/username/Movies/examplemedia.mxf'

And when handling an absolute and relative filesystem path to URL we get:

>>> url_utils.url_from_filepath("/Users/username/Movies/examplemedia.mxf")
'file:///Users/username/Movies/examplemedia.mxf'
>>> url_utils.url_from_filepath("Users/username/Movies/examplemedia.mxf")
'Users/username/Movies/examplemedia.mxf'

There are some other exotic cases in the url_utils tests.

Another data point is that the embedded content.otio file OTIOZ/D file bundle format writes URLs relative to the zip/directory root. There is some talk in the docs about how the adapter handles absolute vs. relative urls - I actually may disagree with the handling that:

the path is relative to the current working directory of the command running the adapter on the source timeline

I'd probably argue it should be relative to the location of the file containing the reference - when using an application like Premiere the CWD is of little relevance and probably even less awareness to users. This behavior would be a good candidate for an adapter arg - should be discussed as part of #1901.

So, the takeaways from this:

1. Some things like look like POSIX paths, but actually are relative URLs
2. There is a lot of documentation in OTIO that sort of hints at all this behavior and even that is inconsistent. That is bad and we should make documentation that directly addresses it - the Application Integrator's Guide is one of several places we should address this directly.
3. We should address more explicitly how absolute URLs should be resolved.

My proposition would be we treat this issue as one about our documentation - we can consider it done when our documentation updates are done?

[jminor]

Here are some other target_url use cases we've seen:

• Absolute paths:
  • Example: “/absolute/path/to/media.mov”
  • Example: “C:\absolute\path\to\media.mov”
• Relative paths:
  • Example: “subfolder/media.mov”
  • Note: OTIOZ makes use of relative paths to reference media contained within an OTIOD (folder) or OTIOZ (zip bundle)
• Absolute File URL:
  • Example: “file:///absolute/path/to/media.mov”
• Relative File URL:
  • Example: “file://subfolder/media.mov”
• HTTP URL:
  • Example: “https://domain.tld/path/to/media.mov”
  • Example: “https://domain.tld/path/to/media.mov?opt=val&opt2=val2”
• Proprietary AMS URI:
  • Example: “mystudio:project/sequence/shot/version.mov”
  • Example: “mystudio:subsystem/ID”

[reinecke]

Another thing we should note in the implementer's guide notes is some direction about how to think about URL schemes that should be handled and the possibility of searching for an alternate MediaReference from the selected one in the media_references collection as a fallback.

[hww-clementpoulain]

Thank you for all your replies and taking the time for it.

What I take from all of this and please correct me if wrong.
Is that DaVinci Resolve read adapter is the one that would need updating, to support target_url's URLs starting with any prefix (but more precisely "file://" in this case).
Potentially via the url_utils module shared by reinecke.

Looking through Blackmagic Forum's topics with keywords like "OTIO" and "target_url", it doesn't look like anyone raised this issue just yet.
I'll hold on creating an issue there for now as you will know best wether it should done before or after the OTIO docs update is done.
But I'm happy to do it or help coordinate between forums if need be.

It sounds to me that it isn't an Adobe Premiere write adapter issue though, so I could update the thread there with an update.

[jminor]

Two questions to the group:

1. Should file:// URLs be URL-escaped in any way? Specifically, should spaces, ampersands and such be percent escaped as %20 %26 etc?

2. Would it be helpful if otiotool had a feature for converting between absolute paths and file URLs?

[reinecke]

1. Should file:// URLs be URL-escaped in any way? Specifically, should spaces, ampersands and such be percent escaped as %20 %26 etc?

FCP 7 XML does this, but also is that because it's in XML? I wonder what Premiere is doing now in OTIO export. I think I'm leaning toward percent-encoding - it feels like it might be a bit more standard - but we should guide implementers to follow the robustness principle and try to be tolerant of things not percent-encoded.

2. Would it be helpful if otiotool had a feature for converting between absolute paths and file URLs?

This is a great idea! It could also be used as a general sanitizer for values to meet the recommended best practices.

[GraemeGriffiths]

Many thanks for this discussion FWIW I almost completely agree with Eric's conclusions.

1. Relative paths should be relative to the otio file
2. URI specifiers include not having the protocol specifier:
   So:
   "dir/file" -> "file://absolute path of dir containing the otio/dir/file"
   "/dir/file" -> "file:///dir/file"
3. URIs should have special characters % substituted.

As I've explained in the TSC meetings I'm currently working to allow export of timelines from our switcher into NLEs, we've chosen OTIO as it offers a standard solution that could be used by multiple NLEs without having to learn each NLE's file formats (although there is a certain amount of specificity as there's no current standard handling of things like Effects - that's another discussion).

Our otio file at the moment specifies just an absolute path with no file: protocol specifier. I did try initially to use relative paths but I could never get them to import in any NLE (no matter how many ../ I added and then by-hand specifying the path from root. (This should work since /.. -> /)

Currently we cannot offer Adobe Premiere as a working solution as it requires absolute paths defined for media elements. As we record the media to an SSD connected to our device the only absolute paths we can supply are those of the SSD where it mounts on our device. For the user to use that media they unmount the SSD and connect it to their NLE host - it's unlikely to be mounted in the same place. Currently to test compatibility I have a script that edits the URIs so if you run it when on the NLE host you get corrected URIs and can import happily to Premiere.

Incidentally the way Premiere deals with Windows paths is to add the drive specifier to the URI (which is unpleasant but, I guess, necessary): "file:///C:/dir/file". They do use Unix-style slashes, though.

Black Magic Resolve is odd - it can import our on-device absolute paths despite the SSD not being mounted in the same place. I can only assume they're doing some work to re-home the paths based on where they find the media files but I've no insight.

For my use-case if I could specify paths relative to the otio file - and, crucially, that would need to allow .. specifiers to access directories outside the current dir-branch - then I could do that in our offering and, assuming the NLE dev-teams adopt the latest OTIO, the NLEs could read them (whether the SSD is mounted or the user has copied the file trees intact).

I am aware that there is the option of creating an otioz file bundle but that would be a lot of file copying for us using up valuable processor cycles and disk-space so it's not really a usable option.

As I said above Adobe Premiere, for us, is blocked at this point so this really needs to go into a release ASAP so far as we're concerned (because to go to market we need NLE dev-teams to adopt that release).

[reinecke]

I think the next steps on this are:

• Draft new guidance to the Application Integrator's guide about how target_url should be populated (including that relative paths should be treated as relative to the OTIO file referencing them). This should likely include a lot of examples or reference unittests showing target_url values and their resolved absolute paths.
• Update utility code and the otioz adapter to to obey these