Explore adding bindings to the webview library to allow creating web-based frontends

[rdw-software]

So this is a weird idea (and probably stupid), but I recently stumbled upon webview while working on some unrelated frontend stuff. Now the big question is, what if we called into that from Lua? (or C, for that matter) It sounds so absurd that it might just work...

Goals:

• Can create a fully-featured web frontend (including WebGL/WebAudio and CSS-styled UIs) with full control over the backend
• Can access the builtin libraries from the JavaScript environment (similar to how NodeJS is exposed in Electron)

Roadmap:

• Implement run_once API to allow integrating webview with libuv's event loop from C
• Add LuaJIT FFI bindings to access the webview APIs
• Implement a pull-style API for exchanging messages with the JavaScript context
  • Option 1: SharedArayBuffer (need to check platform support)
  • Option 2: Transferable buffers with window.postMessage (need to check how to get at the buffer from C/Lua)
  • Option 3: Local hosted HTTP server (bad solution, better find a direct way of messaging that works)
  • Option 4: IPC or WebSockets (still too much overhead?)
  • Option 5: Call into C from JS via webview_bind (need to check if it can exchange data)
  • Just use the default JSON RPC (will suck for large messages, so use HTTP/WS for those?)
  • At this point, might as well use WebSockets exclusively and get messages and streaming via TCP for free
• Tests for typical scenarios, like TCP messaging or file system requests
• Benchmarks for the various options (if there are several that turn out to work)
• Investigate native platform capabilities to determine what/if other APIs to add (must exist on all platforms)
• Add some documentation for the entire thing (API, background info, limitations, design decisions)
• Upstream as many API changes as possible (if they will take them)
• Update the build process to statically link webview (and the native SDK dynamically)
• Update the changelog since this is a "notable change"

What are the drawbacks? Well, there's a few...

• The library is tiny and it's unclear how well it is maintained... it looks to be actively developed now, but that wasn't always the case
• The current threading model doesn't makes it easy to control the main loop from outside, though it seems that could be changed
• The docs are really bad, which doesn't inspire confidence. Then again, the library is a tiny wrapper and proper docs could be added
• On Windows, the WebView2 runtime cannot currently be linked statically due to Microsoft holding the entry point hostage (proprietary Loader DLL) - this might eventually change since there's a PR that seems to replace it with an open source wrapper
• Even if you did link it statically, the browser runtime is gigantic (like, 250 MB) so that would be even worse than Electron
• On the other hand, since it's provided by the operating system and NOT embedded this should generally be pretty lightweight
• However, it MUST be present on the system. That is not the default for Windows 10, so you need to ship it to end users
• Not all of the features that the native browser implements are exposed, in fact most aren't - Not sure what exactly is missing
• You would be running a JavaScript engine (V8) AND a Lua engine (LuaJIT), which is pretty wacky on a conceptual level

[rdw-software]

So there are two main difficulties in integrating the library, and one annoyance:

1. There's no run_once API to allow interweaving the webview loop with libuv's event loop
2. How should the libuv APIs (and others) be exposed to the frontend? Callbacks from C to Lua are SLOW - that leaves IPC?
3. What about native APIs for things one might want to do that aren't already available? Reimplement here, PR to usptream? Will they accept it (their goal seems to be to keep it minimal)?

[rdw-software]

One more concern: What about the different browser backends? Do they have similar capabilities? EDGE is just Chromium and the Linux WebGTK looks alright, but I'm sure there's going to be at least some gotchas when it comes to adding more native APIs (point three), e.g. for opening native file dialogs etc. - all the stuff that Electron already provides.

[rdw-software]

As for the missing APIs, I guess there would need to be a libuv-like abstraction for the three main platforms, that is compatible with how webview does things internally. The easiest would be for them to merge that in if it could be developed, but then I can't really do OSX development efficiently so it's unlikely to happen anytime soon anyway. The main features I would be using are:

• HTML/CSS based UI (loaded from disk or a local HTTP server)
• WebGL rendering and WebAudio, input etc. (basically BJS)
• Probably typed arrays and similar builtin APIs, which may not be as performant as NodeJS buffers (or LuaJIT string buffers)?
• Possibly WebSockets if I can't get to the libuv TCP APIs from JavaScript without incurring a lot of overhead

I would also be interested to see how far into the browser's innards we can reach. Probably depends on the OS API (not very far)?

[rdw-software]

Exchanging data efficiently with the frontend would be annoying. The options are:

• SharedArrayBuffer and atomic locks (yikes). Doesn't work on WebkitGTK AFAIK?
• Pass everything via process messages and JSON RPC (yeah, no...)
• Send HTTP requests from frontend to backend... which is probably OK for one-way polling, but not great overall
• Stream via WebSockets - that will make for one hell of a bumpy ride compared to just calling Node APIs in Electron
• Create bindings for every required API that can be accessed from JS (but must use C and not Lua due to the FFI callback issue)

So I guess HTTP requests for data fetching combined with a localhost server might work. If more is needed, either a way to use shared buffers in non-Windows frontends would have to be found, or the C functions might need to be bound (again)...

[rdw-software]

Also worth keeping in mind what kind of tasks would actually have to be handed off to the backend:

• File system access
• TCP connections (if not using WebSockets, which is likely easier even if there's some 2-byte overhead per message)
• UDP connections (I guess that's not really needed though)
• Decoding and decompression tasks, if they can't be handled by the browser engine (seems unlikely -> just use REST/HTTP calls?)
• Anything that could benefit from multithreading and is easier to do with HTTP requests + backend computation compared to WebWorkers

In this design, the backend would be the Lua/C engine and it can do anything a regular backend server could do... plus interact with the frontend directly if needed. It seems a bit odd to separate the UI and runtime this way, but I guess that's web apps for you.

The good news is that this would keep the frontend largely portable so the code can always be run in a regular browser, unless closer integration is needed (then JSON RPC or shared buffers might be used).

[rdw-software]

Overall I like the HTTP-based frontend/backend flow best, even if it's not ideal. There are several fallback options to explore if need be, so hopefully this design would still be flexible enough and yet relatively simple. I guess you could even do most API calls via HTTP, i.e., frontend requests runtime APIs via REST, or WebSockets (with significant overhead...). That wouldn't be much better than JSON RPC, though likely more optimized by the browser engine?

OTOH, the things that you might request from the backend are almost always heavily CPU-bound tasks or I/O-bound things that won't significantly be impacted by the overhead of a single HTTP request/response on top. Smaller stuff can just be done in JS land, which I guess means that the overhead isn't a real concern. Or it would be fast enough anyway, even with HTTP.

The only "sad" part is you cannot use NodeJS packages that aren't compatible with browsers, but that's a given (and the most popular ones might support both environments) when working on the Lua/C level. The trade-off is not running electron, i.e. more control and significantly less bloat - no more 150MB hello world apps. Is that worth it? Maybe...

Oh, and bonus points for getting to avoid all UI/sync API issues. Everything is async/await and sync stuff handled by the backend.

[rdw-software]

I think I'll just build a quick prototype for a minimal integration, then move the rest to separate issues if needed.