WKApp - A framework for building apps with modern HTML5 based user interfaces in a WKWebView #113
Replies: 1 comment 10 replies
-
|
@M4nw3l |
Beta Was this translation helpful? Give feedback.
-
|
@M4nw3l Unfortunately, marimo is not fully working in Pythonista 3. This is working:
This is not (yet) working:
I was able to overcome some of the problems by "monkey-patching" Pythonista's sys, os, and platform modules. import sys,os,io
import types
# Fix Pythonista3's sys.stdin/stdout/stderr
for sysstdstream, i, name in [(sys.stdin, 0, "stdin"), (sys.stdout, 1, "stdout"), (sys.stderr, 2, "stderr")]:
try:
_ = sysstdstream.errors
except (io.UnsupportedOperation, AttributeError):
sysstdstream.errors='strict'
try:
_ = sysstdstream.fileno()
except (io.UnsupportedOperation, AttributeError):
sysstdstream._fileno = i
sysstdstream.fileno = types.MethodType(lambda self: self._fileno, sysstdstream)
try:
_ = sysstdstream.name
except (io.UnsupportedOperation, AttributeError):
sysstdstream.name = name
_os_write=os.write
def os_write(fd,data):
file = {1:sys.stdout,2:sys.stderr}.get(fd,None)
if file:
print(data.decode('utf-8', errors='ignore'), end='',file=file)
file.flush()
return len(data)
else:
return _os_write(fd,data)
os.write=os_write
import platform
platform.system=lambda:'Darwin'Here are the module versions I installed to site_packages(user): I think we would need help from @omz to get marimo working in Pythonista, but for him PythonistaLab has probably a higher priority right now. For now, I have a marimo server installed on a raspberry pi 5 on my local network. I access it from my iPad using the Safari browser. That works very well when I am at home, but it would be really nice to have marimo running locally on the iPad. P.S.: I think the browser is a ui.WebView. To be honest, I was surprised that marimo automatically opened a new browser panel in Pythonista without me having to do anything. |
Beta Was this translation helpful? Give feedback.
-
I don't wanna use web as a basic 'frontend' for my framework, cause it require js runtime which is heavy for simple GUIs. So the basic idea to have ability to write easy-portable GUIs between pythonista and PC platforms using the common Pythonista.ui concepts. |
Beta Was this translation helpful? Give feedback.
-
|
@RichardPotthoff That’s a useful approach to know for fixing the platform setting, almost might as well add that to the I have been having a dig into how marimo’s WASM/Pyodide runtime is working from the typescript implementation here: From this I’ve ported some of the initial bootstrapping into a view that will run with WKApp below. While it isn’t fully set up enough to show a notebook yet, it does successfully bootstrap and start pyodide with marimo’s runtime. There’s some problems with packages installing correctly currently, however I suspect this might be just due to my implementation needing more work still. <%!
class WebAsmView:
def on_init(self):
pass
view_class = WebAsmView
%>
<%inherit file="view.html"/>
<script defer type="module">
import { loadPyodide } from "https://cdn.jsdelivr.net/pyodide/v0.29.3/full/pyodide.mjs";
async function pyodideWebWorker (){
const { loadPyodide } = await import("https://cdn.jsdelivr.net/pyodide/v0.29.3/full/pyodide.mjs")
let pyodideReadyPromise = loadPyodide({
});
self.onmessage = async (event) => {
// make sure loading is done
const pyodide = await pyodideReadyPromise;
const { id, python, context } = event.data;
// Now load any packages we need, run the code, and send the result back.
await pyodide.loadPackagesFromImports(python);
// make a Python dictionary with the data from `context`
const dict = pyodide.globals.get("dict");
const globals = dict(Object.entries(context));
try {
// Execute the python code in this context
pyodide.setStdout({ batched: (msg) => self.postMessage({ stdout: msg, id })});
pyodide.setStderr({ batched: (msg) => self.postMessage({ stderr: msg, id })});
const result = await pyodide.runPythonAsync(python, { globals });
self.postMessage({ result, id });
} catch (error) {
self.postMessage({ error: error.message, id });
}
};
}
async function marimoWebWorker (){
async function loadMarimoDeps(pyodide, code, foundPackages, force)
{
if (code.includes("mo.sql")) {
// We need pandas and duckdb for mo.sql
code = `import pandas\n$${code}`;
code = `import duckdb\n$${code}`;
code = `import sqlglot\n$${code}`;
// Polars + SQL requires pyarrow, and installing
// after notebook load does not work. As a heuristic,
// if it appears that the notebook uses polars, add pyarrow.
if (code.includes("polars")) {
code = `import pyarrow\n$${code}`;
}
}
// Add:
// 1. additional dependencies of marimo that are lazily loaded.
// 2. pyodide-http, a patch to make basic http requests work in pyodide
//
// These packages are included with Pyodide, which is why we don't add them
// to `foundPackages`:
// https://pyodide.org/en/stable/usage/packages-in-pyodide.html
code = `import docutils\n$${code}`;
code = `import pygments\n$${code}`;
code = `import jedi\n$${code}`;
code = `import pyodide_http\n$${code}`;
const imports = [...foundPackages];
// Load from pyodide
await pyodide.loadPackagesFromImports(code, {
//errorCallback: Logger.error,
// messageCallback: Logger.log,
});
// Load from micropip
const missingPackages = imports.filter(
(pkg) => !force && !pyodide.loadedPackages[pkg],
);
if (missingPackages.length > 0) {
await pyodide.runPythonAsync(`
import micropip
import sys
# Filter out builtins
missing = [p for p in $${JSON.stringify(missingPackages)} if p not in sys.modules]
if len(missing) > 0:
print("Loading from micropip:", missing)
await micropip.install(missing)
`).catch((error) => {
// Don't let micropip loading failures stop the notebook from loading
// Logger.error("Failed to load packages from micropip", error);
});
}
}
const decodeUtf8 = (array) => {
const str = new TextDecoder().decode(array);
return str;
};
function syncFileSystem(pyodide, populate) {
// Sync the filesystem. This brings IndexedDBFS up to date with the in-memory filesystem
// `true` when starting up, `false` when shutting down
return new Promise((resolve, reject) => {
pyodide.FS.syncfs(populate, (err) => {
if (err instanceof Error) {
reject(err);
return;
}
resolve();
});
});
}
const NOTEBOOK_FILENAME = "notebook.py";
const HOME_DIR = "/marimo";
const WasmFileSystem = {
NOTEBOOK_FILENAME,
HOME_DIR,
createHomeDir: (pyodide) => {
// Create and change to the home directory
const FS = pyodide.FS;
try {
FS.mkdirTree(HOME_DIR);
} catch {
// Ignore if the directory already exists
}
FS.chdir(HOME_DIR);
},
mountFS: (pyodide) => {
const FS = pyodide.FS;
// Mount the filesystem
FS.mount(pyodide.FS.filesystems.IDBFS, { root: "." }, HOME_DIR);
},
populateFilesToMemory: async (pyodide) => {
await syncFileSystem(pyodide, true);
},
persistFilesToRemote: async (pyodide) => {
await syncFileSystem(pyodide, false);
},
readNotebook: (pyodide) => {
const FS = pyodide.FS;
const absPath = `$${HOME_DIR}/$${NOTEBOOK_FILENAME}`;
return decodeUtf8(FS.readFile(absPath));
},
initNotebookCode: (opts) => {
const { pyodide, filename, code } = opts;
const FS = pyodide.FS;
const readIfExist = (filename) => {
try {
return decodeUtf8(FS.readFile(filename));
} catch {
return null;
}
};
// If there is a filename, read the file if it exists
// We don't want to change the contents of the file if it already exists
if (filename && filename !== NOTEBOOK_FILENAME) {
const existingContent = readIfExist(filename);
if (existingContent) {
return {
code: existingContent,
filename,
};
}
}
// If there is no filename, write the code to the last used file
FS.writeFile(NOTEBOOK_FILENAME, code);
return {
code: code,
filename: NOTEBOOK_FILENAME,
};
},
};
async function mountFilesystem(pyodide, code, filename) {
// Set up the filesystem
WasmFileSystem.createHomeDir(pyodide);
WasmFileSystem.mountFS(pyodide);
await WasmFileSystem.populateFilesToMemory(pyodide);
return WasmFileSystem.initNotebookCode({
pyodide: pyodide,
code: code,
filename: filename,
});
}
async function runMarimo(pyodideConfig, pyodide, code, notebook_run)
{
const nbFilename = NOTEBOOK_FILENAME;
await mountFilesystem(pyodide, code, nbFilename);
await loadMarimoDeps(pyodide, "", pyodideConfig.packages, true);
const [bridge, init, packages] = pyodide.runPython(
`
print("[py] Starting marimo...")
import asyncio
import js
from marimo._pyodide.bootstrap import create_session, instantiate
assert js.messenger, "messenger is not defined"
assert js.query_params, "query_params is not defined"
session, bridge = create_session(
filename="$${nbFilename}",
query_params=js.query_params.to_py(),
message_callback=js.messenger.callback,
user_config=js.user_config.to_py(),
)
def init(auto_instantiate=True):
instantiate(session, auto_instantiate)
asyncio.create_task(session.start())
# Find the packages to install
with open("$${nbFilename}", "r") as f:
packages = session.find_packages(f.read())
bridge, init, packages`
);
loadMarimoDeps(pyodide, code, packages.toJs()).then(notebook_run);
return bridge;
}
const { loadPyodide } = await import("https://cdn.jsdelivr.net/pyodide/v0.29.3/full/pyodide.mjs")
const pyodideConfig = {
packages: [
"micropip",
"msgspec",
"marimo-base",
"Markdown",
"pymdown-extensions",
"narwhals",
"packaging",
],
lockFileURL: `https://wasm.marimo.app/pyodide-lock.json?v=0.20.2&pyodide=v0.29.3`,
// Without this, this fails in Firefox with
// `Could not extract indexURL path from pyodide module`
// This fixes for Firefox and does not break Chrome/others
indexURL: `https://cdn.jsdelivr.net/pyodide/v0.29.3/full/`,
};
let pyodideReadyPromise = loadPyodide(pyodideConfig);
self.onmessage = async (event) => {
// make sure loading is done
const pyodide = await pyodideReadyPromise;
pyodide.setStdout({ batched: (msg) => self.postMessage({ stdout: msg, id })});
pyodide.setStderr({ batched: (msg) => self.postMessage({ stderr: msg, id })});
const { id, python, context } = event.data;
// make a Python dictionary with the data from `context`
const dict = pyodide.globals.get("dict");
const globals = dict(Object.entries(context));
try {
// Execute the python code in this context
const bridge = await runMarimo(pyodideConfig, pyodide, python, () => { });
self.postMessage({ bridge, id });
} catch (error) {
self.postMessage({ error: error.message, id });
}
};
}
function scrollToBottom (t) {
t.selectionStart = t.selectionEnd = t.value.length;
t.blur()
t.focus()
t.blur()
}
function writeln(text) {
let elem = $("#stdout")
text = elem.text() + '\n' + text;
elem.text(text);
elem.scrollTop(elem[0].scrollHeight);
}
function fn2workerURL(fn) {
const blob = new Blob([`($${fn.toString()})()`], { type: "text/javascript" });
return URL.createObjectURL(blob);
}
function getPromiseAndResolve() {
let resolve;
let promise = new Promise((res) => {
resolve = res;
});
return { promise, resolve };
}
function getId() {
return crypto.randomUUID()
}
// Add an id to msg, send it to worker, then wait for a response with the same id.
// When we get such a response, use it to resolve the promise.
function requestResponse(worker, msg) {
const { promise, resolve } = getPromiseAndResolve();
const idWorker = getId();
worker.addEventListener("message", function listener(event) {
if (event.data?.id !== idWorker) {
return;
}
if(event.data.stdout || event.data.stderr) {
writeln(event.data.stdout ?? event.data.stderr);
return;
}
// This listener is done so remove it.
worker.removeEventListener("message", listener);
// Filter the id out of the result
const { id, ...rest } = event.data;
resolve(rest);
});
worker.postMessage({ id: idWorker, ...msg });
return promise;
}
const pyodideWorker = new Worker(fn2workerURL(pyodideWebWorker), { type: "module" });
function pyodideAsyncRun(script, context) {
return requestResponse(pyodideWorker, {
context,
python: script,
});
}
async function runPyodide(){
try
{
writeln("Pyodide: run worker");
const { result, error } = await pyodideAsyncRun(`
def main():
print("stdout")
return "hello from" + " pyodide worker"
main()
`,{});
if (result) {
writeln(`Pyodide: $${result}`);
} else if (error) {
writeln(`Pyodide error: $${error}`);
}
}
catch(e)
{
writeln(`Error: $${e}`);
}
}
const marimoWorker = new Worker(fn2workerURL(marimoWebWorker), { type: "module" });
function marimoAsyncRun(notebook, context) {
return requestResponse(marimoWorker, {
context,
python: notebook,
});
}
async function runMarimo(){
writeln("Marimo: run notebook");
const { marimo, error } = await marimoAsyncRun(`
# notebook.py
import marimo
__generated_with = "unknown"
app = marimo.App()
@app.cell
def _():
import marimo as mo
mo.md("# Welcome to [marimo](https://github.com/marimo-team/marimo)! 🌊🍃")
return
if __name__ == "__main__":
app.run()
`,{});
if (marimo) {
writeln(`Marimo: $${marimo}`);
} else if (error) {
writeln(`Marimo error: $${error}`);
}
}
const pyodideConfig = {
packages: [
"micropip",
"msgspec",
"marimo-base",
"Markdown",
"pymdown-extensions",
"narwhals",
"packaging",
],
lockFileURL: `https://wasm.marimo.app/pyodide-lock.json?v=0.20.2&pyodide=v0.29.3`,
// Without this, this fails in Firefox with
// `Could not extract indexURL path from pyodide module`
// This fixes for Firefox and does not break Chrome/others
indexURL: `https://cdn.jsdelivr.net/pyodide/v0.29.3/full/`,
};
$(async function () {
writeln("Document: ready");
$("#window_secure_context").val(window.isSecureContext ? 'true' : 'false');
$("#window_cross_origin_isolated").val(window.crossOriginIsolated ? 'true': 'false');
writeln("Pyodide: init");
await loadPyodide(pyodideConfig);
//await runPyodide();
await runMarimo();
//let marimo = await runMarimo(pyodide);
});
//runPython();
</script>
<div class="p-1">
<p>
<label>window.isSecureContext</label><br />
<input id="window_secure_context" type="text" value="" />
<br />
<label>window.crossOriginIsolated</label><br />
<input id="window_cross_origin_isolated" type="text" value="" />
<br />
<style type="text/css">
textarea#stdout {
width : 95%;
height : 15vh;
overflow : auto;
font-style : monospace;
font-size : 10pt;
font-color : #808080;
}
</style>
<textarea id="stdout" readonly>
</textarea>
</p>
</div>
—— |
Beta Was this translation helpful? Give feedback.
-
|
@M4nw3l It has been a while since I installed marimo in Pythonista. If I remenber correctly, I had to copy the installed packages from my Raspberry Pi, because the installation involves a build process that creates the files in the I wrote a little program that allows the cloning and installation of python packages from a remote computer to Pythonista's If you enter e.g. the command |
Beta Was this translation helpful? Give feedback.
-
|
@RichardPotthoff I am not actually installing marimo into Pythonista at all here. This just needs
Then run app.py with Pythonista. My first attempt was taking a static html export and shoving it into a template, but it did not pan out too well unfortunately and is quite hard to analyse after Node.js has had its way with it. Although really this is just an experiment inspired by your mention of marimo. It seemed like a good way to try test some capabilities of this approach as it’s really quite flexible. Here’s a different small example that’s setting up preact in a view which is a purely in browser version of the React JS framework which doesn’t need to run through Node.js to work. <%!
class PreactView:
def on_init(self):
pass
view_class = PreactView
%>
<%inherit file="view.html"/>
<div id="root">
</div>
<script type="importmap">
{
"imports": {
"preact": "https://unpkg.com/preact@10.25.4/dist/preact.module.js",
"preact/hooks": "https://unpkg.com/preact@10.25.4/hooks/dist/hooks.module.js",
"htm": "https://unpkg.com/htm@3.1.1/dist/htm.module.js"
}
}
</script>
<script defer type="module">
import { h, render } from "preact";
import { useState } from "preact/hooks";
import htm from "htm";
const html = htm.bind(h);
function App() {
let [count, setCount] = useState(0);
return html`
<div>
<a href="/">Back to index.html</a>
<p>Count: $${count}</p>
<button onClick=$${() => setCount(count + 1)}>Increment</button>
</div>
<div>
</div>
`;
}
render(html`<$${App} />`, $('#root')[0]);
</script>
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I’ve been developing this app framework for a little while as a way to create web based user interfaces with just plain HTML5, CSS, JavaScript and Python in Pythonista with minimal code. It’s a comprehensive alternative to creating user interfaces with Pythonista’s shipped
uilibrary. Displaying application views instead in a native iOS WKWebView browser component, rendered from intermixed HTML/Python templates using the Mako templating engine and served with a local WSGI server provided by Bottle.https://github.com/M4nw3l/pythonista-wkapp
At its core a WKApp based application is just an extended Bottle application bundled with a WKWebView browser component to show it. And as the browser is just WebKit / Safari it supports all modern web technologies including HTML5 canvases, WebAssembly and modern JavaScript libraries/TypeScript bundles. Anything that displays and works in the Safari browser on your device, should also work in a WKApp view. As long as it supports using browser features marked as supported by WKWebView in Mozilla’s documentation it should run without modifications.
It’s intended to be lightweight and ready to use in Pythonista on install. After installing the pythonista-wkapp package with pip and StaSh, making an app is as simple creating app.py in a folder somewhere to run the application from and adding your HTML:
HTML views templates can then be added straight away into a
./viewsfolder relative to theapp.pyfile in a typical website structuring starting with anindex.html.Plain class instance backends are used to persist state, define functions/methods which can be called through interop. A mixin provides accessors to manipulate the page DOM and functions evaluate client side JavaScript on the page directly from Python. The interop mechanism supporting arbitrary bi-directional json compatible arguments passing between Python and JavaScript. While also GET and POST data values are passed and set automatically to view classes defining attributes of the same names.
It’s similar to frameworks like Electron or .NETs MAUI, from which it’s inspired by, except it focuses on Pythonista integration, and leaves cross platform support as a separate concern. Imagining working directly in Pythonista with just the onscreen keyboard as the default general use case.
Here’s some code showing an example
index.htmldemonstrating these features briefly, taken from the front page of the repository:Which then looks like this when it’s run in Pythonista:
The WKWebView component bundled with the library is an extended version of the WKWebView components for Pythonista found at:
This may also be imported and used independently from WKApp, if you so choose. It has some new features from the original versions:
window.isSecureWindowandwindow.isCrossOriginIsolatedcontexts.Also bear in mind it’s still fairly experimental at the moment, while most parts are somewhat stable there may be some bugs, oversights or missing integration features and it’s also a bit light on api documentation at the moment too. There’s only so much time I have to work on personal projects and documentation can take a while to do and is comparatively less fun and possibly harder than developing features, so there’s certainly more needed in this area currently. The project is open source so contributions are also very much welcome.
Some known issues at the moment:
pip install Makothen killing the command with StaSh when it’s got onto the MarkupSafe dependency should fix this circumstance.For a little bit of background, I started this project off the back of another that I knew would need some fairly sophisticated ui and while I could find extensions of the ui library and cross platform ui ports, I had already somewhat ruled this kind of thing out. As I aimed to create a desktop-class ui for the project but at the same time I also wanted avoid working on a laptop to be a change from my professional work. With web/html based UI approaches having been prevalent for a while however, for both the speed of development and flexibility of designs and functionality a browser can provide out of the box. I felt this sort of approach would also be ideal for developing an app’s ui with Pythonista as well as keeping to this newer norm. But with my searches for similar libraries/components falling a bit flat, this is what I’ve came up with after going down a few rabbit holes to get to here.
I would love to hear any thoughts and feedback you may have if you try it out? And I’m also wondering if there have been other attempts at creating something like this previously in Pythonista someone may know of? My searches for similar came up fairly empty before embarking on this although I possibly also just didn’t search hard enough and in the right places.
Beta Was this translation helpful? Give feedback.
All reactions