Quick experiment: remove Ply dep#32
Draft
StachuDotNet wants to merge 42 commits intomainfrom
Draft
Conversation
Source kept on disk; restore when HTTP-server functionality returns. - fsdark.sln: comment out BwdServer Project, ConfigurationPlatforms, and NestedProjects entries. - Tests.fsproj: drop BwdServer ProjectReference; disable BwdServer.Tests.fs and HttpClient.Tests.fs (both reference BwdServer scaffolding). - Tests.fs: inline BwdServer.Server.initSerializers (three Json.Vanilla.allow calls) so surviving Serialization.* tests still have the registrations they need; drop the BwdServer/HttpClient test list entries and init/Wait scaffolding.
Language is now the lowest project in the chain. Files keep their
existing F# module declarations (e.g. ``module LibExecution.RuntimeTypes``),
so namespace paths are unchanged and 18 downstream projects don't
need any edits — they still ProjectReference LibExecution and pick up
the new sub-projects transitively.
Layering (lowest to highest):
Language Prelude only
DarkDateTime · PackageRefs (shared primitives)
ProgramTypes · ProgramTypesAst · ProgramTypesParser
Runtime Prelude, Language
RuntimeTypes · ValueType · Blob · Stream
Dval · Builtin · TypeChecker · Interpreter · AnalysisTypes
RTQueryCompiler · ProgramTypesToRuntimeTypes
DarkTypes Prelude, Language, Runtime
DvalDecoder · CommonToDarkTypes
RuntimeTypesToDarkTypes · ProgramTypesToDarkTypes
LibExecution umbrella; just Execution.fs
The package-ref-hashes.txt embedded resource and EnsurePackageRefHashes
MSBuild target moved to Language alongside PackageRefs.fs; .gitignore
updated.
Build: 32 s on this machine, same as before — no new floor introduced.
The watcher's log redirected from _vscode-post-start-command via shell ``&>>`` had been growing without bound (15 MB / 86 K lines on a long session). Now _build-server checks the file's size at startup and after every compile cycle: above 5 MB, the file is rewritten in place to keep only the most recent ~500 KB plus a header line noting the rotation. In-place rewrite is required because the shell-redirected fd is held open in append mode by the parent ``nohup`` process. Renaming the file would orphan that fd; truncating in place works because O_APPEND seeks to EOF before each write, so the next append lands at the new EOF post-rotation. Best-effort: any exception in the rotation path is swallowed so a broken rotate never kills the build server.
The watcher was being launched by the devcontainer's postStartCommand, so it ran whenever the container was up — even if no editor was attached. Agents that don't edit through VS Code paid the CPU cost and the false-build-failure noise (mid-edit saves the agent didn't make). Now: - Container start does a one-shot compile so binaries are fresh, then idles. No watcher. - ``.vscode/tasks.json`` gains a "Watch & rebuild backend" task with ``runOn: folderOpen`` so VS Code starts the watcher when the workspace opens and stops it when VS Code closes. - Agents can opt-in explicitly with ``./scripts/build/_build-server --watch``.
Until now telemetry.jsonl was CLI-only. Per-component .log files (build-server.log, fsharp-tests.log, packages-canvas.log, …) have been the only trail for humans, and the only trail at all for build/test events. Agents had to know which log to grep for what. Expanding the existing Telemetry mechanism to cover build + tests: - ``scripts/build/_build-server`` gains an ``emit_event`` helper and fires ``build.compile.start|end|fail`` per cycle and ``build.initial.start|end|fail`` for the cold compile, including ms timings. - ``backend/tests/Tests/Tests.fs`` now ``Telemetry.init``s the same ``rundir/logs/telemetry.jsonl`` and emits ``test.suite.start|end`` with the exit code. The per-component .log files keep being written exactly as before (humans tail them for streaming visual feedback). The .jsonl is the structured shadow — one file, JSON-per-line, greppable by ``event`` or ``ctx``. Format already matches what the F# Telemetry module and Dark-side traces use.
Reflects the changes from this branch: - LibExecution split into Language / Runtime / DarkTypes (umbrella). - BwdServer disabled at sln/fsproj level (source still on disk). - Watcher only runs when VS Code is attached; agents compile via ``_dotnet-wrapper`` or opt into ``_build-server --watch``. - ``build-server.log`` auto-rotates above 5 MB. - ``telemetry.jsonl`` is now the unified structured event log, fed by CLI / build-server / Tests entry points; per-component .log files keep streaming for humans tailing them. - ``package-ref-hashes.txt`` moved to ``backend/src/Language/``.
Defensive: a leftover obj/ cache from the pre-split layout can briefly recreate the old ``backend/src/LibExecution/package-ref-hashes.txt`` file before the next clean build. Ignoring both paths means the file won't sneak into a commit while the cache rolls over.
The split-LibExecution commit moved package-ref-hashes.txt to ``backend/src/Language/`` but missed three hardcoded references to the old ``LibExecution/`` path that only show up at runtime: - ``LocalExec/PackageRefsGenerator.fs`` and ``LibPackageManager/PackageRefsGenerator.fs`` both write the hash file via a hardcoded ``../LibExecution/package-ref-hashes.txt`` source-tree path. ``./scripts/build/reload-packages`` would happily regenerate hashes into the now-empty old location and the freshly- built CLI would read from the new location, so the runtime saw a zero-hash map → "Function ... couldn't be found" on every command. - ``Language/PackageRefs.fs`` looks up the embedded resource by ``LibExecution.package-ref-hashes.txt``, but after moving to the Language assembly the manifest name is ``Language.package-ref-hashes.txt``. Only matters for published / AOT builds (the source-tree path wins in dev), but a real bug nonetheless. All three updated to point at ``Language/``. ``reload-packages`` now writes to the right place and the embedded-resource fallback finds its bytes. Caught while exercising the new setup against the CLI; the kind of hardcoded-path landmine ``git mv`` doesn't surface.
Fantomas's preferred shape now that the strings fit on one line (post-rename from ``../LibExecution/...`` to ``../Language/...``). Caught running ``./scripts/formatting/format check`` before the B.1 baseline commit.
- Snapshot from ./scripts/run-local-exec bench appended to benchmarks/results/history.jsonl. - Captured the rows in scratch/ply-replacement/baseline.md (gitignored) with framework-drift notes. - Patched scratch/ply-replacement/10-baseline.md so the next agent uses the bench-script flow instead of the retired --filter-test-list measurement invocation. - Updated README state and progress log.
When ``run-in-docker`` is called from a non-TTY context whose stdin is a socket that never sees EOF — exactly the shape of a Claude ``run_in_background`` Bash invocation — the existing ``cat <&0 | fix_dir_stdin | docker exec -i ...`` pipeline blocks forever on the ``cat``: stdin never closes, so ``cat`` never finishes, so docker exec sits idle behind it, so the build never starts. Two B.3 build attempts hung this way before the diagnosis stuck. Branched the non-TTY case in two: - pipe / regular file / non-empty stdin → keep the existing ``cat`` pipeline, which is the path interactive shells and ``./scripts/run-in-docker expect ...`` use. - everything else (``/dev/null``, empty socket, anything where ``[-p][-f][-s]`` are all false) → skip the cat and just ``docker exec -i ... </dev/null``. Smoke-tested with ``./scripts/run-in-docker echo hi </dev/null`` and with the same agent-runtime invocation that previously hung.
Upstream tracing PR (`6370b2335 Tracing: merge trace_fn_results + trace_fn_arguments into trace_fn_calls`) merged two tables into one; ``internal.dark:17`` still hardcoded the pre-merge count of 23. Actual count today is 21 (verified via ``sqlite3 rundir/test-data.db .tables``). Caught running the full backend suite while landing T.1 — passed under the pre-rebase tip but errored on the rebased tip.
T.1 in scratch/ply-replacement (re-scoped: the original target —
LibExecution/Dval.fs — has no uply blocks anymore after the
blobs-and-streams cleanup; the hot path moved to Runtime/Stream.fs).
Changes:
- Prelude/Ply.fs: add ``Ply.ofTask`` for the reverse bridge. The
plan presumed both directions exist; only ``Ply.toTask`` did.
- Runtime/Stream.fs: swap ``pullImpl``, ``readNext``, ``readChunk``
from ``uply { } : Ply<...>`` to ``task { } : Task<...>``.
- The inner ``next`` of ``newChunked`` stays Ply because
``StreamImpl.FromIO``'s ``next`` field is still ``unit -> Ply<...>``;
cascading that field type is a later chunk. Bridges
(``|> Ply.toTask``) at the FromIO/Mapped/Filtered call sites
inside ``pullImpl`` keep the inputs flowing.
- Removed a ``return`` inside a ``while`` loop in ``readChunk``'s
fallback path that ``task { }`` (stricter than ``uply { }`` about
early returns inside loops) wouldn't accept; ``Exception.raiseInternal``
still throws so the behavior is identical.
- Tests/Stream.Tests.fs: stripped ``|> Ply.toTask`` bridges that
the tests added when the functions returned Ply — they now
return Task directly.
Build: 32 s. Tests: 10 134 / 10 134 passing (after the trace-table-
count fix in the immediately-prior commit).
T.2 in scratch/ply-replacement. Six builtin ``fn`` bodies converted
from ``uply { } : Ply<Dval>`` to ``task { } |> Ply.ofTask``:
- streamFromList, streamUnfold, streamNext, streamToList,
streamToBlob, streamMap
The ``fn`` field's type ``BuiltInFnSig`` still requires Ply (T.13
swaps the signature itself), so each task body is bridged with
``|> Ply.ofTask`` at the boundary.
Inner callbacks stay uply because their types are dictated by
``StreamImpl`` in ``RuntimeTypes.fs``: ``newFromIO``'s ``next``
parameter is ``unit -> Ply<Option<Dval>>`` and ``Mapped(_, fn, _)``'s
``fn`` is ``Dval -> Ply<Dval>``. Cascading those is a separate chunk.
Inside task bodies, ``let! v = somePly()`` now needs explicit
``|> Ply.toTask`` (the F# task builder doesn't bind Ply natively the
way Ply's uply binds Task). ``Ply known`` constants become
``Task.FromResult known``.
Build 32 s. **10 134 / 10 134 backend tests passing.**
T.3 in scratch/ply-replacement. Two outer ``BuiltInFnSig`` bodies
swapped from ``uply { } : Ply<Dval>`` to ``task { } |> Ply.ofTask``:
``httpClientRequest`` and ``httpClientStream``.
Inner blocks staying uply (constrained by external types):
- ``Ply.List.mapSequentially`` callbacks (Ply-typed contract)
- ``nextChunk`` callback for ``Stream.newChunked`` (FromIO callback type
in RuntimeTypes is still ``int -> Ply<Option<byte[]>>``)
The nested ``uply { match reqHeaders, method with ... }`` inside
``httpClientRequest`` flattened to ``task { ... }`` — its only Ply
binding is the outer-scope ``makeRequest`` which already returns
``Task<RequestResult>`` natively, so the rewrite is a no-op shape
change.
``Blob.readBytes state bodyRef`` and the ``Ply.List.mapSequentially
... |> Ply.map Result.collect`` chain bridge with ``|> Ply.toTask``;
``makeRequest`` and ``openStreamingRequest`` already return ``Task<_>``
so they bind directly.
Build 40 s. **10 134 / 10 134 backend tests passing.**
T.4 in scratch/ply-replacement. Bench snapshot appended on
``ply-to-task`` after T.1–T.3 (hot-path Ply→Task swap covering
``Runtime/Stream.fs`` + ``BuiltinExecution/Libs/{Stream,HttpClient}.fs``).
Every row is within GC noise of the baseline; the explicit 10 MB
streaming-shaped scenario (``streamToBlob 10 MB``) shows 0.02 %
difference on 20 MB allocated. The biggest delta in the table is
−1.9 % on ``manyBlobs 10000×256 B``, which is jitter.
Conclusion per the T.4 decision rule: the hot-path swap is neutral on
allocations, which is expected (``task { }`` and ``uply { }`` are both
struct-state-machine builders). Continue to T.5; the AOT payoff lives
at the trim-graph end of the plan, not here.
Detail in ``scratch/ply-replacement/iterations/01-task.md`` (gitignored).
T.5 in scratch/ply-replacement. ~21 ``BuiltInFnSig`` outer bodies
across six files swapped from ``uply { } : Ply<Dval>`` to
``task { } |> Ply.ofTask``:
- ``Builtins.fs`` (1) — ``getBuiltins`` simplified to ``Ply v``
since its body is sync (no awaits).
- ``Base64.fs`` (2) — ``base64Encode`` + ``base64UrlEncode``.
- ``String.fs`` (2) — ``stringFromBlob`` (both versions).
- ``NoModule.fs`` (2) — ``debug`` + ``toRepr``. ``Exe.dvalToRepr``
already returns ``Task<string>``, so it
binds in task context without ``|> Ply.toTask``.
- ``Crypto.fs`` (5) — sha256/sha384/md5/sha256hmac/sha1hmac.
- ``Blob.fs`` (7) — every ``Blob.readBytes``-binding builtin.
Inner uply blocks remaining (intentionally deferred to T.13's
``BuiltInFnSig`` swap):
- ``Stream.fs``: 5 callbacks for ``Stream.newFromIO`` / ``newChunked`` /
``Mapped`` / ``Filtered`` (callback types in RuntimeTypes still
``-> Ply<...>``).
- ``HttpClient.fs``: 3 ``Ply.List.mapSequentially`` / ``nextChunk``
callbacks bound to Ply-typed slots.
- ``Json.fs``: 4 helpers used by ``Ply.List.flatten`` /
``Ply.List.mapSequentially`` and the recursive ``convert`` whose
signature is ``... -> Ply<Dval>``.
- ``List.fs``: 2 internal sort-comparator helpers with explicit
``: Ply<unit>`` signatures.
Build 33 s. **10 134 / 10 134 backend tests passing.**
T.6 in scratch/ply-replacement. Single ``BuiltInFnSig`` body in
``BuiltinHttpServer/Libs/HttpServer.fs`` — ``httpServerServe`` —
swapped from ``uply { } : Ply<Dval>`` to ``task { } |> Ply.ofTask``.
The two helpers it awaits (``executeHandler`` and
``Http.Response.toHttpResponse``) already return ``Task<_>`` so the
inner ``let!`` binds are no-op shape changes.
The inner ``Func<HttpContext, Task>(fun ctx -> task { ... })``
ASP.NET request handler was already a `task { }` — unchanged.
Build 43 s, **10 134 / 10 134 backend tests passing**.
T.7 (partial — BuiltinCli portion). 16 outer ``BuiltInFnSig`` bodies
swapped from ``uply { } : Ply<Dval>`` across six files:
- ``Time.fs`` (1) — task wrap (``do! Task.Delay``).
- ``Posix.fs`` (1) — ``posixFdWrite`` task wrap with
``Blob.readBytes |> Ply.toTask``.
- ``Environment.fs`` (1) — ``getBuildHash`` collapsed to ``Ply v``
(sync, no awaits).
- ``Directory.fs`` (3) — all collapsed to direct ``Ply`` (sync).
- ``Process.fs`` (3) — all collapsed to direct ``Ply`` (sync).
- ``File.fs`` (7) — split: 3 task-wraps for the async file IO
(``ReadAllBytesAsync``, ``WriteAllBytesAsync``,
``AppendAllTextAsync``), 4 collapsed to
``Ply`` (sync attribute checks).
Pure-sync bodies use ``... |> Ply`` rather than ``task { return X }
|> Ply.ofTask`` — no point allocating a state machine for an
already-synchronous result.
Build 32 s, **10 134 / 10 134 backend tests passing**.
T.7 continues in the next iteration with BuiltinCliHost / BuiltinPM /
BuiltinCloudExecution (94 uply blocks across those three projects).
T.7 (continued — BuiltinCliHost portion). 13 outer ``BuiltInFnSig``
bodies + 1 internal helper migrated across three files:
- ``Canvas.fs`` (5) — DBCreate / GetOrCreateForAccount / DBListAll
/ DBDrop / UserGetByName. The DBListAll body
has a Ply.List.mapSequentially callback that
stays uply per the Ply contract.
- ``Traces.fs`` (6) — cliTracesList / cliTracesView / cliTracesListByFn
/ cliTracesGetInput / cliTracesClear, plus the
``loadFnCalls`` private helper (also migrated
to ``Task<Dval>`` since its only caller is now
in task context).
- ``Cli.fs`` (2 outer; 3 helpers retained as Ply) — cliParseAndExecuteScript
and cliEvaluateExpression converted; their
``parseCliScript`` / ``loadCanvasAndDBs``
/ ``execute`` helpers stay Ply and bridge with
``|> Ply.toTask`` at the call sites.
All ``Sql.executeAsync`` / ``Sql.executeRowAsync`` / ``Canvas.*`` /
``Account.getUserByName`` calls already return ``Task<_>``, so most
inner ``let!`` binds were no-op shape changes. The Ply-returning
helpers in Cli.fs needed explicit ``|> Ply.toTask`` bridges.
Build 48 s, **10 134 / 10 134 backend tests passing**.
T.7 continues with BuiltinPM (58) and BuiltinCloudExecution (18).
T.7 (continued — BuiltinCloudExecution portion). 15 outer
``BuiltInFnSig`` bodies in ``Libs/DB.fs`` swapped from
``uply { } : Ply<Dval>`` to ``task { } |> Ply.ofTask``:
- dbSet / dbGet / dbGetMany / dbGetExisting / dbGetManyWithKeys
- dbDelete / dbDeleteAll / dbGetAll / dbGetAllWithKeys / dbCount
- dbKeys
- dbQuery / dbQueryWithKey / dbQueryOne / dbQueryCount
All bind ``UserDB.*`` helpers; mix of Task-returning
(delete / deleteAll / getAllKeys / count) and Ply-returning
(set / getOption / getMany / getManyWithKeys / getAll /
executeCompiledQuery). The Ply ones bridge with ``|> Ply.toTask``;
the Task ones bind directly. ``compileQueryLambda`` (also Ply)
bridges similarly at each query call site.
Three internal helpers stay uply: ``resolveLoadValues``,
``compileQueryLambda``, plus a nested ``Ply.List.mapSequentially``
callback inside ``resolveLoadValues``. They have explicit
``Ply.Ply<...>`` signatures and are called from the converted
outer bodies via the ``|> Ply.toTask`` bridge — cascading them
into Task is left for T.13 (``BuiltInFnSig`` / ``DvalTask`` flip).
Build 45 s, **10 134 / 10 134 backend tests passing**.
T.7 continues with BuiltinPM (58 blocks).
T.7 (continued — BuiltinPM, partial). 10 outer ``BuiltInFnSig``
bodies converted to ``task { } |> Ply.ofTask`` across four files:
- ``Seed.fs`` (1) — pmSeedExport.
- ``Merge.fs`` (2) — scmMerge / scmCanMerge.
- ``Rebase.fs`` (2) — scmRebase / scmGetRebaseConflicts.
- ``Scripts.fs`` (5) — pmScriptsList / pmScriptsGet / pmScriptsAdd
/ pmScriptsUpdate / pmScriptsDelete.
All inner ``LibPackageManager.*.X`` and ``Scripts.*`` calls already
return ``Task<_>`` so `let!` binds were no-op shape changes — no
explicit ``|> Ply.toTask`` bridges needed in any of these.
Build 32 s. **10 134 / 10 134 backend tests passing**.
T.7 continues with BuiltinPM Branches (9) / PackageOps (13) /
Dependencies (6) / Packages (20) — 48 uply blocks remaining in
the chunk.
T.7 (continued — BuiltinPM, partial). 13 outer ``BuiltInFnSig`` bodies
swapped from ``uply { } : Ply<Dval>`` to ``task { } |> Ply.ofTask``:
- ``Dependencies.fs`` (4 outer) — depsGetDependents / depsGetDependencies
/ depsGetDependentsBatch
/ depsResolveLocations.
The ``Ply.List.flatten`` callback inside
depsResolveLocations stays uply
(Ply contract) and bridges via
``|> Ply.toTask``. The ``getLocationAny``
private helper also stays Ply.
- ``Branches.fs`` (9) — scmBranchCreate / scmBranchList /
scmBranchListAll / scmBranchGet /
scmBranchGetByName / scmBranchRename /
scmBranchDelete / scmBranchArchive /
scmBranchUnarchive.
All ``LibPackageManager.Branches.*`` and ``LibPackageManager.Queries.*``
calls already return ``Task<_>``, so binds were no-op shape changes
except inside ``depsResolveLocations`` where a
``Ply.List.flatten``-driven Ply pipeline kept its Ply shape and bridges
once at the boundary.
Build 32 s. **10 134 / 10 134 backend tests passing**.
T.7 continues with BuiltinPM PackageOps (13) + Packages (20) — 33 uply
blocks remaining.
T.7 (continued — BuiltinPM PackageOps). 13 outer ``BuiltInFnSig`` bodies converted: - pmStabilizeHashes (sync — collapsed to ``... |> Ply``). - scmAddOps / scmCommit / scmDiscard (try/with patterns; task wrap). - scmGetRecentOps / scmGetWipOps / scmGetWipSummary / scmGetWipItems / scmGetWipOpCount / scmGetCommitCount / scmGetCommits / scmGetCommitsForBranchChain / scmGetCommitOps (all simple ``Queries.X`` binds). All ``LibPackageManager.Inserts.*``, ``LibPackageManager.Queries.*``, and ``LibPackageManager.WipRefresh.*`` calls return ``Task<_>`` — no Ply.toTask bridges needed. Build 32 s, **10 134 / 10 134 backend tests passing**. T.7 continues with BuiltinPM/Libs/Packages.fs (20 blocks — last file).
T.7 (final BuiltinPM piece). 20 outer ``BuiltInFnSig`` bodies in
Packages.fs swapped to ``task { } |> Ply.ofTask``: pmGetStats,
pmFindType / pmGetType, pmFindValue / pmGetValue,
pmFindValuesByValueType, pmEvaluateValue, pmFindFn / pmGetFn,
pmSearch, pmGetLocationsByType / Value / Fn, pmGetAllPreviousHashes,
pmPropagate, pmAtomicUndo, pmGetDeprecationSets,
pmGetCurrentDeprecation.
Bridge work:
- ``LibPackageManager.Stats.get``, ``PMPT.Type/Value/Fn.find``,
``pm.getType/getValue/getFn``, ``RTPM.Value.findByValueType``,
``PMPT.search``, ``pm.get*Locations`` all return Ply, so each
``let!`` site got an explicit ``|> Ply.toTask``.
- ``Execution.executeExpr`` and
``LibPackageManager.Queries.getDeprecationSets`` already return
``Task<_>``, so no bridge there.
- ``LibPackageManager.Queries.getAllPreviousHashes``,
``LibPackageManager.Inserts.insertAndApplyOps``,
``LibPackageManager.Propagation.propagate``,
``LibPackageManager.Inserts.findCommittedHash``, ``Branches.*``
all already return Task.
- The ``match`` inside pmAtomicUndo had an inner uply branch that
produced ``Ok targetHash`` — switched to ``task { return Ok ... }``
to match the surrounding task block.
Build 29 s, **10 134 / 10 134 backend tests passing**. T.7 complete:
58/58 BuiltinPM blocks migrated; 110/110 across BuiltinCli +
BuiltinCliHost + BuiltinPM + BuiltinCloudExecution since T.7 started.
- changed `and DvalTask = Ply<Dval>` → `Task<Dval>` in Runtime/RuntimeTypes.fs
- bulk-converted Ply constructor sites at builtin fn-field tail positions
to Task.FromResult across BuiltinExecution/, BuiltinCli/, BuiltinCliHost/,
BuiltinCloudExecution/, BuiltinPM/, BuiltinHttpServer/, BuiltinDarkInternal/
- migrated BuiltinDarkInternal outer bodies (uply { } → task { })
- migrated tests/TestUtils/LibTest.fs
- 10 134 / 10 134 backend tests passing
- StreamImpl.FromIO `next`/`nextChunk`, Mapped.fn, Filtered.pred are now Task<...> in Runtime/RuntimeTypes.fs - Runtime/Stream.fs newFromIO/newChunked signatures + the inner byte-carry `next` task block; pullImpl/readChunk lose their Ply.toTask bridges - BuiltinExecution/Libs/Stream.fs cascades: streamFromList, streamUnfold, streamMap, streamFilter callback closures - BuiltinExecution/Libs/HttpClient.fs: nextChunk body - LocalExec/BenchmarkScenarios.fs: streamToBlob harness - Stream.Tests.fs + Blob.Tests.fs: list/chunk pull-fn helpers plus the in-test predicate/map closures - 10 134 / 10 134 backend tests passing
- readBytes / promote return Task<...>
- promoteWalk extracted to top-level `let rec` because the F# task
builder rejects `let rec` inside resumable code (FS3511)
- two internal bridges remain pending PackageManager-record
conversion: state.blobs.get and the insert callback (used
via do!) still come in as Ply
- callers across BuiltinExecution/Libs/{Base64,Blob,Crypto,
HttpClient,String}, BuiltinCli/Libs/{File,Posix}, and the two
test files drop their `|> Ply.toTask` bridges
- 10 134 / 10 134 backend tests passing
- getFnBody / partialEvaluate return Task<...> - two inner Ply.toTask bridges remain pending PackageManager record + Interpreter migration - sync call sites use .Result directly (drop |> Ply.toTask) - 10 134 / 10 134 backend tests passing
- executionPointToString, callStackString, rteToString return Task<string> - inner Ply.List.mapSequentially -> Task.mapSequentially - one Ply.ofTask bridge in LibCloudExecution/CloudExecution.fs for the still-Ply extraMetadata helper - 10 134 / 10 134 backend tests passing
- 10 top-level helpers flipped to Task<...>: unifyValueType,
unify, resolveType, checkFnParam, checkFnResult, and the five
DvalCreator helpers
- Ply.List.* → Task.* throughout
- added Task.foldSequentiallyWithIndex to Prelude/Task.fs
- 16 |> Ply.toTask bridges added for callees still on Ply
(Types.find, TypeReference.{unwrapAlias, toVT})
- task builder doesn't allow early `return raiseRTE` in if-no-else
the way uply does; dropped `return` keyword (raiseRTE throws
inline, semantically identical)
- one test (Serialization.DarkTypes.Tests.fs) drops a Ply.toTask
- 10 134 / 10 134 backend tests passing
- executeInner / execute return Task<Dval> - inner uply blocks -> task; Ply <expr> -> Task.FromResult <expr> - Ply.List.* -> Task.* - 6 |> Ply.toTask bridges for still-Ply callees (fns.package, fns.isHarmful, values.package, TypeReference.toVT) - 5 sites: drop `return` on raiseRTE in task to avoid early-return unit/generic mismatch (semantically identical -- raise throws) - RTQueryCompiler.fs Interpreter.execute caller drops Ply.toTask - Interpreter.Tests.fs drops 2 Ply.toTask bridges - 10 134 / 10 134 backend tests passing
- Runtime/RuntimeTypes.fs ExceptionReporter, Notifier, consoleReporter, consoleNotifier - LibCloudExecution/CloudExecution.fs extraMetadata/notify/sendException (extraMetadata's prior Ply.ofTask bridges collapse) - Cli/Cli.fs, LibPackageManager/Seed.fs, LocalExec/BenchmarkScenarios.fs in-test notify/sendException - tests/TestUtils/TestUtils.fs exceptionReporter + notifier - 10 134 / 10 134 backend tests passing
- RT.PackageManager record callbacks (getType/Value/Fn, getBlob,
persistBlob, isHarmful, init) now Task
- Types/Values/Fns/Blobs ExecutionState helper records: callbacks
Task
- Types.find and TypeReference.{unwrapAlias, toVT} flipped to Task
- Task.NEList.mapSequentially added to Prelude/Task.fs
- 16+ |> Ply.toTask bridges dropped at now-Task call sites
- LibPackageManager.PackageManager.rt bridges PMRT.Type/Fn/Value.get
(still Ply) with |> Ply.toTask at field assignments
- PT2RT.PackageManager.toRT bridges Ply→Task for the three lookups
- BuiltinExecution.Libs.Stream resolveElemVT/KT flipped to Task
- BuiltinExecution.Libs.Json's convert (still Ply) wraps now-Task
TypeReference.toVT and Types.find with |> Ply.ofTask
- Blob.promote's insert parameter flipped to Task<unit>; tests use a
pmInsertTask adapter for the still-Ply PMBlob.insert
- 10 134 / 10 134 backend tests passing
- 11 PT.PackageManager callbacks flipped: findType/Value/Fn, search,
getType/Value/Fn, getTypeLocations/getValueLocations/getFnLocations,
init
- PackageManager.empty + withExtras converted
- LibPackageManager.PackageManager: rt's PMRT bridges + pt's PMPT
bridges per field (still-Ply backends), createInMemory and combine
use Task.FromResult / task { } for the merge paths
- combine's search-merge needs explicit SearchResults annotation on
each let! to disambiguate record fields through the Task path
- PT2RT.PackageManager.toRT collapses its old Ply.toTask bridges
- LibParser.NameResolver: 3 resolve helpers wrap pm.find* with
Ply.ofTask for the still-Ply resolveGenericName helpers
- LibPackageManager.DeferredResolver: 7 walker sites wrap pm.find*
with Ply.ofTask
- BuiltinPM.Libs.Packages drops 6 |> Ply.toTask bridges
- tests/TestValues.fs flips Ply -> Task.FromResult on PM-overlay
field constructors
- 10 134 / 10 134 backend tests passing
- LibPackageManager/RuntimeTypes.fs: Type/Value/Fn.get,
findByValueType, Blob.get/insert/sweepOrphans
- LibPackageManager/ProgramTypes.fs: findItem/getItem/getItemLocations
and the Type/Value/Fn/search public wrappers
- Caching.withCache flipped Ply -> Task to match both backends
- LibPackageManager.PackageManager.{rt,pt} drop their
|> Ply.toTask bridges around withCache results
- Propagation.fs ItemProcessingContext<'T> callbacks (getItem,
getLocations) typed Task
- BuiltinPM.Libs.Packages drops 5 Ply.toTask bridges
- tests/Tests/Blob.Tests.fs drops 9 Ply.toTask bridges
- 10 134 / 10 134 backend tests passing
- DeferredResolver.fs: 20 uply -> task; reResolveNameResolution + reResolveTypeName/Fn/Value + AST walkers (TypeRef/StringSegment/ MatchCase/PipeExpr/Expr) flipped to Task<...>; Ply leaves collapsed to Task.FromResult; the temporary Ply.ofTask bridges earlier T.8 inserted around pm.find* are gone - Stats.fs::get and PackageRefsGenerator.fs::generate flipped - WipRefresh.fs: 3 |> Ply.toTask bridges around DR.reResolve* sites collapsed - BuiltinPM/Libs/Packages.fs::pmStats: explicit `(stats : LibPackageManager.Stats.Stats)` annotation to disambiguate stats.types from RT.Types.types - LibPackageManager/ is now Ply-free outside of the Ply package - 10 134 / 10 134 backend tests passing
…are/tests/LocalExec/Cli to Task
Single-pass bulk substitution across the remaining tree:
- Ply<...> -> Task<...>, uply { } -> task { }, Ply.List.foo ->
Task.foo, Ply.NEList.foo -> Task.NEList.foo, Ply.map/bind ->
Task.map/bind, Ply(x) -> Task.FromResult(x), |> Ply ->
|> Task.FromResult; stripped |> Ply.toTask and |> Ply.ofTask
bridges that collapsed to identity
- Auto-added `open System.Threading.Tasks` to every file that
referenced Task and lacked the import
Surgical fixes after the bulk pass:
- Record-disambiguation annotations in
LibParser/WrittenTypesToProgramTypes.fs (8 sites — F#'s task
builder pins TOverall outermost-in, so record literals shared
by multiple types resolve via closest-by-name; uply was
more permissive)
- let rec extractPath extracted out of a task { } block (FS3511)
- Canvas.fs (CliHost): `Ply "unknown"` -> `Task.FromResult "unknown"`
- TestUtils.testManyPly drops `>> Ply.toTask` chain
Tree is now Ply-free except `Prelude/Ply.fs` itself + the one
`let uply = Ply.uply` re-export in Prelude.fs (both retired in T.14).
10 134 / 10 134 backend tests passing.
README + 30-track-valuetask.md updated to: (1) prescribe
bulk-pass methodology for future migrations of this shape, and
(2) re-anchor V to fork from the tip of T (cleaner V diff,
B/T/V three-way comparison still preserved in comparison.md).
- rm backend/src/Prelude/Ply.fs
- backend/paket.dependencies: drop `nuget Ply = 0.3.1`
- 10 paket.references files: drop `Ply` line
(Prelude/Runtime/DarkTypes/Language/LibExecution/
BuiltinCli/CliHost/Execution/PM/HttpServer/Cli)
- paket install: removes Ply (0.3.1) +
System.Threading.Tasks.Extensions (4.6.3 — transitive)
- ~25 files: strip `open FSharp.Control.Tasks` and
`open FSharp.Control.Tasks.Affine.Unsafe` (Ply-package
namespaces; F# 10's task { } is in framework)
- backend/src/Prelude/Prelude.fs: drop the `type Ply<'a> = …`
/ `let uply = Ply.uply` re-exports
- 5 FS3511 (let rec inside task { } resumable code) fixes:
lift recursive helpers above the outer task block
- LibExecution/Execution.fs::groupConsecutiveWithCounts
- LibPackageManager/WipRefresh.fs::processOps
- LibPackageManager/Propagation.fs::discoverDependentsLoop
- tests/TestUtils/TestUtils.fs::executionStateFor
(let rec exceptionReporter outside the outer task)
- BuiltinExecution/Libs/HttpClient.fs SocketBasedHandler.handler:
vtask { } (provided by Ply pkg) -> task { … } wrapped in
ValueTask<Stream>(inner) for the ConnectCallback signature
- LibCloudExecution/CloudExecution.fs::extraMetadata: explicit
(result : Metadata) annotation (Metadata is a tuple-list alias
whose elements aren't all strings, so F# inferred wrong)
- Stale "stays uply" comments in 4 files deleted
- 30-track-valuetask.md: link to Darklang's "Optimizing F# tasks"
blog post as required prior reading for V
`grep -rn 'uply\|Ply\.Ply\|open Ply' backend/src` returns nothing.
10 134 / 10 134 backend tests passing.
Now that the Ply→Task migration removed the thread-affine-lock hazard (Monitor.Exit throwing on Ply continuation hops), the single-consumer invariant on `DStream` can be enforced cleanly. - Runtime/Stream.fs: Finalizer carries a permit-1 SemaphoreSlim on its lockObj - readNext / readChunk: Wait(0) at entry, raise "concurrent consumer on a single-consumer DStream" on contention, release in a `finally` so a raised callback can't strand the lock. Used Wait(0) (not WaitAsync) because the task body needs the result synchronously to decide whether to short-circuit - Finalizer.Finalize disposes the SemaphoreSlim to release the underlying handle - new test: concurrentReadNextRaises (Stream.Tests.fs) parks first readNext at a gated callback, asserts second raises, confirms first still completes after the gate 10 135 / 10 135 backend tests passing.
The release build's resumable-code analyzer (under PublishTrimmed) can't always statically reduce complex recursive task patterns. Specifically Runtime/TypeChecker.fs::unifyValueType — `match!` over `Types.find` (Task-returning) inside a recursive task. The compiler emits FS3511 and falls back to a dynamic-dispatch state machine, which is correct but slightly slower than the inlined version. `--warnaserror` was turning that into a hard build failure. The restructure to make the analyzer happy would split unifyValueType in invasive ways for the central type-checker entry point — not worth it. nowarn:3511 is the standard workaround. Documented inline. Affects Runtime/TypeChecker.fs and a couple of similarly-shaped recursive-task helpers. Verification chunks T.16/T.17/T.18 all green: - 10 135 / 10 135 backend tests passing - Release exe size B 76,416,031 -> T 76,797,646 (+381 KB / +0.5%) - Cold-start steady median (5 runs, same machine, same DB): B 0.484s -> T 0.397s (-85 ms / -18%) - Trim/AOT warnings: IL2 0->0, IL3 0->0, NU1510 60->38 (-22)
- Cli.fs top-level catch walks AggregateException/InnerException
recursively; prints type-name + message per layer, full stack
trace from the outermost throw at the bottom. The single
`e.Message` print collapsed every nested error to the AggregateException
surface, hiding what actually crashed.
- Db.fs executeRow{,Option}Async swap the literal "fail" exception
message for "SQL query failed in <fn>: <err.Message>". The
underlying exn was only ever in the structured metadata, which
the top-level catch dropped before stringifying.
Independently useful (any startup DB error now surfaces its cause);
load-bearing for the AOT spike, where the binary dies inside
cli.growIfNeeded with the literal string "fail" and zero hint at
which trim-broken Microsoft.Data.Sqlite path is the culprit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wanted to see some numbers, ran this in the background. incomplete, may abandon or may revisit later.