Switch to the rust component graph

[shsms]

• Also implements the new Formula/Formula3Phase classes to replace the old formula engine, for supporting the new formula functions - max/min/coalesce.
• Drops the old component graph and the old formula engine

[shsms]

Based on #1283

[Copilot]

Pull Request Overview

This PR switches from a Python-based component graph and formula engine to a new Rust-based implementation, introducing new Formula and Formula3Phase classes to support advanced formula functions (max/min/coalesce) and removing deprecated code.

Key Changes:

• Replaced old Python formula engine with new Rust-based component graph
• Implemented new Formula and Formula3Phase classes with support for max/min/coalesce functions
• Updated component data structures to use new client library types
• Removed deprecated formula engine tests and implementation

Reviewed Changes

Copilot reviewed 109 out of 118 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/timeseries/test_formula_engine.py	Entire file deleted - old formula engine tests removed
tests/timeseries/test_consumer.py	Added missing resampler calls for battery and PV inverter power
tests/timeseries/mock_resampler.py	Updated to use new Metric enum instead of ComponentMetricId
tests/timeseries/mock_microgrid.py	Updated component types and connection structures to match new API
tests/timeseries/_formulas/test_lexer.py	New test file for formula lexer functionality
tests/timeseries/_formulas/test_formulas_3_phase.py	New test file for three-phase formula support
tests/timeseries/_formulas/test_formulas.py	New comprehensive formula tests with max/min/coalesce support
tests/timeseries/_formulas/test_formula_composition.py	Updated to use new formula API and async context managers
src/frequenz/sdk/timeseries/formulas/_token.py	New token definitions for formula parsing
src/frequenz/sdk/timeseries/formulas/_resampled_stream_fetcher.py	New component for fetching resampled telemetry streams
src/frequenz/sdk/timeseries/formulas/_peekable.py	New peekable iterator implementation for parsing
src/frequenz/sdk/timeseries/formulas/_parser.py	New formula parser implementation
src/frequenz/sdk/timeseries/formulas/_lexer.py	New formula lexer implementation
src/frequenz/sdk/timeseries/formulas/_functions.py	New function implementations (max/min/coalesce)

Comments suppressed due to low confidence (1)

tests/timeseries/mock_microgrid.py:9

• The import Callable from collections.abc on line 10 is unused after the changes. The Callable type is imported again on line 12 from the typing module via the Coroutine import, and all usages in the file appear to use the typing module's Callable (e.g., line 260). This unused import should be removed.

from collections.abc import Callable

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[llucax]

Can you rebase it on top of Based on #1293 instead?

[shsms]

Can you rebase it on top of Based on #1293 instead?

#1293 is making changes to a lot of the files that are being deleted by this PR. I'd rather take this in first, then drop the changes on these files, then we only need to review required changes.

[shsms]

Oh, maybe those changes are from the api PR. don't know. Let's take the API PR first then, we can decide.

[shsms]

It was actually trivial to rebase on top of #1293. Done.

[shsms]

Damn, this causes my tests to fail. Switched back to based on #1283.

[llucax]

This needs a rebase now.

[shsms]

Rebased

[llucax]

FYI: I started reviewing 👀

[llucax]

So, yeah, 45 comments 😬

Of course most of it are minor and some are just about the same issue.

To summarize:

• Missing docstrings, I would add them to this PR, as it should be very low effort and important.

• __format__(), I would create an issue about it if you think it is a good idea, not necessary at all for this PR.

• TelemetryStream using float: Same, only discuss if it makes sense and if it does can be a separate issue.

• @dataclass with kw_only, I would do it for this PR, as it should also be low effort, but not very important so also fine to create an issue.

• Function.from_string() vs parse_function: Also curious to discuss, but OK to leave as it is.

• Peekable: I think support for None should be fixed in this PR, not sure if by using peekable from more-itertools or fix ours. I have mixed feelings, pulling a dependency for such a small thing seems overkill, but then this lib has a lot of other useful stuff, so maybe having it as a dependency maybe is not bad. Similar to asyncstdlib, at some point maybe it is good to have it in the toolbox by default.

• Docstring rendering fixes should probably be applied in this PR too

• with pytest.raises(StopIteration) also probably good to change in this PR, as it is low effort

• Adding error case to lexer, and having more tests in general (is it me or the new stuff is a bit undertested? Or is it tested indirectly?) could be done as separate issue/PR.

• Rename components -> stream can be done in this PR if you agree, but optional

• About using TaskGroup, I would push for it. According to Python docs of gather():

  Note

  A new alternative to create and run tasks concurrently and wait for their completion is asyncio.TaskGroup. TaskGroup provides stronger safety guarantees than gather for scheduling a nesting of subtasks: if a task (or a subtask, a task scheduled by a task) raises an exception, TaskGroup will, while gather will not, cancel the remaining scheduled tasks).

• Overriding cancel() + await() instead of stop() is important, but if things work for now can also be done in a separate issue/PR.

• Use of pytest.mark.parametrize also of course optional, but I would do it (even if in a separate PR/issue), AI should do it well in a few seconds and error reporting will be better when things fail.

• Formala3Phase, also fine to leave as it is in this PR, but I think we need to take the opportunity in this refactoring to get it right ASAP.

• NON_EXISTING_COMPONENT_ID same, I would love to see it go, probably not in this PR, but I would like to use this refactoring impulse to get rid of it ASAP.

• Making somehow formulas more generic to take a metrici + quantity: definitely not for this PR, but I would like to discuss

• Move _graph_traversal, probably for this PR

• Having a ComponentGraph that inherits (and specializes) from the new rust component graph: I would definitely do it. I know many advantages are now pointless because you updated the code already, but still, I think it makes things much easier than this long type hints and the mix of methods and free functions we have in this PR.

• I would also remove the is_xxx() wrappers over isinstance() in this PR unless there is a very good reason to have them.

• Remove types-networkx in this PR.

• Why 2 inverters now? would be nice to know and document better

• And also try to keep the deprecated aliases FormulaEngine and FormulaEngine3Phase, if not in this PR add them later, but before the release so hopefully we can make it a non-breaking change.

There are a few other small comments and questions that are completely optional.

[llucax]

If FormulaEngine and FormulaEngine3Phase were part of the public interface, and you say the interface didn't really changed (so the class were only renamed), can we keep a deprecated alias with the old names so we can make this a non-breaking change?

[shsms]

Any ideas how?

With newer python, we could do type FormulaEngine[W] = Formula[W].

[shsms]

Or I can just rename the thing back to FormulaEngine. Just thought it was weird because technically we can call everything an engine, resampling engine, etc.

[llucax]

I guess with TypeAlias, not sure if it works with generics too, but also not sure if deprecated works with it, so probably the easiest approach would be to use inheritance:

@deprecated("...")
class FormulaEngine(Formula[T]): pass

Then you should be able to pass a FormulaEngine to every place a Formula is expected and get the deprecation message both in the interpreter and type checkers.

[shsms]

We can actually rename the whole thing back to FormulaEngine. It was convenient to have two names during development, but FormulaEngine makes it more easily identifiable, different from the abstract concept of a formula.

[shsms]

This is ready again @llucax .

And so is #1322

[llucax]

I think the backwards compatibility thing is really important, but we can do anytime before the release.