-
Notifications
You must be signed in to change notification settings - Fork 273
fix: add depth limit to EventSerializer to prevent hangs on complex objects #1577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
43114a6
ab2fa6f
b1b0189
75ad582
4395d4d
20e6dd5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -36,11 +36,24 @@ | |||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| class EventSerializer(JSONEncoder): | ||||||||||||||||||||||||||
| _MAX_DEPTH = 20 | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| def __init__(self, *args: Any, **kwargs: Any) -> None: | ||||||||||||||||||||||||||
| super().__init__(*args, **kwargs) | ||||||||||||||||||||||||||
| self.seen: set[int] = set() # Track seen objects to detect circular references | ||||||||||||||||||||||||||
| self._depth = 0 | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| def default(self, obj: Any) -> Any: | ||||||||||||||||||||||||||
| if self._depth >= self._MAX_DEPTH: | ||||||||||||||||||||||||||
| return f"<{type(obj).__name__}>" | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| self._depth += 1 | ||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||
| return self._default_inner(obj) | ||||||||||||||||||||||||||
| finally: | ||||||||||||||||||||||||||
| self._depth -= 1 | ||||||||||||||||||||||||||
|
Check failure on line 54 in langfuse/_utils/serializer.py
|
||||||||||||||||||||||||||
|
Comment on lines
+44
to
+54
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The Extended reasoning...What the bug isThe PR adds depth tracking by wrapping the existing serialization logic in a new if hasattr(obj, "__slots__"):
return self.default(
{slot: getattr(obj, slot, None) for slot in obj.__slots__}
)That Step-by-step proofConsider a chain
So Why existing code doesn't prevent itBefore this PR, the redundant Impact
How to fixOne-line change in the slots branch — call the inner helper directly so it matches the if hasattr(obj, "__slots__"):
return self._default_inner(
{slot: getattr(obj, slot, None) for slot in obj.__slots__}
)The PR author has already offered to make this change ("address Greptile |
||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| def _default_inner(self, obj: Any) -> Any: | ||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||
| if isinstance(obj, (datetime)): | ||||||||||||||||||||||||||
| # Timezone-awareness check | ||||||||||||||||||||||||||
|
|
@@ -167,6 +180,7 @@ | |||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| def encode(self, obj: Any) -> str: | ||||||||||||||||||||||||||
| self.seen.clear() # Clear seen objects before each encode call | ||||||||||||||||||||||||||
| self._depth = 0 | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||
| return super().encode(self.default(obj)) | ||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 The new depth guard in
default()(lines 44-48) returnsf"<{type(obj).__name__}>"for any object — including primitive strings — before any type analysis. Because the dict branch in_default_innerroutes both keys and values throughself.default()({self.default(k): self.default(v) for k, v in obj.items()}), at the truncation boundary a normal string key like'a'is replaced with the literal'<str>', silently corrupting user trace data. Fix: either move the depth check inside_default_innerafter the primitive type checks, or change the dict branch to{k: self.default(v) for k, v in obj.items()}(matching the__dict__branch convention on line 165).Extended reasoning...
What the bug is
The depth guard added at the top of
default()(lines 46-48) is unconditional — it runs before any type dispatch:This means at the boundary, any value passed to
default()— including plainstrandintprimitives — gets replaced by its type-name placeholder. Combined with the pre-existing dict branch in_default_inner(line 142-143):which routes both keys and values through
self.default(), the result is that dict keys at the truncation boundary are silently rewritten to"<str>"(or"<int>", etc.) instead of being preserved.Step-by-step proof
Consider a chain of 22 nested
{"a": {...}}dicts encoded byEventSerializer:encode(D_0)resets_depth=0, callsdefault(D_0).default(D_k)enters with_depth=k, checksk >= 20(false for k<20), increments tok+1, calls_default_inner(D_k).{self.default("a"): self.default(D_{k+1})}— both at_depth=k+1.k=19:_default_innercallsself.default("a")while_depth=20. The guard_depth >= _MAX_DEPTH(20 >= 20) fires first, returning"<str>"instead of dispatching to theisinstance(obj, str)branch which would have returned"a".Verified live output:
{"a": {"a": {"a": ... (19 levels) ... {"<str>": "<dict>"}}}}The last surviving key — the most informative one in the trace — is the one that gets clobbered.
Why existing tests miss it
test_max_depth_returns_type_nameuses aLevelclass with__dict__. That path goes through the__dict__branch at line 165:Note: keys are not routed through
self.default()here. So the test never exercises the corrupted-key path that the raw-dict branch creates.Impact
This is a regression introduced by this PR. Before the change,
default("a")always hit theisinstance(obj, str)branch in_default_innerand returned"a"unchanged. Now, for any traced input/output containing a deeply nested dict (a common shape: nested JSON payloads, message threads, agent state), the key name at the truncation boundary is silently replaced with"<str>". Worse,"<str>"is indistinguishable from a legitimate string a user might have placed there. The__slots__branch is also affected, because line 153-156 wraps slot names in a synthetic dict that gets routed throughdefault()— so slot names get the same treatment.How to fix
Two simple options:
_default_innerafter the primitive type checks. Primitives (str,int,float,None,bytes, etc.) are leaves and cannot recurse, so they should never be subject to the depth guard.__dict__branch convention: change line 142-143 to{k: self.default(v) for k, v in obj.items()}. Keys are already required to be JSON-encodable (strings/ints), so routing them throughdefault()was never necessary.