|
return [row.to_dict() for _, row in df.iterrows()] |
Current implementation:
data = [row.to_dict() for _, row in df.iterrows()]
Recommended replacement:
data = df.apply(lambda row: row.to_dict(), axis=1).tolist()
Using iterrows() introduces overhead because each row is returned as a Series object and to_dict() is repeatedly called in pure Python. This approach creates a large number of temporary objects and results in slow performance when the DataFrame becomes large.
By contrast, df.apply(lambda row: row.to_dict(), axis=1) keeps the row-wise transformation within Pandas' optimized Cython internals. Although still row-based, this method reduces Python-level overhead and improves performance while preserving the same output structure: List[Dict[str, Any]].
dataops-testgen/testgen/ui/views/data_catalog.py
Line 279 in 418ec3a
Current implementation:
data = [row.to_dict() for _, row in df.iterrows()]Recommended replacement:
data = df.apply(lambda row: row.to_dict(), axis=1).tolist()Using iterrows() introduces overhead because each row is returned as a Series object and to_dict() is repeatedly called in pure Python. This approach creates a large number of temporary objects and results in slow performance when the DataFrame becomes large.
By contrast, df.apply(lambda row: row.to_dict(), axis=1) keeps the row-wise transformation within Pandas' optimized Cython internals. Although still row-based, this method reduces Python-level overhead and improves performance while preserving the same output structure: List[Dict[str, Any]].