-
Notifications
You must be signed in to change notification settings - Fork 1
Testing and Simulation
- Introduction
- Mock Datafeed Implementation
- Deterministic Bar Generation
- Historical Market Replay
- Unit Testing with pytest
- Edge Case Simulation
- Integration with TradingView Widget
- Performance Benchmarking
- Conclusion
This document provides comprehensive guidance on testing and simulating custom datafeeds during development. It covers the creation of mock datafeed implementations for offline testing, deterministic bar generation, historical market replay, and unit testing strategies. The BADatafeed class serves as a reference implementation for backtesting scenarios, demonstrating how to simulate various market conditions and edge cases.
The BADatafeed class provides a complete example of a mock datafeed implementation that inherits from TVDatafeed. This implementation demonstrates how to create a testable datafeed interface that can be used for offline development and testing without requiring real market data connections.
classDiagram
class TVDatafeed {
+_configuration TVDatafeedConfiguration
+_subscribers Dict[str, Any]
+_quote_subscribers Dict[str, Any]
+onReady(callback) void
+searchSymbols(userInput, exchange, symbolType, onResult) void
+resolveSymbol(symbolName, onResolve, onError, extension) void
+getBars(symbolInfo, resolution, periodParams, onResult, onError) void
+subscribeBars(symbolInfo, resolution, onTick, listenerGuid, onResetCacheNeededCallback) void
+unsubscribeBars(listenerGuid) void
+getQuotes(symbols, onDataCallback, onErrorCallback) void
+subscribeQuotes(symbols, fastSymbols, onRealtimeCallback, listenerGUID) void
+unsubscribeQuotes(listenerGUID) void
}
class BADatafeed {
+_configuration TVDatafeedConfiguration
+onReady(callback) void
+searchSymbols(userInput, exchange, symbolType, onResult) void
+resolveSymbol(symbolName, onResolve, onError, extension) void
+getBars(symbolInfo, resolution, periodParams, onResult, onError) void
+subscribeBars(symbolInfo, resolution, onTick, listenerGuid, onResetCacheNeededCallback) void
}
BADatafeed --|> TVDatafeed : inherits
The getBars method in BADatafeed demonstrates how to generate deterministic bar data for testing purposes. By using the periodParams.from_ timestamp as a seed, the same sequence of bars can be reproduced across multiple test runs, ensuring consistent test results.
flowchart TD
Start([getBars Entry]) --> ValidateInput["Validate Input Parameters"]
ValidateInput --> InputValid{"Input Valid?"}
InputValid --> |No| ReturnError["Return Error Response"]
InputValid --> |Yes| GenerateBars["Generate Deterministic Bars"]
GenerateBars --> SetTime["Set time = periodParams.from_ + i * interval"]
SetTime --> SetOHLC["Set OHLC values with deterministic formula"]
SetOHLC --> AddBar["Add bar to results"]
AddBar --> MoreBars{"More bars needed?"}
MoreBars --> |Yes| SetTime
MoreBars --> |No| CreateMetadata["Create TVHistoryMetadata"]
CreateMetadata --> ReturnResult["Call onResult with bars and metadata"]
ReturnError --> End([Function Exit])
ReturnResult --> End
The BADatafeed implementation supports historical market replay by allowing developers to pre-define sequences of bars that represent specific market conditions. This enables testing of trading strategies against known market scenarios, such as bull markets, bear markets, or volatile periods.
sequenceDiagram
participant Client as "Test Client"
participant Datafeed as "BADatafeed"
participant Storage as "Mock Data Storage"
Client->>Datafeed : getBars(symbolInfo, resolution, periodParams)
Datafeed->>Datafeed : Validate parameters
Datafeed->>Storage : Retrieve historical bars
Storage-->>Datafeed : Return bar sequence
Datafeed->>Datafeed : Apply deterministic transformations
Datafeed->>Client : onResult(bars, metadata)
loop Real-time simulation
Datafeed->>Datafeed : Wait for next bar interval
Datafeed->>Storage : Retrieve next bar
Storage-->>Datafeed : Return next bar
Datafeed->>Client : onTick(new_bar)
end
Unit tests for the datafeed implementation should cover symbol resolution, bar formatting, and real-time update logic. The use of async mocks allows for testing asynchronous callbacks without requiring actual network connections.
classDiagram
class TestBADatafeed {
+test_onReady_config_returns_configuration() void
+test_searchSymbols_returns_matching_symbols() void
+test_resolveSymbol_returns_valid_symbol_info() void
+test_getBars_returns_deterministic_bars() void
+test_subscribeBars_triggers_realtime_updates() void
+test_error_handling_in_resolveSymbol() void
}
class MockCallback {
+called bool
+result Any
+__call__(data) void
}
class AsyncMock {
+await_count int
+side_effect Exception
+reset_mock() void
}
TestBADatafeed --> MockCallback : uses
TestBADatafeed --> AsyncMock : uses
The testing framework should simulate various edge cases to ensure robustness, including delayed messages, duplicate bars, and sudden disconnections. These scenarios can be implemented by modifying the behavior of the subscribeBars method in the mock datafeed.
flowchart TD
Start([subscribeBars Entry]) --> RegisterSubscriber["Register subscriber in _subscribers"]
RegisterSubscriber --> StartSimulation["Start real-time simulation"]
subgraph Edge Cases
StartSimulation --> DelayedMessages["Simulate delayed messages with random delays"]
DelayedMessages --> DuplicateBars["Generate duplicate bars at random intervals"]
DuplicateBars --> Disconnections["Simulate sudden disconnections"]
Disconnections --> Reconnections["Test reconnection logic"]
end
Reconnections --> MonitorSubscribers["Monitor active subscribers"]
MonitorSubscribers --> SubscriberActive{"Subscriber still active?"}
SubscriberActive --> |No| Cleanup["Remove from _subscribers"]
SubscriberActive --> |Yes| SendNextBar["Send next bar via onTick"]
SendNextBar --> WaitInterval["Wait for next interval"]
WaitInterval --> StartSimulation
Cleanup --> End([Function Exit])
The example.py file demonstrates how to integrate the mock datafeed with the TradingView widget for visualization. This allows developers to visually verify the behavior of their datafeed implementation and test trading strategies in a realistic environment.
sequenceDiagram
participant Engine as "TVEngine"
participant Widget as "TradingView Widget"
participant Datafeed as "BADatafeed"
Engine->>Engine : Initialize TVEngine
Engine->>Datafeed : Create BADatafeed instance
Engine->>Widget : Configure widget with datafeed
Widget->>Datafeed : onReady(callback)
Datafeed->>Widget : callback(configuration)
Widget->>Datafeed : searchSymbols("BTC", "", "", onResult)
Datafeed->>Widget : onResult(results)
Widget->>Datafeed : resolveSymbol("BTCUSDT", onResolve, onError)
Datafeed->>Widget : onResolve(symbol_info)
Widget->>Datafeed : getBars(symbolInfo, "1", periodParams, onResult, onError)
Datafeed->>Widget : onResult(bars, metadata)
Widget->>Datafeed : subscribeBars(symbolInfo, "1", onTick, listenerGuid, onReset)
Datafeed->>Widget : onTick(new_bar) periodically
Performance benchmarking should focus on memory usage and response times for key operations such as symbol resolution, bar retrieval, and real-time updates. The mock implementation allows for controlled testing of performance characteristics under various load conditions.
flowchart TD
Start([Performance Test]) --> SetupEnvironment["Setup test environment"]
SetupEnvironment --> MeasureMemory["Measure baseline memory usage"]
subgraph Benchmark Tests
MeasureMemory --> TestSymbolResolution["Test resolveSymbol performance"]
TestSymbolResolution --> TestBarRetrieval["Test getBars performance"]
TestBarRetrieval --> TestRealtimeUpdates["Test subscribeBars performance"]
TestRealtimeUpdates --> TestHighFrequency["Test high-frequency update handling"]
end
TestHighFrequency --> MeasureResults["Measure memory and CPU usage"]
MeasureResults --> CompareResults["Compare with performance targets"]
CompareResults --> Optimize["Identify optimization opportunities"]
Optimize --> Report["Generate performance report"]
Report --> End([Test Complete])
The BADatafeed implementation provides a comprehensive framework for testing and simulating custom datafeeds during development. By leveraging mock implementations, deterministic bar generation, and comprehensive unit testing, developers can ensure their datafeed logic is robust and reliable before deploying to production. The integration with the TradingView widget enables visual verification of behavior, while performance benchmarking ensures production readiness. This testing approach significantly reduces development time and increases confidence in the reliability of trading strategies.