Benchmarks

We have tested the performance of our implementation. The first test was to compute the MarketSimulator’s order processing throughput. Then we tested the time it takes for a trader to produce an order.

MarketSimulator order processing speed

This performance test implemented in the MarketSimulatorBenchmark.scala file calculates the time it takes for the Market simulator to process a certain amount of orders.

Orders

Two types of order sets are used in the benchmark. The orders from the finance.csv file which contain only limit and delete orders. Since this set didn’t contain any market orders and to add a bit of randomness, a random order generator was implemented. To generate these orders, I have calculated the occurrences of each type of order in the finance.csv file which gave me the following numbers:

Limit ask = 39%, Limit Bid = 26%, Delete = 35%

From each type of limit order, I have taken 10% and made them market orders. I also removed 10% of the delete orders and made 5% of them market ask orders and the other 5% market bid orders which gives the following distribution:

Limit ask = 35.1%, Limit Bid = 23.4%, Market Ask = 5.7%, Market Bid = 4.2%, Delete = 31.6%

Each order volume is a random value v = i ∗ 10(i = 1, ..., 10) and the limit order price is a Double between 90$ and 110$.

Test

First, orders are generated or loaded, depending on which set is used. An OrderFeeder component is used to send the orders to the BenchmarkMarketSimulator. It appends a LastOrder to the end of the list which will be used by the MarketSimulator to know it has processed all the orders. Before, starting to send the orders to the MarketSimulator, it sends the current time to a TimeCounter Component. When the MarketSimulator receives the LastOrder it sends the current time and the books size to the TimeCounter which computes the time and prints the data in the console.

Results

Here are the results when run on a Mac OSX 10.9,Intel i-7 Quad Core 2.3 GHz processor, 16 GB DDR3 and SSD Hard Drive. These numbers are the average of 5 runs.

Generated orders

OrdersCount	time (ms)
50k	317.2
100k	447
150k	583.4
200k	654.6
250k	700.4
500k	1552.2

finance.csv orders

OrdersCount	time (ms)
50k	326.6
100k	481.2
150k	566.8
200k	725.2
250k	797.2
500k	1358.8
1M	2277.6

Trader Reaction time

The goal of this test is to measure the time it takes for a trader’s order to be executed since the moment when the order that will trigger the trader’s action is sent to the MarketSimulator. The benchmark configuration consists of an OrderFeeder, a MarketSimulator, a Backloop (with a TransactionPersistor) and a BenchmarkTrader. The OrderFeeder generates two orders: a limit bid order and a market ask order. This will generate an order execution and the output transaction will travel through the BackLoop (who saves it to the Persistor) to the Trader who will send a new order to the MarketSimulator. The start counter signal is sent by the OrderFeeder to the TimeCounter immediately before the limit order is sent and the stop counter signal is sent by the MarketSimulator when it receives the trader’s order.

Results

setup	time (ms)
with BackLoop	2.8
with BackLoop (no persistor)	2.8
without BackLoop	4.2