Independent tasks parallelism scaling experiment

[NHDaly]

This experiment is testing the basic scaling of julia's new multithreading with very-many CPUs.

Testing the performance speedup of extremely simple, totally independent tasks (multiply lots of numbers) as we add more cores. The tasks are all independent, and should contain no allocation, so there should be (ideally) zero contention.

This allows us to measure a baseline performance scaling result, which we can expect to be the maximum potential performance gains we could get in any real program by adding more cores.

This experiment lives here:
src/bench/simple_independent.jl

Tracking issue: #1

[NHDaly]

Results from benchmark:

Full output from 96 vCPU machine (r5d.24xlarge) at bae621b:
https://gist.github.com/rai-nhdaly/da9f522eea01ea320a83c8192236f374

Speedup plot per thread:

It looks like there is a clear, (almost) linear relationship up to the number of real cores, and then basically neutral to slightly slower as you add more hyperthreads.

Results (omitted printing latencies column):
9×6 DataFrames.DataFrame
│ Row │ nthreads │ time_secs │ allocs │ memory │ gctime_secs │ thread_counts                                                                                                                                                                                                                                                                                                                                                                                    │
│     │ Int64    │ Float64   │ Int64  │ Int64  │ Float64     │ Array{Int64,1}                                                                                                                                                                                                                                                                                                                                                                                   │
├─────┼──────────┼───────────┼────────┼────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1   │ 1        │ 292.127   │ 7507   │ 817296 │ 0.0         │ [1000]                                                                                                                                                                                                                                                                                                                                                                                           │
│ 2   │ 12       │ 26.5832   │ 8027   │ 825776 │ 0.0         │ [83, 84, 83, 83, 84, 83, 84, 83, 83, 83, 84, 83]                                                                                                                                                                                                                                                                                                                                                 │
│ 3   │ 24       │ 13.3046   │ 8034   │ 826080 │ 0.0         │ [42, 42, 41, 42, 41, 41, 41, 42, 42, 42, 42, 42, 41, 42, 42, 42, 42, 42, 41, 42, 42, 41, 41, 42]                                                                                                                                                                                                                                                                                                 │
│ 4   │ 36       │ 8.8861    │ 8029   │ 826192 │ 0.0         │ [27, 28, 28, 27, 28, 28, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 27, 27, 27, 27]                                                                                                                                                                                                                                                 │
│ 5   │ 48       │ 6.84518   │ 8046   │ 826720 │ 0.0         │ [20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 20, 21, 21, 21, 20, 21, 21, 20, 21, 21, 21, 21, 21, 20, 21, 21, 21, 21, 21, 21, 20, 21, 20, 21, 20, 21, 21, 21, 21, 21, 21, 21]                                                                                                                                                                                                 │
│ 6   │ 60       │ 6.84695   │ 8028   │ 826592 │ 0.0         │ [21, 21, 10, 21, 21, 10, 21, 21, 20, 21, 10, 10, 21, 21, 21, 21, 21, 21, 21, 21, 21, 10, 21, 21, 21, 21, 21, 10, 10, 10, 21, 10, 21, 21, 21, 11, 21, 10, 10, 10, 21, 11, 11, 21, 21, 20, 21, 21, 21, 10, 10, 21, 10, 11, 11, 10, 10, 10, 11, 21]                                                                                                                                                 │
│ 7   │ 72       │ 7.06662   │ 8029   │ 826800 │ 0.0         │ [10, 10, 21, 21, 10, 10, 21, 21, 10, 11, 11, 21, 10, 11, 10, 11, 21, 21, 21, 21, 10, 10, 21, 10, 10, 10, 10, 10, 21, 10, 11, 11, 21, 11, 11, 10, 21, 20, 21, 21, 10, 10, 21, 21, 10, 10, 11, 21, 21, 10, 10, 10, 11, 21, 10, 21, 21, 21, 10, 11, 10, 11, 10, 11, 11, 10, 10, 11, 10, 11, 10, 11]                                                                                                 │
│ 8   │ 84       │ 7.15322   │ 8030   │ 827104 │ 0.0         │ [10, 11, 11, 10, 20, 11, 11, 10, 11, 11, 11, 20, 21, 10, 11, 10, 11, 11, 11, 10, 10, 21, 10, 10, 11, 10, 10, 11, 21, 11, 10, 11, 10, 11, 11, 10, 10, 11, 10, 10, 11, 11, 20, 11, 21, 10, 10, 11, 10, 20, 10, 11, 10, 10, 21, 10, 10, 11, 10, 21, 11, 11, 11, 10, 20, 11, 10, 11, 10, 11, 10, 11, 10, 10, 10, 20, 11, 11, 10, 10, 10, 10, 11, 10]                                                 │
│ 9   │ 96       │ 7.10565   │ 8037   │ 827376 │ 0.0         │ [10, 10, 11, 11, 11, 11, 10, 11, 10, 11, 10, 11, 11, 10, 10, 11, 10, 11, 11, 10, 10, 11, 10, 11, 10, 11, 11, 11, 10, 10, 10, 10, 10, 10, 11, 11, 10, 10, 10, 10, 11, 11, 11, 10, 11, 10, 10, 11, 11, 11, 10, 11, 11, 10, 10, 11, 10, 10, 11, 11, 10, 10, 10, 11, 10, 10, 11, 10, 10, 10, 10, 10, 11, 10, 10, 10, 11, 10, 10, 10, 11, 10, 10, 10, 11, 11, 10, 10, 10, 10, 11, 10, 11, 11, 10, 10] │
Processed:
9×6 DataFrames.DataFrame
│ Row │ nthreads │ time_secs │ speedup_factor │ marginal_speedup │ utilization │ latency_quantiles_ms                                                  │
│     │ Int64    │ Float64   │ Float64        │ Float64          │ Float64     │ Tuple…                                                                │
├─────┼──────────┼───────────┼────────────────┼──────────────────┼─────────────┼───────────────────────────────────────────────────────────────────────┤
│ 1   │ 1        │ 292.127   │ 1.0            │ 1.0              │ 1.0         │ (0.1=>291.902, 0.5=>291.988, 0.9=>292.279, 0.99=>294.721, 1=>300.417) │
│ 2   │ 12       │ 26.5832   │ 10.9892        │ 0.908105         │ 0.915763    │ (0.1=>316.5, 0.5=>316.565, 0.9=>316.643, 0.99=>316.888, 1=>317.432)   │
│ 3   │ 24       │ 13.3046   │ 21.9569        │ 0.91398          │ 0.914871    │ (0.1=>316.616, 0.5=>316.673, 0.9=>316.824, 0.99=>317.537, 1=>329.503) │
│ 4   │ 36       │ 8.8861    │ 32.8746        │ 0.909809         │ 0.913184    │ (0.1=>316.832, 0.5=>316.941, 0.9=>317.503, 0.99=>319.746, 1=>325.41)  │
│ 5   │ 48       │ 6.84518   │ 42.6763        │ 0.81681          │ 0.88909     │ (0.1=>324.988, 0.5=>325.099, 0.9=>325.403, 0.99=>328.32, 1=>342.312)  │
│ 6   │ 60       │ 6.84695   │ 42.6653        │ -0.000919783     │ 0.711088    │ (0.1=>325.227, 0.5=>325.314, 0.9=>651.411, 0.99=>652.33, 1=>652.412)  │
│ 7   │ 72       │ 7.06662   │ 41.339         │ -0.110524        │ 0.574153    │ (0.1=>324.99, 0.5=>327.907, 0.9=>651.638, 0.99=>652.062, 1=>652.347)  │
│ 8   │ 84       │ 7.15322   │ 40.8386        │ -0.0417036       │ 0.486173    │ (0.1=>324.917, 0.5=>651.125, 0.9=>651.334, 0.99=>651.619, 1=>657.11)  │
│ 9   │ 96       │ 7.10565   │ 41.1119        │ 0.0227817        │ 0.428249    │ (0.1=>651.572, 0.5=>651.66, 0.9=>652.12, 0.99=>655.869, 1=>690.404)   │

For this run, the overhead was around 10%. ~8.5% for most of the graph, then 11% at maximum cores (48 threads). Adding more cores than physical cpus does not improve performance.

So for 10 cores, we would get ~9x speedup, and for 48 cores (the max on this machine), ~42.6x speedup.

I saw approximately the same results across several experiments, each taking around 30 min to run.

(The slight increase in overhead from 8% to 11% could be because the machine is slightly utilized for other "system" things on the machine (such as the OS, monitoring, etc), and as we approach the maximum number of cores, we start having to share more of our cpu with the OS / other processes...)

The other plots we generated are here: