Examine performance under concurrency

[russellpierce]

Realistic scenarios frequently involve multiple analysts using the same db at the same time. Examine performance with 6+ queries going at once. Under Redshift's default options this will likely result in issues (http://docs.aws.amazon.com/redshift/latest/dg/cm-c-defining-query-queues.html) that shouldn't be experienced by BigQuery and experienced to a lesser extent on Snowflake.

[georgewfraser]

I expect Redshift's performance to be pretty linear under concurrency---that's what I've seen in other warehouses, and it makes sense given how MPP query planners work. But we should quantitatively evaluate the linearity of each warehouse and report it.

[russellpierce]

I think you'll be horrifyingly surprised when it comes to Redshift - maybe the dedicated storage resources per shard - it does seem particularly worse on dense storage.

[georgewfraser]

In addition to concurrent readers, it would be good to have concurrent writers, representing an ETL process that is appending, deleting, and updating at the same time that you are querying.