Sysbench OLTP performance test

I have tried sysbench OLTP test on CockRoachDB v1.1.2. Testing result is as below, is there any thing wrong with my test?

OLTP benchmark test did not work out and report the error:

FATAL: PQexecPrepared() failed: 7 restart transaction: HandledRetryableTxnError: TransactionRetryError: retry txn (RETRY_POSSIBLE_REPLAY): “sql txn” id=cf018cb1 key=/Table/168/1/723495/0 rw=false pri=0.01925151 iso=SERIALIZABLE stat=PENDING epo=0 ts=1510888533.122007904,137 orig=1510888533.074163880,0 max=1510888533.074163880,0 wto=false rop=false seq=16
FATAL: `thread_run’ function failed: /usr/share/sysbench/oltp_common.lua:482: SQL error, errno = 0, state = ‘40001’: restart transaction: HandledRetryableTxnError: TransactionRetryError: retry txn (RETRY_POSSIBLE_REPLAY): “sql txn” id=cf018cb1 key=/Table/168/1/723495/0 rw=false pri=0.01925151 iso=SERIALIZABLE stat=PENDING epo=0 ts=1510888533.122007904,137 orig=1510888533.074163880,0 max=1510888533.074163880,0 wto=false rop=false seq=16

Select banchmark test has a pool performance as below:

table count table size sysbench threads qps latency(avg/.95)
32 1,000,000 32\*3 11125 8.88ms/26.68ms
32 1,000,000 64\*3 10041 19.47ms/69.29ms
32 1,000,000 128\*3 9902 39.15ms/132.49ms
32 1,000,000 256\*3 10170 76ms/257.95ms

testing environment

CockroachDB runs on three nodes, every node has the same configuration:

OS CentOS 7
CPU 64 vCPUs, Intel(R) Xeon(R) CPU E7-4820 v2 @ 2.00GHz
RAM 256G
DISK 275G SSD

start cockroachdb with --cache=25% --max-sql-memory=25%

testing script: https://github.com/yuhit/sysbench

Cluster Monitoring





I can’t be certain without looking at the code, but it looks like retryable errors aren’t being handled properly.

Cockroach transactions will need to be retried more frequently than a system like postgresql, making client-side handling of retries very important. You can find more details in our docs.

We also have examples in multiple languages, including transaction retries.

Thx.Why select test got pool performance?

Apparently, the not-yet-released sysbench 1.1 has a fix to support retries with CockroachDB. See https://github.com/akopytov/sysbench/commit/0169d42681ab699a05533a1e9294e98ab6494535.

I have done oltp testing with sysbench 1.1, but did not get a good result.

sysbench threads tps qps latency(avg / .95)
32*3 860.99 17226.47 112.37/200.47
64 * 3 678.15 13571.81 284.07/569.67
128 * 3 382.13 7655.22 1001.88/1771.29
256 * 3 235.71 4759.9 3249.03/14302.94

I noticed that raft log queue pending too much and cost more times when increasing sysbench threads, is there any thing wrong with that?

_status/vars is like below:
https://pastebin.com/rQND9sKG

@arjun do you mind providing some insight here?