Read only "Lock wait timeout exceeded; try restarting transaction" (1 reply)

This is a plea for help! I'm seeing loads of "Lock wait timeout exceeded; try restarting transaction" exceptions when hitting the cluster with a large number of SELECTs. The question is WHY?!

I'm not inserting, updating or deleting, the isolation level is READ-COMMITTED and my queries are not explicitly locking. Additionally, non of the columns are BLOB or TEXT. Also the timeouts are set really long, so the sub-second query ought to be fine. I'm at a bit of a loss. Details as follows:

I have a small cluster with 2 data nodes, 1 mgt node and 2 MySQL nodes. The data nodes are multi-threaded, config files below (my.cnf are identical on the MySQL setup, config.ini is the mgt node setup).

I have a jdbc-based test harness, it sets up many readers to concurrently fire a non-trivial (but still sub-second) query. It joins a largish table (1.2M rows) to itself and onto another similar sized table (690K rows) and a couple of ref data tables.

The harness uses C3P0 to pool connections, setup thus:
cpds.setMinPoolSize(10);
cpds.setAcquireIncrement(5);
cpds.setAutoCommitOnClose(true);
cpds.setMaxPoolSize(200);

The connect string is:
jdbc:mysql:loadbalance://10.194.192.74:3306,10.194.192.75:3306/test?roundRobinLoadBalance=true

----config.ini----
[NDBD DEFAULT]
NoOfReplicas=2
LockPagesInMainMemory=1

DataMemory=5244M
IndexMemory=768M

ODirect=1

NoOfFragmentLogFiles=300
MaxNoOfConcurrentOperations=100000
TimeBetweenGlobalCheckpoints=1000
TimeBetweenEpochs=200
DiskCheckpointSpeed=10M
DiskCheckpointSpeedInRestart=100M
RedoBuffer=32M
# MaxNoOfLocalScans=64
MaxNoOfTables=1024
MaxNoOfOrderedIndexes=256

MaxNoOfConcurrentScans=500
MaxNoOfExecutionThreads=2
TransactionDeadlockDetectionTimeout=60000

[MYSQLD DEFAULT]

[NDB_MGMD DEFAULT]

[TCP DEFAULT]
# Managment Server
SendBufferMemory=8M
ReceiveBufferMemory=8M

[NDB_MGMD]
# the IP of THIS SERVER
HostName=10.194.192.71

[NDBD]
# the IP of the FIRST SERVER (Data Node)
HostName=10.194.192.76
DataDir= /var/lib/mysql-cluster

[NDBD]
# the IP of the SECOND SERVER (Data Node)
HostName=10.194.192.78
DataDir=/var/lib/mysql-cluster

[MYSQLD]...x24
-----end config.ini ----

----my.cnf----
[client]
port=3306
socket=/var/lib/mysql/mysql.sock

[mysqld]

ndbcluster
# IP address of the cluster management node
ndb-connectstring=10.194.192.71
default-storage-engine=NDBCLUSTER

ndb_cluster_connection_pool=10
max_connections=1000
transaction-isolation = READ-COMMITTED
#query_cache_size=16M
#thread_concurrency = 4

----end my.cnf----