Hi,
I developed an application server that uses the C++ ndb API. I did a test with a transaction that inserts data into two tables. The records are a few tens of bytes, everything works fine except that if I do 1000 times the same transaction in a single thread, I have very heterogeneous latencies of 3ms to 732ms, the average is 27ms per transaction. Last year on a previous version of MySQL Cluster I had done a similar test and I did not notice such differences. What do you think ? Is it normal behavior or not? I note that at the two data-nodes, very few resources are used (CPU, memory, disk and network). how do i know if these latencies come from the ndb api or cluster side?
Best regards
Bruno
MySQL Cluster 7.6.8 config.ini file :
[NDB_MGMD DEFAULT]
ArbitrationRank=1
HeartbeatIntervalMgmdMgmd=1500
Portnumber=1186
[NDB_MGMD]
NodeId=49
HostName=192.168.1.70
DataDir=/home/mysql/cluster/49/
Portnumber=1186
[NDB_MGMD]
NodeId=56
HostName=192.168.1.71
DataDir=/home/mysql/cluster/56/
Portnumber=1186
[TCP DEFAULT]
SendBufferMemory=8M
ReceiveBufferMemory=8M
[NDBD DEFAULT]
NoOfReplicas=2
LateAlloc=1
StartNoNodeGroupTimeout=0
MaxBufferedEpochs=100
MaxBufferedEpochBytes=26214400
MaxDiskWriteSpeed=40M
MinDiskWriteSpeed=10M
MaxDiskWriteSpeedOtherNodeRestart=100M
MaxDiskWriteSpeedOwnRestart=200M
BackupMaxWriteSize=1M
BackupWriteSize=256K
BackupDataBufferSize=1M
BackupLogBufferSize=16M
BackupReportFrequency=10
BackupDiskWriteSpeedPct=50
CompressedBackup=true
CompressedLCP=false
MemReportFrequency=0
LogLevelStartup=1
LogLevelShutdown=0
LogLevelStatistic=0
LogLevelCheckpoint=0
LogLevelNodeRestart=0
LogLevelConnection=0
LogLevelError=0
LogLevelCongestion=0
LogLevelInfo=0
EventLogBufferSize=8K
StartupStatusReportFrequency=0
DataMemory=2G
MinFreePct=5
MaxNoOfAttributes=1000
MaxNoOfTables=128
MaxNoOfOrderedIndexes=128
MaxNoOfTriggers=768
MaxNoOfSubscriptions=0
MaxNoOfSubscribers=0
MaxNoOfConcurrentSubOperations=256
StringMemory=25
MaxAllocate=32M
MaxNoOfSavedMessages=25
LcpScanProgressTimeout=60
Diskless=false
LockPagesInMainMemory=0
Arbitration=Default
DiskPageBufferEntries=10
DiskPageBufferMemory=128M
SharedGlobalMemory=384M
DiskIOThreadPool=2
DiskSyncSize=4M
LongMessageBuffer=64M
MaxParallelCopyInstances=0
MaxNoOfConcurrentTransactions=4096
MaxNoOfConcurrentOperations=131072
MaxDMLOperationsPerTransaction=4294967295
MaxNoOfConcurrentScans=256
MaxParallelScansPerFragment=256
BatchSizePerLocalScan=256
FragmentLogFileSize=256M
NoOfFragmentLogFiles=15
InitFragmentLogFiles=SPARSE
RedoBuffer=64M
RedoOverCommitCounter=3
RedoOverCommitLimit=20
TwoPassInitialNodeRestartCopy=false
BuildIndexThreads=0
MaxNoOfExecutionThreads=4
RealtimeScheduler=false
SchedulerSpinTimer=0
TimeBetweenLocalCheckpoints=20
TimeBetweenGlobalCheckpoints=2000
TimeBetweenGlobalCheckpointsTimeout=120000
TimeBetweenEpochs=100
TimeBetweenEpochsTimeout=32000
TimeBetweenWatchdogCheck=6000
TimeBetweenWatchdogCheckInitial=60000
TransactionInactiveTimeout=4294967039
TransactionDeadlockDetectionTimeout=1200
TimeBetweenInactiveTransactionAbortCheck=1000
HeartbeatIntervalDbDb=1500
HeartbeatIntervalDbApi=60000
ConnectCheckIntervalDelay=0
StartPartialTimeout=30000
StartPartitionedTimeout=4294967039
StartFailureTimeout=0
ArbitrationTimeout=7500
RestartSubscriberConnectTimeout=12000
StopOnError=0
MaxStartFailRetries=3
[NDBD]
NodeId=1
HostName=192.168.1.70
DataDir=/home/mysql/cluster/1/
[NDBD]
NodeId=2
HostName=192.168.1.71
DataDir=/home/mysql/cluster/2/
[MYSQLD DEFAULT]
AutoReconnect=true
BatchByteSize=16K
BatchSize=256
MaxScanBatchSize=256K
[MYSQLD]
NodeId=52
HostName=192.168.1.70
[MYSQLD]
NodeId=55
HostName=192.168.1.71
[API]
NodeId=50
[API]
NodeId=51
[API]
NodeId=53
[API]
NodeId=54
I developed an application server that uses the C++ ndb API. I did a test with a transaction that inserts data into two tables. The records are a few tens of bytes, everything works fine except that if I do 1000 times the same transaction in a single thread, I have very heterogeneous latencies of 3ms to 732ms, the average is 27ms per transaction. Last year on a previous version of MySQL Cluster I had done a similar test and I did not notice such differences. What do you think ? Is it normal behavior or not? I note that at the two data-nodes, very few resources are used (CPU, memory, disk and network). how do i know if these latencies come from the ndb api or cluster side?
Best regards
Bruno
MySQL Cluster 7.6.8 config.ini file :
[NDB_MGMD DEFAULT]
ArbitrationRank=1
HeartbeatIntervalMgmdMgmd=1500
Portnumber=1186
[NDB_MGMD]
NodeId=49
HostName=192.168.1.70
DataDir=/home/mysql/cluster/49/
Portnumber=1186
[NDB_MGMD]
NodeId=56
HostName=192.168.1.71
DataDir=/home/mysql/cluster/56/
Portnumber=1186
[TCP DEFAULT]
SendBufferMemory=8M
ReceiveBufferMemory=8M
[NDBD DEFAULT]
NoOfReplicas=2
LateAlloc=1
StartNoNodeGroupTimeout=0
MaxBufferedEpochs=100
MaxBufferedEpochBytes=26214400
MaxDiskWriteSpeed=40M
MinDiskWriteSpeed=10M
MaxDiskWriteSpeedOtherNodeRestart=100M
MaxDiskWriteSpeedOwnRestart=200M
BackupMaxWriteSize=1M
BackupWriteSize=256K
BackupDataBufferSize=1M
BackupLogBufferSize=16M
BackupReportFrequency=10
BackupDiskWriteSpeedPct=50
CompressedBackup=true
CompressedLCP=false
MemReportFrequency=0
LogLevelStartup=1
LogLevelShutdown=0
LogLevelStatistic=0
LogLevelCheckpoint=0
LogLevelNodeRestart=0
LogLevelConnection=0
LogLevelError=0
LogLevelCongestion=0
LogLevelInfo=0
EventLogBufferSize=8K
StartupStatusReportFrequency=0
DataMemory=2G
MinFreePct=5
MaxNoOfAttributes=1000
MaxNoOfTables=128
MaxNoOfOrderedIndexes=128
MaxNoOfTriggers=768
MaxNoOfSubscriptions=0
MaxNoOfSubscribers=0
MaxNoOfConcurrentSubOperations=256
StringMemory=25
MaxAllocate=32M
MaxNoOfSavedMessages=25
LcpScanProgressTimeout=60
Diskless=false
LockPagesInMainMemory=0
Arbitration=Default
DiskPageBufferEntries=10
DiskPageBufferMemory=128M
SharedGlobalMemory=384M
DiskIOThreadPool=2
DiskSyncSize=4M
LongMessageBuffer=64M
MaxParallelCopyInstances=0
MaxNoOfConcurrentTransactions=4096
MaxNoOfConcurrentOperations=131072
MaxDMLOperationsPerTransaction=4294967295
MaxNoOfConcurrentScans=256
MaxParallelScansPerFragment=256
BatchSizePerLocalScan=256
FragmentLogFileSize=256M
NoOfFragmentLogFiles=15
InitFragmentLogFiles=SPARSE
RedoBuffer=64M
RedoOverCommitCounter=3
RedoOverCommitLimit=20
TwoPassInitialNodeRestartCopy=false
BuildIndexThreads=0
MaxNoOfExecutionThreads=4
RealtimeScheduler=false
SchedulerSpinTimer=0
TimeBetweenLocalCheckpoints=20
TimeBetweenGlobalCheckpoints=2000
TimeBetweenGlobalCheckpointsTimeout=120000
TimeBetweenEpochs=100
TimeBetweenEpochsTimeout=32000
TimeBetweenWatchdogCheck=6000
TimeBetweenWatchdogCheckInitial=60000
TransactionInactiveTimeout=4294967039
TransactionDeadlockDetectionTimeout=1200
TimeBetweenInactiveTransactionAbortCheck=1000
HeartbeatIntervalDbDb=1500
HeartbeatIntervalDbApi=60000
ConnectCheckIntervalDelay=0
StartPartialTimeout=30000
StartPartitionedTimeout=4294967039
StartFailureTimeout=0
ArbitrationTimeout=7500
RestartSubscriberConnectTimeout=12000
StopOnError=0
MaxStartFailRetries=3
[NDBD]
NodeId=1
HostName=192.168.1.70
DataDir=/home/mysql/cluster/1/
[NDBD]
NodeId=2
HostName=192.168.1.71
DataDir=/home/mysql/cluster/2/
[MYSQLD DEFAULT]
AutoReconnect=true
BatchByteSize=16K
BatchSize=256
MaxScanBatchSize=256K
[MYSQLD]
NodeId=52
HostName=192.168.1.70
[MYSQLD]
NodeId=55
HostName=192.168.1.71
[API]
NodeId=50
[API]
NodeId=51
[API]
NodeId=53
[API]
NodeId=54