made a rolling restart and this happened while restarting data nodes. no problem with management nodes, so there should not be any issue with config file
```
Status: Temporary error, restart node
Message: Insufficent nodes for system restart (Restart error)
Error: 2353
Error data: Unable to start missing node group! starting: 0000000000000000000000000000000000000400 (missing working fs for: 0000000000000000000000000000000000000000)
Error object: QMGR (Line: 2515) 0x00000002
Program: ndbmtd
Pid: 36197 thr: 0
Version: mysql-8.0.27 ndb-8.0.27
Trace file name: ndb_10_trace.log.10
Trace file path: /var/lib/mysql/data/ndb_10_trace.log.10 [t1..t4]
***EOM***
```
I tried also --nowait-nodes.
also tied to make nodegoups same time while starting nodes but nodes shits down before I am able to, also management gives same error
```
ndb_mgm> Node 11: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
Node 10: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.
Node 20: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
Node 21: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
```
```
ndb_mgm> CREATE NODEGROUP 10,11
* 5005: Error
* 5005-Send to process or receive failed.: Permanent error: Application error
```
I can not start as --initial I have data there that can not be removed
I need urgent help, how to start datanodes without loosing data.
thank you advanced
```
Status: Temporary error, restart node
Message: Insufficent nodes for system restart (Restart error)
Error: 2353
Error data: Unable to start missing node group! starting: 0000000000000000000000000000000000000400 (missing working fs for: 0000000000000000000000000000000000000000)
Error object: QMGR (Line: 2515) 0x00000002
Program: ndbmtd
Pid: 36197 thr: 0
Version: mysql-8.0.27 ndb-8.0.27
Trace file name: ndb_10_trace.log.10
Trace file path: /var/lib/mysql/data/ndb_10_trace.log.10 [t1..t4]
***EOM***
```
I tried also --nowait-nodes.
also tied to make nodegoups same time while starting nodes but nodes shits down before I am able to, also management gives same error
```
ndb_mgm> Node 11: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
Node 10: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2308: 'Another node failed during system restart, please investigate error(s) on other node(s)(Restart error). Temporary error, restart node'.
Node 20: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
Node 21: Forced node shutdown completed. Occurred during startphase 1. Caused by error 2353: 'Insufficent nodes for system restart(Restart error). Temporary error, restart node'.
```
```
ndb_mgm> CREATE NODEGROUP 10,11
* 5005: Error
* 5005-Send to process or receive failed.: Permanent error: Application error
```
I can not start as --initial I have data there that can not be removed
I need urgent help, how to start datanodes without loosing data.
thank you advanced