Marine Institute Logo Marine Institute ERDDAP Server
Serving data from the Marine Institute (Ireland)
   
Brought to you by the Marine Institute(Ireland)    

ERDDAP > Status

Current time is 2020-06-06T03:18:06+00:00
Startup was at  2020-05-29T07:01:43+00:00
Last major LoadDatasets started 33.000 s ago and is still running.
nGridDatasets  = 9
nTableDatasets = 37
nTotalDatasets = 46
n Datasets Failed To Load (in the last major LoadDatasets) = 0
Response Failed    Time (since last major LoadDatasets) n =        2,  median ~=        1 ms
Response Failed    Time (since last Daily Report)       n =     1230,  median ~=        0 ms
Response Failed    Time (since startup)                 n =    11566,  median ~=        1 ms
Response Succeeded Time (since last major LoadDatasets) n =       60,  median ~=       20 ms
Response Succeeded Time (since last Daily Report)       n =    10891,  median ~=       30 ms
Response Succeeded Time (since startup)                 n =    98432,  median ~=       27 ms
TaskThread has finished 0 out of 0 tasks.  Currently, no task is running.
TaskThread Failed    Time (since last Daily Report)     n =        0
TaskThread Failed    Time (since startup)               n =        0
TaskThread Succeeded Time (since last Daily Report)     n =        0
TaskThread Succeeded Time (since startup)               n =        0
Number of threads: Tomcat-waiting=0, inotify=0, other=67
MemoryInUse=   187 MB (highWaterMark=  1704 MB) (Xmx ~= 5888 MB)

Major LoadDatasets Time Series: MLD    Datasets Loaded    Requests (medianTime in seconds)     Number of Threads      Memory (MB)
  timestamp                    time   nTry nFail nTotal  nSuccess (median) nFailed (median)  tomWait inotify other  inUse highWater
  2020-06-06T03:06:41+00:00    247s      8     0     46       112 (     0)       7 (     0)        0       0    67     94      1704
  2020-06-06T02:51:48+00:00    255s      7     0     46        81 (     0)       8 (     0)        0       0    67     95      1704
  2020-06-06T02:36:45+00:00    251s      6     0     46       111 (     0)      11 (     0)        0       0    67     95      1704
  2020-06-06T02:22:13+00:00    280s      8     0     46        80 (     0)       6 (     0)        0       0    67     94      1704
  2020-06-06T02:06:37+00:00    244s      6     0     46        95 (     0)       2 (     0)        0       0    67     95      1704
  2020-06-06T01:52:05+00:00    272s      9     0     46       100 (     0)      11 (     0)        0       0    67     94      1704
  2020-06-06T01:36:33+00:00    240s      6     0     46       112 (     0)       8 (     0)        0       0    67     94      1704
  2020-06-06T01:21:57+00:00    264s      7     0     46        99 (     0)       6 (     0)        0       0    67     94      1704
  2020-06-06T01:06:50+00:00    257s      6     0     46       112 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-06T00:52:23+00:00    291s      7     0     46        91 (     0)       8 (     0)        0       0    67     95      1704
  2020-06-06T00:36:37+00:00    244s      9     0     46       103 (     0)      11 (     0)        0       0    67     94      1704
  2020-06-06T00:21:40+00:00    248s      7     0     46        97 (     0)      15 (     0)        0       0    67     94      1704
  2020-06-06T00:06:35+00:00    242s      6     0     46        93 (     0)       3 (     0)        0       0    67     95      1704
  2020-06-05T23:52:21+00:00    289s      7     0     46       103 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-05T23:36:40+00:00    248s      6     0     46        82 (     0)       4 (     0)        0       0    67     95      1704
  2020-06-05T23:21:50+00:00    258s      9     0     46        83 (     0)       7 (     0)        0       0    67     94      1704
  2020-06-05T23:07:05+00:00    273s      6     0     46       114 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-05T22:51:50+00:00    258s      8     0     46        88 (     0)       5 (     0)        0       0    67     95      1704
  2020-06-05T22:37:09+00:00    277s      6     0     46       114 (     0)      11 (     0)        0       0    67     95      1704
  2020-06-05T22:21:43+00:00    251s      9     0     46       105 (     0)      13 (     0)        0       0    67     93      1704
  2020-06-05T22:06:37+00:00    245s      6     0     46       108 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-05T21:51:46+00:00    254s      8     0     46       124 (     0)       7 (     0)        0       0    67     95      1704
  2020-06-05T21:36:37+00:00    245s      6     0     46        92 (     0)       2 (     0)        0       0    67     94      1704
  2020-06-05T21:22:31+00:00    299s      7     0     46       114 (     0)      19 (     0)        0       0    67     95      1704
  2020-06-05T21:07:25+00:00    293s      9     0     46       122 (     0)      29 (     0)        0       0    69    182      1704
  2020-06-05T20:51:41+00:00    249s      7     0     46       114 (     0)      22 (     0)        0       0    67     95      1704
  2020-06-05T20:36:24+00:00    232s      6     0     46        95 (     0)      15 (     0)        0       0    67     94      1704
  2020-06-05T20:21:52+00:00    260s      8     0     46       114 (     0)      19 (     0)        0       0    67    174      1704
  2020-06-05T20:06:39+00:00    247s      8     0     46       132 (     0)       8 (     0)        0       0    67     95      1704
  2020-06-05T19:51:37+00:00    245s      7     0     46       125 (     0)      14 (     0)        0       0    67     95      1704
  2020-06-05T19:36:35+00:00    243s      6     0     46       108 (     0)       1 (     0)        0       0    71     94      1704
  2020-06-05T19:22:12+00:00    280s      7     0     46       133 (     0)      17 (     0)        0       0    67     93      1704
  2020-06-05T19:06:36+00:00    244s      8     0     46       159 (     0)      11 (     0)        0       0    67     94      1704
  2020-06-05T18:51:38+00:00    247s      7     0     46       123 (     0)      24 (     0)        0       0    67     94      1704
  2020-06-05T18:36:35+00:00    244s      6     0     46       129 (     0)      12 (     0)        0       0    67     95      1704
  2020-06-05T18:22:11+00:00    280s      7     0     46       172 (     0)      26 (     0)        0       0    67     94      1704
  2020-06-05T18:07:10+00:00    279s      6     0     46       148 (     0)      28 (     0)        0       0    67     94      1704
  2020-06-05T17:51:38+00:00    246s     10     0     46       163 (     0)      10 (     0)        0       0    67     93      1704
  2020-06-05T17:36:38+00:00    247s      6     0     46       133 (     0)      12 (     0)        0       0    67     96      1704
  2020-06-05T17:21:44+00:00    253s      7     0     46       125 (     0)      14 (     0)        0       0    67     97      1704
  2020-06-05T17:06:45+00:00    254s      6     0     46       117 (     0)       5 (     0)        0       0    67     95      1704
  2020-06-05T16:52:00+00:00    269s      8     0     46        76 (     0)       9 (     0)        0       0    67     95      1704
  2020-06-05T16:40:32+00:00    481s      9     0     46       193 (     0)      28 (     0)        0       0    67     95      1704
  2020-06-05T16:21:53+00:00    262s      8     0     46       171 (     0)      20 (     0)        0       0    67     95      1704
  2020-06-05T16:06:38+00:00    247s      6     0     46       125 (     0)      11 (     0)        0       0    67     94      1704
  2020-06-05T15:51:54+00:00    263s      7     0     46       538 (     0)      10 (     0)        0       0    67     94      1704
  2020-06-05T15:36:39+00:00    248s      6     0     46       140 (     0)      20 (     0)        0       0    76     94      1704
  2020-06-05T15:22:00+00:00    269s      9     0     46       172 (     0)      22 (     0)        0       0    67     95      1704
  2020-06-05T15:06:40+00:00    249s      6     0     46       135 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-05T14:52:19+00:00    288s      7     0     46       114 (     0)      24 (     0)        0       0    67     95      1704
  2020-06-05T14:36:27+00:00    237s      6     0     46       146 (     0)      15 (     0)        0       0    67     94      1704
  2020-06-05T14:22:01+00:00    271s      9     0     46       243 (     0)      32 (     0)        0       0    71     94      1704
  2020-06-05T14:05:06+00:00    155s      6     3     43       179 (     0)       8 (     0)        0       0    68     94      1704
  2020-06-05T13:52:05+00:00    275s      7     0     46       163 (     0)      18 (     0)        0       0    67     94      1704
  2020-06-05T13:37:20+00:00    290s      6     0     46       229 (     0)      26 (     0)        0       0    67     92      1704
  2020-06-05T13:21:48+00:00    258s      8     0     46       110 (     0)       6 (     0)        0       0    67     91      1704
  2020-06-05T13:07:04+00:00    274s      7     0     46       117 (     0)       8 (     0)        0       0    67    270      1704
  2020-06-05T12:51:38+00:00    248s      7     0     46       110 (     0)      21 (     0)        0       0    67     95      1704
  2020-06-05T12:36:26+00:00    236s      6     0     46       128 (     0)      14 (     0)        0       0    67     94      1704
  2020-06-05T12:21:42+00:00    252s      9     0     46       133 (     0)      14 (     0)        0       0    67     94      1704
  2020-06-05T12:06:56+00:00    266s      7     0     46       173 (     0)      47 (     0)        0       0    77    215      1704
  2020-06-05T11:51:43+00:00    253s      7     0     46       124 (     0)       6 (     0)        0       0    69     95      1704
  2020-06-05T11:37:17+00:00    287s      8     0     46       152 (     0)      30 (     0)        0       0    67     94      1704
  2020-06-05T11:22:15+00:00    285s      6     0     46        99 (     0)       5 (     0)        0       0    67     94      1704
  2020-06-05T11:07:30+00:00    300s      9     0     46       139 (     0)       8 (     0)        0       0    67     94      1704
  2020-06-05T10:51:43+00:00    253s      7     0     46        95 (     0)      16 (     0)        0       0    67     94      1704
  2020-06-05T10:36:47+00:00    258s      6     0     46       150 (     0)      33 (     0)        0       0    67     94      1704
  2020-06-05T10:21:36+00:00    246s      6     0     46       347 (     0)      65 (     0)        0       0    83     94      1704
  2020-06-05T10:06:39+00:00    250s     11     0     46       105 (     0)      46 (     0)        0       0    67     95      1704
  2020-06-05T09:51:58+00:00    269s      7     0     46       118 (     0)      16 (     0)        0       0    67     94      1704
  2020-06-05T09:36:57+00:00    268s      6     0     46       183 (     0)      21 (     0)        0       0    68     93      1704
  2020-06-05T09:22:09+00:00    280s      6     0     46       214 (     0)      63 (     0)        0       0    67    156      1704
  2020-06-05T09:06:43+00:00    255s      7     0     46       203 (     0)      22 (     0)        0       0    67     94      1704
  2020-06-05T08:52:10+00:00    282s     11     0     46       148 (     0)      14 (     0)        0       0    69     94      1704
  2020-06-05T08:36:36+00:00    247s      6     0     46       161 (     0)      19 (     0)        0       0    67     92      1704
  2020-06-05T08:22:01+00:00    272s      5     0     46       118 (     0)      13 (     0)        0       0    67     93      1704
  2020-06-05T08:08:11+00:00    343s     25     0     46       122 (     0)       8 (     0)        0       0    67     92      1704
  2020-06-05T07:51:55+00:00    266s      9     0     46        87 (     0)       6 (     0)        0       0    67     94      1704
  2020-06-05T07:36:42+00:00    254s      5     0     46       102 (     0)       3 (     0)        0       0    67     93      1704
  2020-06-05T07:23:45+00:00    377s      7     0     46       145 (     0)      19 (     0)        0       0    88     91      1704
  2020-06-05T07:06:49+00:00    261s      9     0     46       111 (     0)       6 (     0)        0       0    67     95      1704
  2020-06-05T06:51:37+00:00    249s      9     0     46       123 (     0)      18 (     0)        0       0    67     94      1704
  2020-06-05T06:37:06+00:00    278s      6     0     46       169 (     0)      26 (     0)        0       0    67     94      1704
  2020-06-05T06:21:46+00:00    258s      6     0     46       120 (     0)      10 (     0)        0       0    67     95      1704
  2020-06-05T06:06:29+00:00    241s      7     0     46       142 (     0)       4 (     0)        0       0    67    100      1704
  2020-06-05T05:51:49+00:00    261s      7     0     46       276 (     0)      14 (     0)        0       0    67     95      1704
  2020-06-05T05:37:12+00:00    284s      9     0     46        93 (     0)       2 (     0)        0       0    67     94      1704
  2020-06-05T05:21:31+00:00    243s      7     0     46        83 (     0)       2 (     0)        0       0    67     95      1704
  2020-06-05T05:06:41+00:00    252s      7     0     46       136 (     0)      12 (     0)        0       0    67     94      1704
  2020-06-05T04:51:56+00:00    268s      9     0     46        86 (     0)       5 (     0)        0       0    67     95      1704
  2020-06-05T04:36:44+00:00    256s      8     0     46       144 (     0)      21 (     0)        0       0    67     94      1704
  2020-06-05T04:21:51+00:00    263s      7     0     46       118 (     0)      14 (     0)        0       0    67     95      1704
  2020-06-05T04:06:31+00:00    243s      8     0     46       111 (     0)       2 (     0)        0       0    67     95      1704
  2020-06-05T03:51:44+00:00    256s      7     0     46       102 (     0)       6 (     0)        0       0    67     98      1704
  2020-06-05T03:36:37+00:00    249s      8     0     46        91 (     0)       2 (     0)        0       0    67     94      1704
  2020-06-05T03:21:45+00:00    257s      6     0     46        87 (     0)       3 (     0)        0       0    67     95      1704
  2020-06-05T03:06:56+00:00    268s      7     0     46       107 (     0)      17 (     0)        0       0    67     94      1704
  2020-06-05T02:52:05+00:00    277s      7     0     46        97 (     0)       8 (     0)        0       0    67     95      1704
  2020-06-05T02:36:29+00:00    242s      5     0     46        80 (     0)       7 (     0)        0       0    67     95      1704
  2020-06-05T02:23:09+00:00    342s      8     0     46       110 (     0)      10 (     0)        0       0    67     94      1704


Major LoadDatasets Times Distribution (since last Daily Report):
    n =       80,  median ~=   213506 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:          77
    <= 10 min:          3
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Major LoadDatasets Times Distribution (since startup):
    n =      753,  median ~=   215585 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:         709
    <= 10 min:         44
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


Minor LoadDatasets Times Distribution (since last Daily Report):
    n =        2,  median ~=    15000 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            2
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Minor LoadDatasets Times Distribution (since startup):
    n =        3,  median ~=    17500 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            2
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           1
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


Response Failed Time Distribution (since last major LoadDatasets):
    n =        2,  median ~=        1 ms
    0 ms:               0
    1 ms:               2
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Failed Time Distribution (since last Daily Report):
    n =     1230,  median ~=        0 ms
    0 ms:             442
    1 ms:             353
    2 ms:              13
    <= 5 ms:           90
    <= 10 ms:          36
    <= 20 ms:          12
    <= 50 ms:         141
    <= 100 ms:         30
    <= 200 ms:         40
    <= 500 ms:         39
    <= 1 s:            17
    <= 2 s:             2
    <= 5 s:             5
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           1
    <= 2 min:           9
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Failed Time Distribution (since startup):
    n =    11566,  median ~=        1 ms
    0 ms:            3743
    1 ms:            2883
    2 ms:             121
    <= 5 ms:          773
    <= 10 ms:         506
    <= 20 ms:         170
    <= 50 ms:        1632
    <= 100 ms:        245
    <= 200 ms:        343
    <= 500 ms:        495
    <= 1 s:           271
    <= 2 s:           166
    <= 5 s:            95
    <= 10 s:            0
    <= 20 s:            3
    <= 1 min:          37
    <= 2 min:          63
    <= 5 min:          20
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


Response Succeeded Time Distribution (since last major LoadDatasets):
    n =       60,  median ~=       20 ms
    0 ms:               2
    1 ms:               2
    2 ms:               3
    <= 5 ms:            1
    <= 10 ms:           6
    <= 20 ms:          16
    <= 50 ms:          19
    <= 100 ms:          2
    <= 200 ms:          2
    <= 500 ms:          2
    <= 1 s:             0
    <= 2 s:             3
    <= 5 s:             0
    <= 10 s:            2
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Succeeded Time Distribution (since last Daily Report):
    n =    10891,  median ~=       30 ms
    0 ms:             678
    1 ms:             352
    2 ms:             346
    <= 5 ms:          288
    <= 10 ms:         497
    <= 20 ms:        2519
    <= 50 ms:        2188
    <= 100 ms:        660
    <= 200 ms:       1073
    <= 500 ms:        856
    <= 1 s:           417
    <= 2 s:           514
    <= 5 s:           267
    <= 10 s:          229
    <= 20 s:            5
    <= 1 min:           1
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          1
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Succeeded Time Distribution (since startup):
    n =    98432,  median ~=       27 ms
    0 ms:            5634
    1 ms:            2933
    2 ms:            3172
    <= 5 ms:         3315
    <= 10 ms:        4871
    <= 20 ms:       24484
    <= 50 ms:       20130
    <= 100 ms:       5745
    <= 200 ms:       8453
    <= 500 ms:       6038
    <= 1 s:          4277
    <= 2 s:          4737
    <= 5 s:          2407
    <= 10 s:         2102
    <= 20 s:           67
    <= 1 min:          24
    <= 2 min:          20
    <= 5 min:          21
    <= 10 min:          2
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


TaskThread Failed Time Distribution (since last Daily Report):
    n =        0

TaskThread Failed Time Distribution (since startup):
    n =        0

TaskThread Succeeded Time Distribution (since last Daily Report):
    n =        0

TaskThread Succeeded Time Distribution (since startup):
    n =        0


SgtMap topography nFromCache=15 nNotFromCache=155
GSHHS: nCached=100 of 100, nCoarse=1352, nSuccesses=804, nTossed=847
NationalBoundaries: nCached=100 of 100,  nCoarse=123, nSuccesses=218, nTossed=0
StateBoundaries: nCached=91 of 100,  nCoarse=100, nSuccesses=209, nTossed=0
Rivers: nCached=100 of 100,  nCoarse=118, nSuccesses=218, nTossed=0
bufferedImage isAccelerated=[unknown]

Number of threads: Tomcat-waiting=0, inotify=0, other=67
(format: #threadNumber Thread[threadName,threadPriority,threadGroup] threadStatus)

#1 Thread[AsyncFileHandlerWriter-1867750575,5,main] TIMED_WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513)
java.base@11.0.7/java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675)
app//org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:159)

#2 Thread[cluster1-connection-reaper-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#3 Thread[cluster1-nio-worker-0,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#4 Thread[cluster1-nio-worker-1,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#5 Thread[cluster1-nio-worker-2,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#6 Thread[cluster1-nio-worker-3,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#7 Thread[cluster1-nio-worker-4,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#8 Thread[cluster1-nio-worker-5,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#9 Thread[cluster1-nio-worker-6,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#10 Thread[cluster1-nio-worker-7,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#11 Thread[cluster1-reconnection-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#12 Thread[cluster1-reconnection-1,5,main] WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#13 Thread[cluster1-scheduled-task-worker-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#14 Thread[cluster1-timeouter-0,5,main] TIMED_WAITING
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:560)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:459)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#15 Thread[cluster2-connection-reaper-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#16 Thread[cluster2-nio-worker-0,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#17 Thread[cluster2-nio-worker-1,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#18 Thread[cluster2-nio-worker-2,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#19 Thread[cluster2-nio-worker-3,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#20 Thread[cluster2-nio-worker-4,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#21 Thread[cluster2-nio-worker-5,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#22 Thread[cluster2-nio-worker-6,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#23 Thread[cluster2-nio-worker-7,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#24 Thread[cluster2-reconnection-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#25 Thread[cluster2-reconnection-1,5,main] WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#26 Thread[cluster2-scheduled-task-worker-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#27 Thread[cluster2-timeouter-0,5,main] TIMED_WAITING
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:560)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:459)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#28 Thread[cluster3-connection-reaper-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#29 Thread[cluster3-nio-worker-0,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#30 Thread[cluster3-nio-worker-1,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#31 Thread[cluster3-nio-worker-2,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#32 Thread[cluster3-nio-worker-3,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#33 Thread[cluster3-nio-worker-4,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#34 Thread[cluster3-nio-worker-5,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#35 Thread[cluster3-nio-worker-6,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#36 Thread[cluster3-nio-worker-7,5,main] RUNNABLE
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#37 Thread[cluster3-reconnection-0,5,main] WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#38 Thread[cluster3-reconnection-1,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#39 Thread[cluster3-scheduled-task-worker-0,5,main] TIMED_WAITING
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
java.base@11.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#40 Thread[cluster3-timeouter-0,5,main] TIMED_WAITING
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:560)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:459)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#41 Thread[Common-Cleaner,8,InnocuousThreadGroup] TIMED_WAITING daemon
java.base@11.0.7/java.lang.Object.wait(Native Method)
java.base@11.0.7/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.7/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)
java.base@11.0.7/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)

#42 Thread[ContainerBackgroundProcessor[StandardEngine[Catalina]],5,main] TIMED_WAITING daemon
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1365)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#43 Thread[FileHandlerLogFilesCleaner-1,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#44 Thread[FileSystemWatchService,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.fs.LinuxWatchService.poll(Native Method)
java.base@11.0.7/sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:316)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#45 Thread[FileSystemWatchService,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.fs.LinuxWatchService.poll(Native Method)
java.base@11.0.7/sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:316)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#46 Thread[Finalizer,8,system] WAITING daemon
java.base@11.0.7/java.lang.Object.wait(Native Method)
java.base@11.0.7/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.7/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
java.base@11.0.7/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)

#47 Thread[http-nio-8080-Acceptor-0,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
java.base@11.0.7/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
java.base@11.0.7/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:446)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:70)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#48 Thread[http-nio-8080-AsyncTimeout,5,main] TIMED_WAITING daemon
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
org.apache.coyote.AbstractProtocol$AsyncTimeout.run(AbstractProtocol.java:1113)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#49 Thread[http-nio-8080-ClientPoller-0,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:742)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#50 Thread[http-nio-8080-ClientPoller-1,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:742)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#51 Thread[http-nio-8080-exec-5418,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#52 Thread[http-nio-8080-exec-5442,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#53 Thread[http-nio-8080-exec-5461,5,main] RUNNABLE daemon
java.base@11.0.7/java.lang.Thread.dumpThreads(Native Method)
java.base@11.0.7/java.lang.Thread.getAllStackTraces(Thread.java:1657)
com.cohort.util.MustBe.allStackTraces(MustBe.java:353)
gov.noaa.pfel.erddap.Erddap.doStatus(Erddap.java:3178)
gov.noaa.pfel.erddap.Erddap.doGet(Erddap.java:595)
javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#54 Thread[http-nio-8080-exec-5474,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#55 Thread[http-nio-8080-exec-5480,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#56 Thread[http-nio-8080-exec-5489,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#57 Thread[http-nio-8080-exec-5490,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#58 Thread[http-nio-8080-exec-5492,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#59 Thread[http-nio-8080-exec-5493,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#60 Thread[http-nio-8080-exec-5494,5,main] WAITING daemon
java.base@11.0.7/jdk.internal.misc.Unsafe.park(Native Method)
java.base@11.0.7/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
java.base@11.0.7/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
java.base@11.0.7/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
java.base@11.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#61 Thread[Java2D Disposer,10,system] WAITING daemon
java.base@11.0.7/java.lang.Object.wait(Native Method)
java.base@11.0.7/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
java.base@11.0.7/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
java.desktop@11.0.7/sun.java2d.Disposer.run(Disposer.java:144)
java.base@11.0.7/java.lang.Thread.run(Thread.java:834)

#62 Thread[LoadDatasets,5,main] RUNNABLE daemon
java.base@11.0.7/java.net.SocketInputStream.socketRead0(Native Method)
java.base@11.0.7/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
java.base@11.0.7/java.net.SocketInputStream.read(SocketInputStream.java:168)
java.base@11.0.7/java.net.SocketInputStream.read(SocketInputStream.java:140)
com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1971)
com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6369)
com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7627)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:576)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:508)
com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7240)
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2869)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:243)
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:218)
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:434)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.getDataForDapQuery(EDDTableFromDatabase.java:860)
gov.noaa.pfel.erddap.dataset.EDDTable.subsetVariablesDataTable(EDDTable.java:11509)
gov.noaa.pfel.erddap.dataset.EDDTable.distinctSubsetVariablesDataTable(EDDTable.java:11575)
gov.noaa.pfel.erddap.dataset.EDDTable.ensureValid(EDDTable.java:654)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.<init>(EDDTableFromDatabase.java:577)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.lowFromXml(EDDTableFromDatabase.java:232)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.fromXml(EDDTableFromDatabase.java:93)
gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:403)
gov.noaa.pfel.erddap.LoadDatasets.run(LoadDatasets.java:336)

#63 Thread[main,5,main] RUNNABLE
java.base@11.0.7/java.net.PlainSocketImpl.socketAccept(Native Method)
java.base@11.0.7/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
java.base@11.0.7/java.net.ServerSocket.implAccept(ServerSocket.java:565)
java.base@11.0.7/java.net.ServerSocket.accept(ServerSocket.java:533)
org.apache.catalina.core.StandardServer.await(StandardServer.java:447)
org.apache.catalina.startup.Catalina.await(Catalina.java:770)
org.apache.catalina.startup.Catalina.start(Catalina.java:716)
java.base@11.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base@11.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base@11.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base@11.0.7/java.lang.reflect.Method.invoke(Method.java:566)
app//org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350)
app//org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)

#64 Thread[NioBlockingSelector.BlockPoller-1,5,main] RUNNABLE daemon
java.base@11.0.7/sun.nio.ch.EPoll.wait(Native Method)
java.base@11.0.7/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
java.base@11.0.7/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
java.base@11.0.7/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:298)

#65 Thread[Reference Handler,10,system] RUNNABLE daemon
java.base@11.0.7/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
java.base@11.0.7/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
java.base@11.0.7/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

#66 Thread[RunLoadDatasets,5,main] TIMED_WAITING daemon
java.base@11.0.7/java.lang.Thread.sleep(Native Method)
gov.noaa.pfel.erddap.RunLoadDatasets.run(RunLoadDatasets.java:282)

#67 Thread[Signal Dispatcher,9,system] RUNNABLE daemon


 
ERDDAP, Version 1.82
Disclaimers | Privacy Policy | Contact