Serving data from the Marine Institute (Ireland)
Brought to you by the Marine Institute(Ireland)
|
Marine Institute ERDDAP Server
Serving data from the Marine Institute (Ireland) |
Brought to you by the Marine Institute(Ireland) |
Current time is 2025-08-11T20:46:47+00:00
Startup was at 2025-05-29T07:16:55+00:00
Last major LoadDatasets started 14m 46s ago and finished after 80 seconds.
nGridDatasets = 9
nTableDatasets = 72
nTotalDatasets = 81
n Datasets Failed To Load (in the last major LoadDatasets) = 6
ireland_marine_research_database_charts, imifeeaghco2, compass_neatl_hindcast_grid,
IMI_Model_Stats, spiddal_obs_adcp, IMI_EATL_WAVE, (end)
Unique users (since startup) n = 220694
Response Failed Time (since last major LoadDatasets) n = 20, median ~= 200 ms
Response Failed Time (since last Daily Report) n = 2051, median ~= 610 ms
Response Failed Time (since startup) n = 329919, median ~= 507 ms
Response Succeeded Time (since last major LoadDatasets) n = 176, median ~= 140 ms
Response Succeeded Time (since last Daily Report) n = 10787, median ~= 143 ms
Response Succeeded Time (since startup) n = 1455061, median ~= 190 ms
TaskThread has finished 0 out of 0 tasks. Currently, no task is running.
TaskThread Failed Time (since last Daily Report) n = 0
TaskThread Failed Time (since startup) n = 0
TaskThread Succeeded Time (since last Daily Report) n = 0
TaskThread Succeeded Time (since startup) n = 0
Number of threads: Tomcat-waiting=42, inotify=4, other=53
MemoryInUse= 1999 MB (highWaterMark= 3426 MB) (Xmx ~= 5888 MB)
Major LoadDatasets Time Series: MLD Datasets Loaded Requests (median times in ms) Number of Threads MB Open
timestamp time nTry nFail nTotal nSuccess (median) nFail (median) memFail tooMany tomWait inotify other inUse Files
---------------------------- ----- ----------------- ------------------------------------------------ --------------------- ----- -----
2025-08-11T20:33:17+00:00 76s 11 6 81 148 ( 238) 14 ( 40) 0 0 20 4 54 330 0%
2025-08-11T20:18:21+00:00 80s 12 6 81 155 ( 206) 25 ( 150) 0 0 20 4 54 334 0%
2025-08-11T20:03:18+00:00 77s 11 6 81 151 ( 130) 26 ( 50) 0 0 20 4 54 332 0%
2025-08-11T19:48:30+00:00 89s 16 6 81 136 ( 65) 16 ( 18) 0 0 20 4 54 334 0%
2025-08-11T19:33:21+00:00 81s 11 6 81 174 ( 284) 24 ( 400) 0 0 20 4 54 332 0%
2025-08-11T19:18:17+00:00 77s 12 6 81 124 ( 46) 10 ( 32) 0 0 20 4 54 332 0%
2025-08-11T19:03:18+00:00 78s 11 6 81 141 ( 154) 26 ( 28) 0 0 20 4 54 332 0%
2025-08-11T18:48:19+00:00 79s 12 6 81 171 ( 82) 27 ( 47) 0 0 30 4 54 332 0%
2025-08-11T18:33:33+00:00 93s 14 6 81 205 ( 268) 22 ( 275) 0 0 20 4 54 424 0%
2025-08-11T18:18:17+00:00 77s 12 6 81 155 ( 44) 14 ( 20) 0 0 20 4 54 331 0%
2025-08-11T18:03:18+00:00 79s 11 6 81 204 ( 83) 31 ( 88) 0 0 23 4 54 332 0%
2025-08-11T17:48:17+00:00 77s 12 6 81 152 ( 50) 14 ( 20) 0 0 20 4 54 329 0%
2025-08-11T17:33:31+00:00 91s 14 6 81 201 ( 154) 19 ( 75) 0 0 27 4 54 320 0%
2025-08-11T17:18:17+00:00 78s 12 6 81 193 ( 41) 35 ( 750) 0 0 24 4 54 332 0%
2025-08-11T17:03:19+00:00 80s 11 6 81 165 ( 306) 31 ( 1250) 0 0 20 4 54 330 0%
2025-08-11T16:48:18+00:00 79s 12 6 81 159 ( 46) 19 ( 5) 0 0 20 4 54 332 0%
2025-08-11T16:33:28+00:00 89s 16 6 81 222 ( 88) 13 ( 39) 0 0 20 4 54 329 0%
2025-08-11T16:18:17+00:00 78s 13 6 81 156 ( 57) 14 ( 20) 0 0 20 4 54 330 0%
2025-08-11T16:03:22+00:00 83s 12 7 80 256 ( 77) 19 ( 31) 0 0 26 4 54 332 0%
2025-08-11T15:48:16+00:00 77s 14 7 80 248 ( 448) 24 ( 33) 0 0 53 4 54 332 0%
2025-08-11T15:33:14+00:00 76s 11 6 81 218 ( 245) 30 ( 750) 0 0 24 4 54 332 0%
2025-08-11T15:18:31+00:00 93s 15 6 81 160 ( 50) 24 ( 750) 0 0 27 4 54 331 0%
2025-08-11T15:03:15+00:00 76s 11 6 81 163 ( 191) 13 ( 28) 0 0 20 4 54 331 0%
2025-08-11T14:48:15+00:00 77s 13 6 81 181 ( 47) 21 ( 18) 0 0 20 4 54 330 0%
2025-08-11T14:33:22+00:00 84s 11 6 81 167 ( 196) 23 ( 29) 0 0 19 4 55 330 0%
2025-08-11T14:18:21+00:00 83s 12 6 81 197 ( 96) 22 ( 37) 0 0 20 4 54 332 0%
2025-08-11T14:03:31+00:00 93s 17 6 81 221 ( 161) 25 ( 1) 0 0 26 4 54 329 0%
2025-08-11T13:48:16+00:00 78s 12 6 81 261 ( 480) 28 ( 33) 0 0 20 4 54 329 0%
2025-08-11T13:33:15+00:00 77s 11 6 81 199 ( 175) 15 ( 46) 0 0 20 4 54 331 0%
2025-08-11T13:18:16+00:00 79s 13 6 81 178 ( 47) 36 ( 667) 0 0 25 4 54 329 0%
2025-08-11T13:03:14+00:00 77s 12 6 81 170 ( 75) 34 ( 50) 0 0 27 4 54 330 0%
2025-08-11T12:48:29+00:00 92s 15 6 81 273 ( 509) 41 ( 28) 0 0 26 4 54 329 0%
2025-08-11T12:33:16+00:00 79s 11 6 81 222 ( 290) 22 ( 425) 0 0 20 4 54 330 0%
2025-08-11T12:18:37+00:00 100s 12 6 81 293 ( 107) 56 ( 1333) 0 0 41 4 54 331 0%
2025-08-11T12:03:16+00:00 79s 12 6 81 282 ( 186) 28 ( 50) 0 0 25 4 55 496 0%
2025-08-11T11:48:29+00:00 93s 16 6 81 210 ( 59) 24 ( 29) 0 0 27 4 54 324 0%
2025-08-11T11:33:14+00:00 78s 18 6 81 230 ( 209) 25 ( 130) 0 0 20 4 54 327 0%
2025-08-11T11:18:15+00:00 79s 12 6 81 169 ( 42) 38 ( 30) 0 0 20 4 54 330 0%
2025-08-11T11:03:33+00:00 97s 16 6 81 158 ( 48) 99 ( 886) 0 0 20 4 54 328 0%
2025-08-11T10:48:32+00:00 95s 15 6 81 290 ( 569) 208 ( 871) 0 0 41 4 55 317 0%
2025-08-11T10:33:13+00:00 77s 11 6 81 175 ( 117) 14 ( 32) 0 0 20 4 54 332 0%
2025-08-11T10:18:13+00:00 78s 12 6 81 196 ( 40) 20 ( 43) 0 0 26 4 54 332 0%
2025-08-11T10:03:49+00:00 113s 32 6 81 212 ( 46) 28 ( 200) 0 0 20 4 54 330 0%
2025-08-11T09:48:25+00:00 89s 13 6 81 241 ( 46) 20 ( 50) 0 0 21 4 54 336 0%
2025-08-11T09:33:45+00:00 110s 14 6 81 191 ( 278) 26 ( 500) 0 26 37 4 54 332 0%
2025-08-11T09:18:14+00:00 79s 12 6 81 177 ( 88) 25 ( 48) 0 0 20 4 54 330 0%
2025-08-11T09:03:13+00:00 77s 11 6 81 228 ( 320) 146 ( 820) 2 0 21 4 54 329 0%
2025-08-11T08:48:25+00:00 90s 13 6 81 200 ( 344) 43 ( 450) 0 0 22 4 54 331 0%
2025-08-11T08:33:14+00:00 79s 13 6 81 294 ( 150) 33 ( 238) 0 0 27 4 54 329 0%
2025-08-11T08:18:15+00:00 81s 11 5 82 213 ( 207) 96 ( 729) 0 0 20 4 54 336 0%
2025-08-11T08:03:17+00:00 82s 10 5 82 184 ( 173) 219 ( 844) 0 0 19 4 55 492 0%
2025-08-11T07:48:17+00:00 83s 11 5 82 198 ( 254) 21 ( 48) 0 0 22 4 54 328 0%
2025-08-11T07:33:36+00:00 101s 21 5 82 192 ( 293) 26 ( 500) 0 0 20 4 54 326 0%
2025-08-11T07:18:18+00:00 84s 14 6 81 153 ( 113) 79 ( 990) 0 0 24 4 54 328 0%
2025-08-11T07:03:17+00:00 83s 14 6 81 204 ( 117) 133 ( 978) 0 0 27 4 54 330 0%
2025-08-11T06:48:21+00:00 87s 12 6 81 328 ( 466) 588 ( 1194) 3 0 29 4 55 387 0%
2025-08-11T06:33:29+00:00 95s 12 6 81 206 ( 209) 69 ( 492) 0 0 24 4 55 333 0%
2025-08-11T06:18:36+00:00 102s 14 6 81 192 ( 248) 44 ( 500) 0 0 20 4 54 328 0%
2025-08-11T06:03:13+00:00 80s 11 6 81 116 ( 75) 11 ( 31) 0 0 20 4 54 327 0%
2025-08-11T05:48:09+00:00 76s 12 6 81 134 ( 91) 12 ( 32) 0 0 20 4 54 327 0%
2025-08-11T05:33:26+00:00 94s 11 6 81 151 ( 278) 26 ( 500) 0 0 19 4 55 329 0%
2025-08-11T05:18:50+00:00 117s 15 6 81 122 ( 69) 34 ( 313) 0 0 20 4 54 327 0%
2025-08-11T05:03:13+00:00 80s 10 5 82 101 ( 88) 11 ( 24) 0 0 20 4 54 331 0%
2025-08-11T04:48:10+00:00 77s 11 5 82 120 ( 77) 12 ( 29) 0 0 20 4 54 331 0%
2025-08-11T04:33:09+00:00 77s 10 5 82 144 ( 252) 12 ( 43) 0 0 20 4 54 332 0%
2025-08-11T04:18:14+00:00 82s 11 5 82 110 ( 67) 11 ( 29) 0 0 20 4 54 329 0%
2025-08-11T04:03:26+00:00 94s 12 5 82 96 ( 56) 8 ( 40) 0 0 20 4 54 332 0%
2025-08-11T03:51:58+00:00 305s 11 5 82 135 ( 78) 57 (180698) 0 0 35 4 54 336 0%
2025-08-11T03:33:15+00:00 83s 12 5 82 149 ( 245) 12 ( 43) 0 0 20 4 54 334 0%
2025-08-11T03:18:12+00:00 80s 11 5 82 95 ( 76) 12 ( 33) 0 0 20 4 54 331 0%
2025-08-11T03:03:27+00:00 95s 13 5 82 104 ( 88) 10 ( 30) 0 0 20 4 54 329 0%
2025-08-11T02:48:14+00:00 82s 11 5 82 121 ( 144) 12 ( 30) 0 0 20 4 54 334 0%
2025-08-11T02:33:10+00:00 78s 10 5 82 146 ( 258) 12 ( 43) 0 0 20 4 54 331 0%
2025-08-11T02:18:13+00:00 81s 11 5 82 114 ( 86) 19 ( 48) 0 0 26 4 54 332 0%
2025-08-11T02:03:20+00:00 89s 11 5 82 170 ( 37) 10 ( 28) 0 0 20 4 54 329 0%
2025-08-11T01:48:24+00:00 92s 16 6 81 134 ( 68) 11 ( 31) 0 0 20 4 54 331 0%
2025-08-11T01:33:10+00:00 79s 10 5 82 137 ( 264) 17 ( 24) 0 0 20 4 54 330 0%
2025-08-11T01:18:09+00:00 78s 11 5 82 132 ( 67) 17 ( 1) 0 0 20 4 54 331 0%
2025-08-11T01:03:15+00:00 84s 11 5 82 211 ( 129) 22 ( 43) 0 0 22 4 54 330 0%
2025-08-11T00:48:14+00:00 83s 11 5 82 135 ( 150) 13 ( 38) 0 0 20 4 54 331 0%
2025-08-11T00:33:35+00:00 104s 13 5 82 145 ( 258) 14 ( 35) 0 0 20 4 54 327 0%
2025-08-11T00:18:08+00:00 77s 11 5 82 136 ( 75) 15 ( 31) 0 0 24 4 55 334 0%
2025-08-11T00:03:12+00:00 82s 10 5 82 180 ( 38) 12 ( 25) 0 0 20 4 54 332 0%
2025-08-10T23:48:10+00:00 79s 12 5 82 144 ( 222) 20 ( 46) 0 0 24 4 54 331 0%
2025-08-10T23:33:08+00:00 77s 14 6 81 149 ( 273) 20 ( 275) 0 0 20 4 54 331 0%
2025-08-10T23:18:22+00:00 92s 15 6 81 120 ( 49) 12 ( 28) 0 0 20 4 54 391 0%
2025-08-10T23:03:08+00:00 77s 11 6 81 120 ( 107) 10 ( 28) 0 0 20 4 54 329 0%
2025-08-10T22:48:11+00:00 81s 12 6 81 149 ( 68) 13 ( 29) 0 0 22 4 54 331 0%
2025-08-10T22:33:06+00:00 76s 11 6 81 191 ( 183) 12 ( 38) 0 0 20 4 54 328 0%
2025-08-10T22:18:21+00:00 90s 15 6 81 131 ( 49) 12 ( 35) 0 0 20 4 54 330 0%
2025-08-10T22:03:11+00:00 81s 11 6 81 151 ( 154) 31 ( 2375) 0 0 27 4 54 335 0%
2025-08-10T21:48:09+00:00 79s 12 6 81 173 ( 193) 36 ( 3500) 0 0 20 4 54 329 0%
2025-08-10T21:33:08+00:00 78s 11 6 81 192 ( 350) 69 ( 4192) 0 0 26 4 55 323 0%
Major LoadDatasets Times Distribution (since last Daily Report):
n = 54, median ~= 90000 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 0
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 0
<= 10 s: 0
<= 20 s: 0
<= 1 min: 0
<= 2 min: 54
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Major LoadDatasets Times Distribution (since startup):
n = 7148, median ~= 90728 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 1
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 2
<= 10 s: 0
<= 20 s: 0
<= 1 min: 2
<= 2 min: 6969
<= 5 min: 142
<= 10 min: 26
<= 20 min: 5
<= 1 hr: 1
> 1 hr: 0
Minor LoadDatasets Times Distribution (since last Daily Report):
n = 1, median ~= 3500 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 0
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 1
<= 10 s: 0
<= 20 s: 0
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Minor LoadDatasets Times Distribution (since startup):
n = 267, median ~= 2496 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 1
<= 200 ms: 47
<= 500 ms: 56
<= 1 s: 10
<= 2 s: 0
<= 5 s: 118
<= 10 s: 33
<= 20 s: 2
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since last major LoadDatasets):
n = 20, median ~= 200 ms
0 ms: 3
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 5
<= 100 ms: 1
<= 200 ms: 1
<= 500 ms: 0
<= 1 s: 1
<= 2 s: 1
<= 5 s: 4
<= 10 s: 4
<= 20 s: 0
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since last Daily Report):
n = 2051, median ~= 610 ms
0 ms: 195
1 ms: 92
2 ms: 2
<= 5 ms: 37
<= 10 ms: 15
<= 20 ms: 71
<= 50 ms: 298
<= 100 ms: 36
<= 200 ms: 54
<= 500 ms: 120
<= 1 s: 479
<= 2 s: 264
<= 5 s: 178
<= 10 s: 164
<= 20 s: 34
<= 1 min: 12
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since startup):
n = 329919, median ~= 507 ms
0 ms: 25119
1 ms: 11958
2 ms: 383
<= 5 ms: 2313
<= 10 ms: 1270
<= 20 ms: 4765
<= 50 ms: 44729
<= 100 ms: 9683
<= 200 ms: 13031
<= 500 ms: 50938
<= 1 s: 56969
<= 2 s: 47926
<= 5 s: 37291
<= 10 s: 14320
<= 20 s: 6074
<= 1 min: 1240
<= 2 min: 284
<= 5 min: 1598
<= 10 min: 17
<= 20 min: 5
<= 1 hr: 2
> 1 hr: 4
Response Succeeded Time Distribution (since last major LoadDatasets):
n = 176, median ~= 140 ms
0 ms: 3
1 ms: 5
2 ms: 7
<= 5 ms: 5
<= 10 ms: 3
<= 20 ms: 13
<= 50 ms: 40
<= 100 ms: 8
<= 200 ms: 10
<= 500 ms: 27
<= 1 s: 16
<= 2 s: 11
<= 5 s: 17
<= 10 s: 9
<= 20 s: 2
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Succeeded Time Distribution (since last Daily Report):
n = 10787, median ~= 143 ms
0 ms: 216
1 ms: 465
2 ms: 241
<= 5 ms: 483
<= 10 ms: 96
<= 20 ms: 588
<= 50 ms: 2305
<= 100 ms: 756
<= 200 ms: 566
<= 500 ms: 1921
<= 1 s: 929
<= 2 s: 857
<= 5 s: 853
<= 10 s: 397
<= 20 s: 85
<= 1 min: 28
<= 2 min: 1
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Succeeded Time Distribution (since startup):
n = 1455061, median ~= 190 ms
0 ms: 41133
1 ms: 59896
2 ms: 29884
<= 5 ms: 63533
<= 10 ms: 20198
<= 20 ms: 72138
<= 50 ms: 257080
<= 100 ms: 83796
<= 200 ms: 111148
<= 500 ms: 367644
<= 1 s: 116053
<= 2 s: 95361
<= 5 s: 88031
<= 10 s: 33816
<= 20 s: 9128
<= 1 min: 5220
<= 2 min: 720
<= 5 min: 132
<= 10 min: 99
<= 20 min: 35
<= 1 hr: 13
> 1 hr: 3
TaskThread Failed Time Distribution (since last Daily Report):
n = 0
TaskThread Failed Time Distribution (since startup):
n = 0
TaskThread Succeeded Time Distribution (since last Daily Report):
n = 0
TaskThread Succeeded Time Distribution (since startup):
n = 0
SgtMap topography nFromCache=118 nNotFromCache=1119
GSHHS: nCached=100 of 100, nCoarse=13012, nSuccesses=4107, nTossed=3263
NationalBoundaries: nCached=100 of 100, nCoarse=1738, nSuccesses=509, nTossed=249
StateBoundaries: nCached=100 of 100, nCoarse=1295, nSuccesses=509, nTossed=211
Rivers: nCached=100 of 100, nCoarse=1419, nSuccesses=508, nTossed=254
bufferedImage isAccelerated=[unknown]
canonical map sizes: 887 + 739 + 271 + 1567 + 1012 + 1465 = 5941
canonicalStringHolder map sizes: 1172 + 1025 + 1253 + 474 + 206 + 348 = 4478
Number of threads: Tomcat-waiting=42, inotify=4, other=53
(format: #threadNumber Thread[threadName,threadPriority,threadGroup] threadStatus)
#1 Thread[AsyncFileHandlerWriter-1,5,main] WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
#2 Thread[Catalina-utility-1,1,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
java.lang.Thread.run(Thread.java:750)
#3 Thread[Catalina-utility-2,1,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
java.lang.Thread.run(Thread.java:750)
#4 Thread[cluster1-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#5 Thread[cluster1-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#6 Thread[cluster1-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#7 Thread[cluster1-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#8 Thread[cluster1-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#9 Thread[cluster1-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:286)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#10 Thread[cluster1-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:286)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#11 Thread[cluster1-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:286)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#12 Thread[cluster1-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#13 Thread[cluster1-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#14 Thread[cluster1-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#15 Thread[cluster1-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#16 Thread[cluster1-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#17 Thread[cluster2-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#18 Thread[cluster2-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#19 Thread[cluster2-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#20 Thread[cluster2-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#21 Thread[cluster2-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#22 Thread[cluster2-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#23 Thread[cluster2-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#24 Thread[cluster2-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#25 Thread[cluster2-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#26 Thread[cluster2-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#27 Thread[cluster2-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#28 Thread[cluster2-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#29 Thread[cluster3-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#30 Thread[cluster3-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#31 Thread[cluster3-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#32 Thread[cluster3-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#33 Thread[cluster3-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#34 Thread[cluster3-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#35 Thread[cluster3-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#36 Thread[cluster3-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#37 Thread[cluster3-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#38 Thread[cluster3-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#39 Thread[cluster3-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#40 Thread[cluster3-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#41 Thread[cluster3-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#42 Thread[FileHandlerLogFilesCleaner-1,5,main] WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
#43 Thread[Finalizer,8,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:188)
#44 Thread[GC Daemon,2,system] TIMED_WAITING daemon
java.lang.Object.wait(Native Method)
sun.misc.GC$Daemon.run(GC.java:117)
#45 Thread[http-nio-8080-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:71)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
java.lang.Thread.run(Thread.java:750)
#46 Thread[http-nio-8080-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:811)
java.lang.Thread.run(Thread.java:750)
#47 Thread[https-jsse-nio-8443-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:71)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
java.lang.Thread.run(Thread.java:750)
#48 Thread[https-jsse-nio-8443-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:811)
java.lang.Thread.run(Thread.java:750)
#49 Thread[Java2D Disposer,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
sun.java2d.Disposer.run(Disposer.java:148)
java.lang.Thread.run(Thread.java:750)
#50 Thread[main,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
org.apache.catalina.core.StandardServer.await(StandardServer.java:524)
org.apache.catalina.startup.Catalina.await(Catalina.java:829)
org.apache.catalina.startup.Catalina.start(Catalina.java:777)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345)
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476)
#51 Thread[Reference Handler,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
java.lang.ref.Reference.tryHandlePending(Reference.java:191)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
#52 Thread[RunLoadDatasets,5,main] TIMED_WAITING daemon
java.lang.Thread.sleep(Native Method)
gov.noaa.pfel.erddap.RunLoadDatasets.run(RunLoadDatasets.java:288)
#53 Thread[Signal Dispatcher,9,system] RUNNABLE daemon