Serving data from the Marine Institute (Ireland)
Brought to you by the Marine Institute(Ireland)
|
Marine Institute ERDDAP Server
Serving data from the Marine Institute (Ireland) |
Brought to you by the Marine Institute(Ireland) |
Current time is 2025-08-31T23:55:31+00:00
Startup was at 2025-05-29T07:16:55+00:00
Last major LoadDatasets started 4m 20s ago and finished after 80 seconds.
nGridDatasets = 9
nTableDatasets = 74
nTotalDatasets = 83
n Datasets Failed To Load (in the last major LoadDatasets) = 5
ireland_marine_research_database_charts, imifeeaghco2, compass_neatl_hindcast_grid,
IMI_Model_Stats, IMI_EATL_WAVE, (end)
Unique users (since startup) n = 235849
Response Failed Time (since last major LoadDatasets) n = 3, median ~= 75 ms
Response Failed Time (since last Daily Report) n = 2407, median ~= 750 ms
Response Failed Time (since startup) n = 400508, median ~= 527 ms
Response Succeeded Time (since last major LoadDatasets) n = 24, median ~= 160 ms
Response Succeeded Time (since last Daily Report) n = 13386, median ~= 286 ms
Response Succeeded Time (since startup) n = 1848651, median ~= 194 ms
TaskThread has finished 0 out of 0 tasks. Currently, no task is running.
TaskThread Failed Time (since last Daily Report) n = 0
TaskThread Failed Time (since startup) n = 0
TaskThread Succeeded Time (since last Daily Report) n = 0
TaskThread Succeeded Time (since startup) n = 0
Number of threads: Tomcat-waiting=19, inotify=4, other=53
MemoryInUse= 713 MB (highWaterMark= 3426 MB) (Xmx ~= 5888 MB)
Major LoadDatasets Time Series: MLD Datasets Loaded Requests (median times in ms) Number of Threads MB Open
timestamp time nTry nFail nTotal nSuccess (median) nFail (median) memFail tooMany tomWait inotify other inUse Files
---------------------------- ----- ----------------- ------------------------------------------------ --------------------- ----- -----
2025-08-31T23:52:28+00:00 77s 11 5 83 107 ( 110) 6 ( 50) 0 0 20 4 54 342 0%
2025-08-31T23:37:30+00:00 79s 11 5 83 228 ( 259) 25 ( 264) 0 0 20 4 54 339 0%
2025-08-31T23:22:43+00:00 93s 13 5 83 91 ( 70) 6 ( 43) 0 0 20 4 54 339 0%
2025-08-31T23:07:28+00:00 78s 10 5 83 154 ( 63) 16 ( 28) 0 0 20 4 54 342 0%
2025-08-31T22:52:31+00:00 80s 13 5 83 121 ( 163) 19 ( 63) 0 0 20 4 54 341 0%
2025-08-31T22:37:29+00:00 79s 12 5 83 247 ( 281) 19 ( 48) 0 0 20 4 54 343 0%
2025-08-31T22:22:28+00:00 78s 10 5 83 85 ( 84) 9 ( 35) 0 0 20 4 54 342 0%
2025-08-31T22:07:41+00:00 91s 14 5 83 135 ( 79) 12 ( 31) 0 0 20 4 54 341 0%
2025-08-31T21:52:27+00:00 77s 11 5 83 315 ( 805) 42 ( 3500) 0 0 30 4 54 338 0%
2025-08-31T21:37:31+00:00 82s 18 5 83 232 ( 274) 19 ( 41) 0 0 11 4 88 670 0%
2025-08-31T21:22:29+00:00 79s 11 5 83 89 ( 117) 28 ( 8) 0 0 20 4 54 340 0%
2025-08-31T21:07:29+00:00 80s 11 6 82 149 ( 123) 29 ( 33) 0 0 20 4 54 341 0%
2025-08-31T20:52:43+00:00 93s 15 6 82 138 ( 304) 12 ( 38) 0 0 20 4 54 339 0%
2025-08-31T20:37:28+00:00 79s 12 6 82 284 ( 358) 33 ( 215) 0 0 21 4 54 344 0%
2025-08-31T20:22:28+00:00 79s 11 6 82 128 ( 364) 17 ( 23) 0 0 29 4 54 321 0%
2025-08-31T20:07:35+00:00 86s 11 6 82 204 ( 91) 44 ( 4500) 0 0 20 4 54 341 0%
2025-08-31T19:52:41+00:00 92s 15 6 82 157 ( 536) 19 ( 28) 0 0 20 4 54 341 0%
2025-08-31T19:37:30+00:00 81s 12 6 82 269 ( 281) 33 ( 475) 0 0 20 4 54 339 0%
2025-08-31T19:22:29+00:00 80s 11 6 82 159 ( 310) 15 ( 41) 0 0 24 4 54 339 0%
2025-08-31T19:07:28+00:00 79s 11 6 82 145 ( 106) 11 ( 32) 0 0 20 4 54 340 0%
2025-08-31T18:52:42+00:00 94s 15 6 82 114 ( 175) 6 ( 100) 0 0 20 4 54 339 0%
2025-08-31T18:37:29+00:00 81s 13 6 82 225 ( 261) 22 ( 300) 0 0 20 4 54 342 0%
2025-08-31T18:22:27+00:00 79s 11 6 82 111 ( 45) 6 ( 38) 0 0 20 4 54 342 0%
2025-08-31T18:07:35+00:00 87s 11 6 82 153 ( 105) 25 ( 45) 0 0 20 4 54 341 0%
2025-08-31T17:52:39+00:00 91s 15 6 82 151 ( 319) 21 ( 2375) 0 0 20 4 54 339 0%
2025-08-31T17:37:29+00:00 80s 12 6 82 248 ( 292) 31 ( 436) 0 0 25 4 54 342 0%
2025-08-31T17:22:29+00:00 81s 11 6 82 115 ( 170) 10 ( 100) 0 0 20 4 54 342 0%
2025-08-31T17:07:29+00:00 81s 14 6 82 176 ( 67) 12 ( 43) 0 0 20 4 54 342 0%
2025-08-31T16:52:26+00:00 78s 12 6 82 120 ( 213) 10 ( 26) 0 0 20 4 54 342 0%
2025-08-31T16:37:43+00:00 95s 15 6 82 295 ( 347) 46 ( 1000) 0 0 26 4 54 340 0%
2025-08-31T16:22:25+00:00 78s 11 6 82 84 ( 90) 7 ( 88) 0 0 20 4 54 341 0%
2025-08-31T16:07:31+00:00 84s 11 6 82 196 ( 79) 18 ( 300) 0 0 26 4 54 341 0%
2025-08-31T15:52:26+00:00 78s 12 6 82 145 ( 428) 10 ( 50) 0 0 20 4 54 339 0%
2025-08-31T15:37:41+00:00 93s 17 6 82 264 ( 315) 36 ( 350) 0 0 20 4 54 337 0%
2025-08-31T15:22:29+00:00 82s 11 6 82 124 ( 70) 23 ( 3500) 0 0 20 4 54 342 0%
2025-08-31T15:07:27+00:00 80s 11 6 82 175 ( 87) 12 ( 43) 0 0 20 4 54 339 0%
2025-08-31T14:52:31+00:00 84s 12 6 82 143 ( 183) 8 ( 44) 0 0 20 4 54 340 0%
2025-08-31T14:37:39+00:00 92s 16 6 82 283 ( 300) 28 ( 414) 0 0 20 4 54 341 0%
2025-08-31T14:22:27+00:00 80s 11 6 82 160 ( 280) 14 ( 45) 0 0 28 4 54 339 0%
2025-08-31T14:07:31+00:00 84s 12 6 82 247 ( 511) 37 ( 2643) 0 0 37 4 54 335 0%
2025-08-31T13:52:25+00:00 79s 13 6 82 131 ( 84) 7 ( 46) 0 0 20 4 54 341 0%
2025-08-31T13:37:40+00:00 93s 15 6 82 240 ( 246) 25 ( 350) 0 0 20 4 54 339 0%
2025-08-31T13:22:29+00:00 83s 11 6 82 149 ( 290) 29 ( 3500) 0 0 20 4 54 345 0%
2025-08-31T13:07:23+00:00 77s 11 6 82 177 ( 95) 13 ( 44) 0 0 28 4 56 603 0%
2025-08-31T12:52:23+00:00 77s 12 6 82 171 ( 306) 17 ( 2300) 0 0 16 4 68 946 0%
2025-08-31T12:37:38+00:00 92s 16 6 82 306 ( 276) 31 ( 275) 0 0 20 4 54 341 0%
2025-08-31T12:22:24+00:00 79s 13 6 82 183 ( 437) 18 ( 100) 0 0 20 4 54 339 0%
2025-08-31T12:07:23+00:00 77s 15 7 81 221 ( 47) 13 ( 35) 0 0 20 4 54 341 0%
2025-08-31T11:52:23+00:00 78s 12 6 82 154 ( 200) 8 ( 43) 0 0 20 4 54 337 0%
2025-08-31T11:37:23+00:00 78s 11 6 82 397 ( 356) 31 ( 264) 0 0 15 4 67 453 0%
2025-08-31T11:23:01+00:00 116s 19 6 82 113 ( 236) 22 ( 400) 0 0 20 4 54 341 0%
2025-08-31T11:07:24+00:00 79s 12 7 81 223 ( 91) 33 ( 41) 0 0 24 4 54 339 0%
2025-08-31T10:52:27+00:00 82s 13 7 81 182 ( 295) 15 ( 175) 0 0 29 4 54 338 0%
2025-08-31T10:37:21+00:00 76s 12 7 81 334 ( 365) 70 ( 1789) 0 0 20 4 54 340 0%
2025-08-31T10:22:38+00:00 94s 23 7 81 186 ( 334) 227 ( 1424) 0 0 42 4 55 334 0%
2025-08-31T10:07:59+00:00 114s 27 7 81 220 ( 141) 50 ( 1400) 0 0 20 4 54 339 0%
2025-08-31T09:52:23+00:00 78s 13 7 81 224 ( 522) 34 ( 2000) 0 0 19 4 64 424 0%
2025-08-31T09:37:57+00:00 113s 13 7 81 334 ( 364) 52 ( 350) 0 0 20 4 54 339 0%
2025-08-31T09:22:24+00:00 80s 13 7 81 235 ( 738) 38 ( 1000) 0 0 29 4 54 320 0%
2025-08-31T09:07:22+00:00 78s 12 7 81 251 ( 533) 277 ( 865) 0 0 20 4 54 340 0%
2025-08-31T08:52:49+00:00 105s 16 7 81 268 ( 633) 172 ( 867) 2 0 18 4 56 818 0%
2025-08-31T08:37:24+00:00 80s 13 7 81 362 ( 326) 34 ( 388) 0 0 20 4 54 342 0%
2025-08-31T08:22:23+00:00 79s 13 7 81 192 ( 459) 31 ( 4700) 0 0 20 4 54 339 0%
2025-08-31T08:07:23+00:00 79s 12 7 81 362 ( 616) 42 ( 1000) 0 0 24 4 54 341 0%
2025-08-31T07:52:40+00:00 96s 16 7 81 219 ( 470) 101 ( 908) 0 0 20 4 65 417 0%
2025-08-31T07:37:33+00:00 90s 20 7 81 383 ( 385) 116 ( 673) 2 0 25 4 54 432 0%
2025-08-31T07:22:31+00:00 88s 13 7 81 181 ( 560) 102 ( 1529) 0 0 21 4 54 335 0%
2025-08-31T07:07:21+00:00 78s 12 7 81 185 ( 99) 13 ( 40) 0 0 20 4 54 340 0%
2025-08-31T06:52:36+00:00 93s 16 7 81 295 ( 844) 52 ( 425) 0 0 22 4 54 341 0%
2025-08-31T06:37:21+00:00 78s 12 7 81 313 ( 281) 19 ( 150) 0 0 24 4 54 341 0%
2025-08-31T06:22:22+00:00 80s 13 7 81 231 ( 314) 41 ( 283) 0 0 20 4 54 337 0%
2025-08-31T06:07:26+00:00 84s 13 7 81 173 ( 98) 15 ( 38) 0 0 20 4 54 344 0%
2025-08-31T05:52:32+00:00 89s 16 7 81 144 ( 325) 11 ( 125) 0 0 20 4 54 338 0%
2025-08-31T05:37:20+00:00 77s 12 7 81 267 ( 276) 19 ( 95) 0 0 20 4 54 338 0%
2025-08-31T05:22:23+00:00 80s 13 7 81 127 ( 119) 7 ( 58) 0 0 21 4 54 339 0%
2025-08-31T05:07:20+00:00 78s 12 7 81 146 ( 75) 21 ( 49) 0 0 20 4 54 341 0%
2025-08-31T04:52:21+00:00 79s 13 7 81 102 ( 150) 7 ( 39) 0 0 20 4 54 344 0%
2025-08-31T04:37:35+00:00 94s 16 7 81 234 ( 271) 26 ( 243) 0 0 20 4 54 341 0%
2025-08-31T04:22:22+00:00 80s 13 7 81 75 ( 98) 8 ( 75) 0 0 20 4 54 343 0%
2025-08-31T04:07:22+00:00 80s 19 7 81 186 ( 92) 13 ( 41) 0 0 36 4 54 338 0%
2025-08-31T03:52:18+00:00 77s 13 7 81 113 ( 338) 29 ( 8500) 0 0 27 4 54 343 0%
2025-08-31T03:37:36+00:00 94s 16 7 81 278 ( 313) 21 ( 150) 0 0 20 4 54 340 0%
2025-08-31T03:22:24+00:00 83s 14 7 81 120 ( 117) 9 ( 41) 0 0 19 4 55 339 0%
2025-08-31T03:07:18+00:00 77s 12 7 81 148 ( 81) 11 ( 37) 0 0 20 4 54 341 0%
2025-08-31T02:52:17+00:00 76s 13 7 81 116 ( 200) 6 ( 43) 0 0 20 4 54 340 0%
2025-08-31T02:37:32+00:00 92s 15 7 81 278 ( 133) 18 ( 50) 0 0 20 4 54 340 0%
2025-08-31T02:22:22+00:00 82s 13 7 81 121 ( 108) 9 ( 47) 0 0 20 4 54 354 0%
2025-08-31T02:07:21+00:00 80s 14 7 81 210 ( 44) 12 ( 37) 0 0 20 4 54 341 0%
2025-08-31T01:52:20+00:00 79s 13 7 81 106 ( 275) 17 ( 43) 0 0 20 4 54 341 0%
2025-08-31T01:37:30+00:00 90s 14 7 81 242 ( 269) 31 ( 425) 0 0 20 4 54 341 0%
2025-08-31T01:22:27+00:00 86s 15 7 81 132 ( 73) 8 ( 50) 0 0 32 4 54 335 0%
2025-08-31T01:07:20+00:00 79s 13 8 80 155 ( 49) 12 ( 34) 0 0 20 4 54 341 0%
2025-08-31T00:52:20+00:00 80s 14 8 80 111 ( 170) 6 ( 50) 0 0 20 4 54 344 0%
Major LoadDatasets Times Distribution (since last Daily Report):
n = 67, median ~= 90000 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 0
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 0
<= 10 s: 0
<= 20 s: 0
<= 1 min: 0
<= 2 min: 67
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Major LoadDatasets Times Distribution (since startup):
n = 9081, median ~= 90711 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 1
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 2
<= 10 s: 0
<= 20 s: 0
<= 1 min: 3
<= 2 min: 8859
<= 5 min: 181
<= 10 min: 28
<= 20 min: 6
<= 1 hr: 1
> 1 hr: 0
Minor LoadDatasets Times Distribution (since last Daily Report):
n = 2, median ~= 3500 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 0
<= 200 ms: 0
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 2
<= 10 s: 0
<= 20 s: 0
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Minor LoadDatasets Times Distribution (since startup):
n = 323, median ~= 2654 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 1
<= 200 ms: 49
<= 500 ms: 67
<= 1 s: 10
<= 2 s: 2
<= 5 s: 149
<= 10 s: 43
<= 20 s: 2
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since last major LoadDatasets):
n = 3, median ~= 75 ms
0 ms: 0
1 ms: 0
2 ms: 0
<= 5 ms: 0
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 1
<= 100 ms: 1
<= 200 ms: 1
<= 500 ms: 0
<= 1 s: 0
<= 2 s: 0
<= 5 s: 0
<= 10 s: 0
<= 20 s: 0
<= 1 min: 0
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since last Daily Report):
n = 2407, median ~= 750 ms
0 ms: 45
1 ms: 36
2 ms: 4
<= 5 ms: 47
<= 10 ms: 18
<= 20 ms: 15
<= 50 ms: 433
<= 100 ms: 58
<= 200 ms: 61
<= 500 ms: 251
<= 1 s: 471
<= 2 s: 339
<= 5 s: 330
<= 10 s: 211
<= 20 s: 87
<= 1 min: 1
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Failed Time Distribution (since startup):
n = 400508, median ~= 527 ms
0 ms: 29973
1 ms: 14139
2 ms: 526
<= 5 ms: 3232
<= 10 ms: 1670
<= 20 ms: 6093
<= 50 ms: 55472
<= 100 ms: 12151
<= 200 ms: 15303
<= 500 ms: 57888
<= 1 s: 70123
<= 2 s: 57215
<= 5 s: 45837
<= 10 s: 19791
<= 20 s: 7445
<= 1 min: 1388
<= 2 min: 334
<= 5 min: 1882
<= 10 min: 26
<= 20 min: 10
<= 1 hr: 6
> 1 hr: 4
Response Succeeded Time Distribution (since last major LoadDatasets):
n = 24, median ~= 160 ms
0 ms: 0
1 ms: 2
2 ms: 0
<= 5 ms: 3
<= 10 ms: 0
<= 20 ms: 0
<= 50 ms: 0
<= 100 ms: 4
<= 200 ms: 5
<= 500 ms: 6
<= 1 s: 2
<= 2 s: 0
<= 5 s: 0
<= 10 s: 0
<= 20 s: 0
<= 1 min: 2
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Succeeded Time Distribution (since last Daily Report):
n = 13386, median ~= 286 ms
0 ms: 200
1 ms: 521
2 ms: 213
<= 5 ms: 498
<= 10 ms: 76
<= 20 ms: 291
<= 50 ms: 2155
<= 100 ms: 1218
<= 200 ms: 641
<= 500 ms: 3083
<= 1 s: 1168
<= 2 s: 750
<= 5 s: 1766
<= 10 s: 665
<= 20 s: 128
<= 1 min: 13
<= 2 min: 0
<= 5 min: 0
<= 10 min: 0
<= 20 min: 0
<= 1 hr: 0
> 1 hr: 0
Response Succeeded Time Distribution (since startup):
n = 1848651, median ~= 194 ms
0 ms: 57955
1 ms: 77200
2 ms: 38142
<= 5 ms: 82011
<= 10 ms: 26034
<= 20 ms: 87834
<= 50 ms: 316728
<= 100 ms: 112721
<= 200 ms: 133226
<= 500 ms: 448688
<= 1 s: 153239
<= 2 s: 129485
<= 5 s: 118597
<= 10 s: 48060
<= 20 s: 11965
<= 1 min: 5628
<= 2 min: 813
<= 5 min: 143
<= 10 min: 115
<= 20 min: 49
<= 1 hr: 15
> 1 hr: 3
TaskThread Failed Time Distribution (since last Daily Report):
n = 0
TaskThread Failed Time Distribution (since startup):
n = 0
TaskThread Succeeded Time Distribution (since last Daily Report):
n = 0
TaskThread Succeeded Time Distribution (since startup):
n = 0
SgtMap topography nFromCache=166 nNotFromCache=1497
GSHHS: nCached=100 of 100, nCoarse=16565, nSuccesses=5025, nTossed=4322
NationalBoundaries: nCached=100 of 100, nCoarse=2223, nSuccesses=624, nTossed=347
StateBoundaries: nCached=100 of 100, nCoarse=1604, nSuccesses=622, nTossed=279
Rivers: nCached=100 of 100, nCoarse=1740, nSuccesses=623, nTossed=352
bufferedImage isAccelerated=[unknown]
canonical map sizes: 151 + 743 + 270 + 1567 + 1016 + 1470 = 5217
canonicalStringHolder map sizes: 401 + 845 + 1083 + 491 + 208 + 339 = 3367
Number of threads: Tomcat-waiting=19, inotify=4, other=53
(format: #threadNumber Thread[threadName,threadPriority,threadGroup] threadStatus)
#1 Thread[AsyncFileHandlerWriter-1,5,main] WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
#2 Thread[Catalina-utility-1,1,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
java.lang.Thread.run(Thread.java:750)
#3 Thread[Catalina-utility-2,1,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
java.lang.Thread.run(Thread.java:750)
#4 Thread[cluster1-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#5 Thread[cluster1-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#6 Thread[cluster1-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#7 Thread[cluster1-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#8 Thread[cluster1-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#9 Thread[cluster1-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#10 Thread[cluster1-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#11 Thread[cluster1-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#12 Thread[cluster1-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#13 Thread[cluster1-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#14 Thread[cluster1-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#15 Thread[cluster1-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#16 Thread[cluster1-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#17 Thread[cluster2-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#18 Thread[cluster2-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#19 Thread[cluster2-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#20 Thread[cluster2-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#21 Thread[cluster2-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#22 Thread[cluster2-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#23 Thread[cluster2-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#24 Thread[cluster2-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#25 Thread[cluster2-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#26 Thread[cluster2-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#27 Thread[cluster2-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#28 Thread[cluster2-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#29 Thread[cluster3-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#30 Thread[cluster3-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#31 Thread[cluster3-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#32 Thread[cluster3-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#33 Thread[cluster3-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#34 Thread[cluster3-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#35 Thread[cluster3-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#36 Thread[cluster3-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#37 Thread[cluster3-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#38 Thread[cluster3-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#39 Thread[cluster3-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#40 Thread[cluster3-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#41 Thread[cluster3-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)
#42 Thread[FileHandlerLogFilesCleaner-1,5,main] WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
#43 Thread[Finalizer,8,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:188)
#44 Thread[GC Daemon,2,system] TIMED_WAITING daemon
java.lang.Object.wait(Native Method)
sun.misc.GC$Daemon.run(GC.java:117)
#45 Thread[http-nio-8080-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:71)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
java.lang.Thread.run(Thread.java:750)
#46 Thread[http-nio-8080-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:811)
java.lang.Thread.run(Thread.java:750)
#47 Thread[https-jsse-nio-8443-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:71)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
java.lang.Thread.run(Thread.java:750)
#48 Thread[https-jsse-nio-8443-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:811)
java.lang.Thread.run(Thread.java:750)
#49 Thread[Java2D Disposer,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
sun.java2d.Disposer.run(Disposer.java:148)
java.lang.Thread.run(Thread.java:750)
#50 Thread[main,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
org.apache.catalina.core.StandardServer.await(StandardServer.java:524)
org.apache.catalina.startup.Catalina.await(Catalina.java:829)
org.apache.catalina.startup.Catalina.start(Catalina.java:777)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345)
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476)
#51 Thread[Reference Handler,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
java.lang.ref.Reference.tryHandlePending(Reference.java:191)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
#52 Thread[RunLoadDatasets,5,main] TIMED_WAITING daemon
java.io.UnixFileSystem.list(Native Method)
java.io.File.normalizedList(File.java:1153)
java.io.File.list(File.java:1126)
gov.noaa.pfel.erddap.RunLoadDatasets.run(RunLoadDatasets.java:186)
#53 Thread[Signal Dispatcher,9,system] RUNNABLE daemon