Marine Institute Logo Marine Institute ERDDAP Server
Serving data from the Marine Institute (Ireland)
   
Brought to you by the Marine Institute(Ireland)    

ERDDAP > Status

Current time is 2022-11-27T07:50:35+00:00
Startup was at  2022-11-17T07:16:55+00:00
Last major LoadDatasets started 2m 13s ago and is still running.
nGridDatasets  = 10
nTableDatasets = 58
nTotalDatasets = 68
n Datasets Failed To Load (in the last major LoadDatasets) = 4
    IMINewportBuoys, imiglenamongarms, imifeeaghco2, ireland_marine_research_database_charts, (end)
Unique users (since startup)                            n = 3369
Response Failed    Time (since last major LoadDatasets) n =       46,  median ~=     4455 ms
Response Failed    Time (since last Daily Report)       n =      153,  median ~=     3958 ms
Response Failed    Time (since startup)                 n =    27004,  median ~=      465 ms
Response Succeeded Time (since last major LoadDatasets) n =      202,  median ~=      473 ms
Response Succeeded Time (since last Daily Report)       n =      626,  median ~=      474 ms
Response Succeeded Time (since startup)                 n =   137419,  median ~=       96 ms
TaskThread has finished 0 out of 0 tasks.  Currently, no task is running.
TaskThread Failed    Time (since last Daily Report)     n =        0
TaskThread Failed    Time (since startup)               n =        0
TaskThread Succeeded Time (since last Daily Report)     n =        0
TaskThread Succeeded Time (since startup)               n =        0
Number of threads: Tomcat-waiting=24, inotify=3, other=55
MemoryInUse=   249 MB (highWaterMark=  1799 MB) (Xmx ~= 5888 MB)

Major LoadDatasets Time Series: MLD    Datasets Loaded            Requests (median times in ms)              Number of Threads      MB    Open
  timestamp                    time   nTry nFail nTotal  nSuccess (median) nFail (median) memFail tooMany  tomWait inotify other  inUse  Files
----------------------------  -----   -----------------  ------------------------------------------------  ---------------------  -----  -----
  2022-11-27T07:37:31+00:00    249s     14     4     68       225 (   373)    58 (  3600)       0       0       10       3    54     91     0%
  2022-11-27T07:21:52+00:00    211s     10     4     68       199 (   672)    49 (  3950)       0       0       10       3    54     86     0%
  2022-11-27T07:07:28+00:00    246s     11     4     68       159 (   410)    35 (  6250)       0       0       10       3    54     89     0%
  2022-11-27T06:51:54+00:00    212s     12     4     68       177 (   446)    37 (  5962)       0       0        9       3    55     93     0%
  2022-11-27T06:37:26+00:00    245s     13     4     68       152 (   667)    36 (  6000)       0       0       10       3    54     92     0%
  2022-11-27T06:21:56+00:00    214s     10     4     68       168 (   565)    39 (  5536)       0       0       10       3    54     91     0%
  2022-11-27T06:06:50+00:00    208s     10     4     68       177 (   413)    36 (  6071)       0       0       10       3    54     92     0%
  2022-11-27T05:51:50+00:00    208s     12     4     68       173 (   539)    37 (  5893)       0       0       10       3    54     91     0%
  2022-11-27T05:37:21+00:00    240s     13     4     68       173 (   717)    47 (  4250)       0       0       10       3    54     92     0%
  2022-11-27T05:21:55+00:00    214s     12     4     68       172 (   467)    39 (  5577)       0       0       10       3    54     93     0%
  2022-11-27T05:06:56+00:00    216s     11     4     68       165 (   463)    35 (  6346)       0       0       10       3    54     93     0%
  2022-11-27T04:51:52+00:00    211s     12     4     68       174 (   571)    37 (  5833)       0       0       10       3    54     91     0%
  2022-11-27T04:37:35+00:00    254s     11     4     68       171 (   583)    36 (  6154)       0       0       10       3    54     93     0%
  2022-11-27T04:22:05+00:00    225s     12     4     68       204 (   221)    38 (  5714)       0       0       10       3    54     90     0%
  2022-11-27T04:07:05+00:00    224s     10     4     68       199 (   191)    36 (  6154)       0       0       10       3    54     92     0%
  2022-11-27T03:52:05+00:00    224s     12     4     68       173 (   560)    37 (  5962)       0       0       10       3    54     91     0%
  2022-11-27T03:37:43+00:00    262s     12     4     68       148 (   473)    37 (  6250)       0       0       10       3    54     93     0%
  2022-11-27T03:21:56+00:00    216s      9     4     68        76 (  1250)    24 (  6111)       0       0       10       3    54     92     0%
  2022-11-27T03:12:33+00:00    553s     12     4     68       177 (   776)    38 (  6071)       0       0       10       3    54     86     0%
  2022-11-27T02:57:20+00:00    540s     12     3     69       157 (   688)    33 (  6607)       0       0       10       3    54     90     0%
  2022-11-27T02:43:05+00:00    585s     11     3     69       157 (   844)    40 (  5417)       0       0       10       3    75     86     0%
  2022-11-27T02:27:20+00:00    540s     13     3     69       170 (   763)    36 (  6071)       0       0       10       3    54     89     0%
  2022-11-27T02:12:03+00:00    523s     12     3     69       164 (   889)    44 (  5000)       0       0       10       3    54     82     0%
  2022-11-27T01:57:16+00:00    536s     11     3     69       135 (   438)    27 (  4700)       0       0       10       3    54     91     0%
  2022-11-27T01:43:17+00:00    597s     10     3     69       229 (   958)    51 (  6029)       0       0       30       3    54     77     0%
  2022-11-27T01:24:57+00:00    397s      9     3     69       187 (  1167)    52 (  5526)       0       0       10       3    54     89     0%
  2022-11-27T01:06:51+00:00    212s      8     3     69       116 (   200)    27 (  7188)       0       0       10       3    54     93     0%
  2022-11-27T00:55:16+00:00    416s     13     3     69       185 (   330)    49 (  8393)       0       0       19       3    54     89     0%
  2022-11-27T00:40:30+00:00    430s      9     3     69        86 (  1250)    34 (  8571)       0       0       10       3    54     90     0%
  2022-11-27T00:28:34+00:00    615s      9     3     69       214 (   482)   112 ( 10000)       0       0       35       3    54     81     0%
  2022-11-27T00:06:54+00:00    215s      9     3     69       177 (   194)    47 (  7500)       0       0       10       3    54     93     0%
  2022-11-26T23:52:13+00:00    234s     12     3     69       172 (   575)    36 (  6154)       0       0       10       3    54     91     0%
  2022-11-26T23:37:44+00:00    265s     11     3     69       202 (   806)    52 (  3846)       0       0       10       3    54     92     0%
  2022-11-26T23:21:55+00:00    216s      9     3     69       186 (   688)    50 (  3667)       0       0       10       3    54     91     0%
  2022-11-26T23:06:54+00:00    215s     10     3     69       181 (   568)    46 (  5556)       0       0       10       3    54     93     0%
  2022-11-26T22:52:05+00:00    226s     13     3     69       195 (   335)    36 (  6250)       0       0       10       3    54     93     0%
  2022-11-26T22:37:30+00:00    251s     11     3     69       202 (   529)    47 (  3833)       0       0       10       3    54     93     0%
  2022-11-26T22:21:47+00:00    209s      9     3     69       169 (   768)    43 (  5156)       0       0       24       3    54     89     0%
  2022-11-26T22:06:52+00:00    214s      9     3     69       198 (   429)    58 (  5000)       0       0       15       3    54     93     0%
  2022-11-26T21:51:50+00:00    212s     12     3     69       151 (   488)    36 (  6000)       0       0       10       3    54     91     0%
  2022-11-26T21:37:14+00:00    236s     10     3     69       193 (   486)    44 (  4455)       0       0       10       3    54     93     0%
  2022-11-26T21:21:51+00:00    212s     12     3     69       162 (   938)    50 (  3800)       0       0       10       3    54     87     0%
  2022-11-26T21:06:53+00:00    214s      9     3     69       232 (   750)    53 (  5156)       0       0       10       3    54     93     0%
  2022-11-26T20:51:50+00:00    212s     13     3     69       170 (  1000)    47 (  6406)       0       0       32       3    54     92     0%
  2022-11-26T20:37:19+00:00    241s     10     3     69       170 (   475)    36 (  6250)       0       0       10       3    54     92     0%
  2022-11-26T20:21:45+00:00    207s      9     3     69       194 (   636)    49 (  4550)       0       0       10       3    54     93     0%
  2022-11-26T20:06:48+00:00    210s     10     3     69       168 (   333)    42 (  5000)       0       0       10       3    54     93     0%
  2022-11-26T19:51:48+00:00    210s     11     3     69       173 (   450)    36 (  6071)       0       0        9       3    55     95     0%
  2022-11-26T19:37:26+00:00    249s     11     3     69       181 (   516)    41 (  4833)       0       0       10       3    54     93     0%
  2022-11-26T19:21:45+00:00    207s      9     3     69       147 (   688)    40 (  5714)       0       0       10       3    54     92     0%
  2022-11-26T19:07:03+00:00    226s      9     3     69       204 (   700)    49 (  3950)       0       0       10       3    54     93     0%
  2022-11-26T18:52:07+00:00    229s     13     3     69       186 (   647)    42 (  5357)       0       0       10       3    54     92     0%
  2022-11-26T18:37:21+00:00    244s     10     3     69       189 (   463)    46 (  4625)       0       0       16       3    54     90     0%
  2022-11-26T18:21:50+00:00    213s     10     3     69       234 (  1467)    74 (  4625)       0       0       10       3    54     86     0%
  2022-11-26T18:06:52+00:00    215s      9     3     69       259 (  1607)    80 (  6190)       0      10       48       3    57    238     0%
  2022-11-26T17:51:48+00:00    211s     11     3     69       146 (   200)    26 (  5500)       0       0       10       3    54     92     0%
  2022-11-26T17:38:19+00:00    302s     10     3     69       201 (   589)    48 (  6667)       0       0       35       3    54     82     0%
  2022-11-26T17:21:43+00:00    206s     11     3     69       175 (   455)    39 (  5536)       0       0       10       3    54     91     0%
  2022-11-26T17:06:59+00:00    222s      9     3     69       183 (   297)    35 (  6458)       0       0       10       3    54     92     0%
  2022-11-26T16:51:51+00:00    214s     12     3     69       156 (   650)    42 (  5313)       0       0       17       3    54     85     0%
  2022-11-26T16:37:37+00:00    260s     12     4     68       174 (   329)    36 (  6154)       0       0       10       3    54     91     0%
  2022-11-26T16:21:50+00:00    214s     11     4     68       146 (   480)    44 (  4500)       0       0       10       3    54     92     0%
  2022-11-26T16:07:11+00:00    235s     12     5     67       191 (   285)    42 (  5000)       0       0       11       3    56     83     0%
  2022-11-26T15:52:03+00:00    227s     13     5     67       168 (   667)    37 (  5893)       0       0       10       3    54     92     0%
  2022-11-26T15:37:58+00:00    282s     13     5     67       267 (   203)    42 (  5417)       0       0       10       3    54     90     0%
  2022-11-26T15:21:56+00:00    220s     12     5     67       160 (   405)    38 (  5667)       0       0       10       3    54     90     0%
  2022-11-26T15:07:01+00:00    224s     11     4     68       221 (   883)    69 (  3300)       0       0       10       3    54     91     0%
  2022-11-26T14:51:57+00:00    221s     12     4     68       251 (   443)    59 (  3038)       0       0       10       3    54     91     0%
  2022-11-26T14:37:53+00:00    277s     12     4     68       209 (   491)    39 (  5179)       0       0       10       3    54     92     0%
  2022-11-26T14:22:10+00:00    234s     10     4     68       207 (   516)    40 (  5333)       0       0       10       3    54     94     0%
  2022-11-26T14:07:13+00:00    237s     12     4     68       196 (   431)    47 (  4250)       0       0       10       3    54     92     0%
  2022-11-26T13:52:09+00:00    233s     13     4     68       211 (   350)    48 (  4100)       0       0       10       3    54     92     0%
  2022-11-26T13:37:27+00:00    251s     11     4     68       212 (   194)    39 (  5536)       0       0       10       3    54     92     0%
  2022-11-26T13:21:57+00:00    222s      9     3     69       190 (   383)    44 (  5000)       0       0       10       3    54     91     0%
  2022-11-26T13:07:03+00:00    227s     10     3     69       199 (   370)    41 (  5167)       0       0       10       3    54     92     0%
  2022-11-26T12:52:01+00:00    226s     13     3     69       225 (   732)    58 (  3500)       0       0       10       3    54     90     0%
  2022-11-26T12:37:34+00:00    258s     11     4     68       179 (   268)    36 (  6071)       0       0       10       3    54     75     0%
  2022-11-26T12:21:59+00:00    223s     10     3     69       191 (   803)    50 (  5833)       0       0       13       3    54     92     0%
  2022-11-26T12:07:13+00:00    237s     10     3     69       212 (   226)    36 (  5385)       0       0       10       3    54     91     0%
  2022-11-26T11:52:10+00:00    235s     12     3     69       172 (   500)    40 (  5714)       0       0       10       3    54     95     0%
  2022-11-26T11:37:53+00:00    278s     11     3     69       192 (   330)    37 (  5893)       0       0       10       3    54     92     0%
  2022-11-26T11:22:18+00:00    243s      9     3     69       211 (   389)    43 (  5156)       0       0       10       3    54     89     0%
  2022-11-26T11:07:18+00:00    243s      9     3     69       204 (   223)    37 (  5962)       0       0       10       3    54     90     0%
  2022-11-26T10:52:10+00:00    235s     12     3     69       190 (   200)    35 (  6167)       0       0       10       3    54     92     0%
  2022-11-26T10:37:30+00:00    256s     11     3     69       200 (   550)    48 (  5882)       0       0       15       3    54     92     0%
  2022-11-26T10:21:51+00:00    217s      9     3     69       169 (   488)    43 (  5167)       0       0       10       3    54     92     0%
  2022-11-26T10:06:57+00:00    223s      9     3     69       191 (   318)    36 (  6071)       0       0       18       3    54     92     0%
  2022-11-26T09:52:42+00:00    268s     12     3     69       217 (   691)    65 (  2844)       0       0       10       3    54     92     0%
  2022-11-26T09:38:04+00:00    289s     13     4     68       193 (   303)    44 (  4571)       0       0        3       3    81    130     0%
  2022-11-26T09:22:47+00:00    273s     10     4     68       208 (   806)    63 (  3676)       0       0       14       3    54     90     0%
  2022-11-26T09:07:02+00:00    228s      9     3     69       171 (   177)    35 (  4167)       0       0       13       3    54     87     0%
  2022-11-26T08:53:44+00:00    330s     32     3     69       229 (   732)    44 (  5882)       0       0       31       3    57     85     0%
  2022-11-26T08:37:38+00:00    264s     16     4     68       192 (   417)    39 (  4813)       0       0       15       3    54     92     0%


Major LoadDatasets Times Distribution (since last Daily Report):
    n =        2,  median ~=   210000 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           2
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Major LoadDatasets Times Distribution (since startup):
    n =      962,  median ~=   233473 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           0
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          0
    <= 1 s:             0
    <= 2 s:             0
    <= 5 s:             0
    <= 10 s:            0
    <= 20 s:            0
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:         763
    <= 10 min:        171
    <= 20 min:         28
    <= 1 hr:            0
    >  1 hr:            0


Minor LoadDatasets Times Distribution (since last Daily Report):
    n =        0

Minor LoadDatasets Times Distribution (since startup):
    n =      154,  median ~=     1367 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           9
    <= 100 ms:         15
    <= 200 ms:          9
    <= 500 ms:          2
    <= 1 s:             2
    <= 2 s:           109
    <= 5 s:             4
    <= 10 s:            0
    <= 20 s:            2
    <= 1 min:           0
    <= 2 min:           1
    <= 5 min:           1
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


Response Failed Time Distribution (since last major LoadDatasets):
    n =       46,  median ~=     4455 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           0
    <= 50 ms:           4
    <= 100 ms:          0
    <= 200 ms:          0
    <= 500 ms:          7
    <= 1 s:             1
    <= 2 s:             2
    <= 5 s:            11
    <= 10 s:           14
    <= 20 s:            7
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Failed Time Distribution (since last Daily Report):
    n =      153,  median ~=     3958 ms
    0 ms:               0
    1 ms:               0
    2 ms:               0
    <= 5 ms:            0
    <= 10 ms:           0
    <= 20 ms:           2
    <= 50 ms:          13
    <= 100 ms:          2
    <= 200 ms:          0
    <= 500 ms:         18
    <= 1 s:             9
    <= 2 s:             9
    <= 5 s:            36
    <= 10 s:           42
    <= 20 s:           22
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Failed Time Distribution (since startup):
    n =    27004,  median ~=      465 ms
    0 ms:            5498
    1 ms:            2114
    2 ms:              61
    <= 5 ms:          108
    <= 10 ms:         154
    <= 20 ms:         384
    <= 50 ms:        2974
    <= 100 ms:        627
    <= 200 ms:        272
    <= 500 ms:       1483
    <= 1 s:          1222
    <= 2 s:          1364
    <= 5 s:          4266
    <= 10 s:         4005
    <= 20 s:         2273
    <= 1 min:         133
    <= 2 min:          57
    <= 5 min:           9
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0


Response Succeeded Time Distribution (since last major LoadDatasets):
    n =      202,  median ~=      473 ms
    0 ms:              11
    1 ms:               7
    2 ms:               5
    <= 5 ms:           10
    <= 10 ms:           2
    <= 20 ms:           6
    <= 50 ms:          25
    <= 100 ms:         17
    <= 200 ms:          8
    <= 500 ms:         11
    <= 1 s:            14
    <= 2 s:            12
    <= 5 s:            29
    <= 10 s:           42
    <= 20 s:            3
    <= 1 min:           0
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Succeeded Time Distribution (since last Daily Report):
    n =      626,  median ~=      474 ms
    0 ms:              22
    1 ms:              24
    2 ms:              13
    <= 5 ms:           36
    <= 10 ms:          11
    <= 20 ms:          11
    <= 50 ms:          57
    <= 100 ms:         58
    <= 200 ms:         39
    <= 500 ms:         46
    <= 1 s:            42
    <= 2 s:            46
    <= 5 s:            96
    <= 10 s:          112
    <= 20 s:            9
    <= 1 min:           4
    <= 2 min:           0
    <= 5 min:           0
    <= 10 min:          0
    <= 20 min:          0
    <= 1 hr:            0
    >  1 hr:            0

Response Succeeded Time Distribution (since startup):
    n =   137419,  median ~=       96 ms
    0 ms:           11052
    1 ms:            9777
    2 ms:            4316
    <= 5 ms:         8457
    <= 10 ms:        1233
    <= 20 ms:        2486
    <= 50 ms:       13196
    <= 100 ms:      19638
    <= 200 ms:       9729
    <= 500 ms:      11674
    <= 1 s:         10983
    <= 2 s:         10909
    <= 5 s:         11624
    <= 10 s:         9846
    <= 20 s:         2273
    <= 1 min:         178
    <= 2 min:          42
    <= 5 min:           3
    <= 10 min:          2
    <= 20 min:          0
    <= 1 hr:            1
    >  1 hr:            0


TaskThread Failed Time Distribution (since last Daily Report):
    n =        0

TaskThread Failed Time Distribution (since startup):
    n =        0

TaskThread Succeeded Time Distribution (since last Daily Report):
    n =        0

TaskThread Succeeded Time Distribution (since startup):
    n =        0


SgtMap topography nFromCache=31 nNotFromCache=59
GSHHS: nCached=100 of 100, nCoarse=1196, nSuccesses=443, nTossed=245
NationalBoundaries: nCached=46 of 100,  nCoarse=115, nSuccesses=94, nTossed=0
StateBoundaries: nCached=46 of 100,  nCoarse=99, nSuccesses=94, nTossed=0
Rivers: nCached=46 of 100,  nCoarse=105, nSuccesses=94, nTossed=0
bufferedImage isAccelerated=[unknown]
canonical map sizes: 69 + 686 + 250 + 1540 + 987 + 1422 = 4954
canonicalStringHolder map sizes: 283 + 624 + 718 + 279 + 178 + 282 = 2364

Number of threads: Tomcat-waiting=24, inotify=3, other=55
(format: #threadNumber Thread[threadName,threadPriority,threadGroup] threadStatus)

#1 Thread[AsyncFileHandlerWriter-159413332,5,main] TIMED_WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:159)

#2 Thread[Catalina-utility-1,1,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:750)

#3 Thread[Catalina-utility-2,1,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:750)

#4 Thread[cluster1-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#5 Thread[cluster1-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#6 Thread[cluster1-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#7 Thread[cluster1-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#8 Thread[cluster1-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#9 Thread[cluster1-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#10 Thread[cluster1-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#11 Thread[cluster1-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#12 Thread[cluster1-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#13 Thread[cluster1-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#14 Thread[cluster1-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#15 Thread[cluster1-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#16 Thread[cluster1-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#17 Thread[cluster2-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#18 Thread[cluster2-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#19 Thread[cluster2-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#20 Thread[cluster2-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:286)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#21 Thread[cluster2-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#22 Thread[cluster2-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#23 Thread[cluster2-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#24 Thread[cluster2-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#25 Thread[cluster2-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#26 Thread[cluster2-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#27 Thread[cluster2-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#28 Thread[cluster2-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#29 Thread[cluster2-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#30 Thread[cluster3-connection-reaper-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#31 Thread[cluster3-nio-worker-0,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#32 Thread[cluster3-nio-worker-1,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#33 Thread[cluster3-nio-worker-2,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#34 Thread[cluster3-nio-worker-3,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#35 Thread[cluster3-nio-worker-4,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#36 Thread[cluster3-nio-worker-5,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#37 Thread[cluster3-nio-worker-6,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:192)
io.netty.channel.epoll.Native.epollWait(Native.java:185)
io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#38 Thread[cluster3-nio-worker-7,5,main] RUNNABLE
io.netty.channel.epoll.Native.epollWait0(Native Method)
io.netty.channel.epoll.Native.epollWait(Native.java:176)
io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:281)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:351)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#39 Thread[cluster3-reconnection-0,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#40 Thread[cluster3-reconnection-1,5,main] WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#41 Thread[cluster3-scheduled-task-worker-0,5,main] TIMED_WAITING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#42 Thread[cluster3-timeouter-0,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569)
io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:750)

#43 Thread[FileHandlerLogFilesCleaner-1,5,main] WAITING daemon
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)

#44 Thread[Finalizer,8,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:188)

#45 Thread[GC Daemon,2,system] TIMED_WAITING daemon
java.lang.Object.wait(Native Method)
sun.misc.GC$Daemon.run(GC.java:117)

#46 Thread[http-nio-8080-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:540)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106)
java.lang.Thread.run(Thread.java:750)

#47 Thread[http-nio-8080-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:787)
java.lang.Thread.run(Thread.java:750)

#48 Thread[https-jsse-nio-8443-Acceptor,5,main] RUNNABLE daemon
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421)
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:540)
org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106)
java.lang.Thread.run(Thread.java:750)

#49 Thread[https-jsse-nio-8443-Poller,5,main] RUNNABLE daemon
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:787)
java.lang.Thread.run(Thread.java:750)

#50 Thread[Java2D Disposer,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
sun.java2d.Disposer.run(Disposer.java:148)
java.lang.Thread.run(Thread.java:750)

#51 Thread[LoadDatasets,5,main] RUNNABLE daemon
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
java.net.SocketInputStream.read(SocketInputStream.java:171)
java.net.SocketInputStream.read(SocketInputStream.java:141)
com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2023)
com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6465)
com.microsoft.sqlserver.jdbc.TDSReader.nextPacket(IOBuffer.java:6368)
com.microsoft.sqlserver.jdbc.TDSReader.ensurePayload(IOBuffer.java:6346)
com.microsoft.sqlserver.jdbc.TDSReader.readBytes(IOBuffer.java:6647)
com.microsoft.sqlserver.jdbc.SimpleInputStream.read(SimpleInputStream.java:302)
com.microsoft.sqlserver.jdbc.SimpleInputStream.read(SimpleInputStream.java:264)
com.microsoft.sqlserver.jdbc.SimpleInputStream.getBytes(SimpleInputStream.java:349)
com.microsoft.sqlserver.jdbc.DDC.convertStreamToObject(DDC.java:660)
com.microsoft.sqlserver.jdbc.ServerDTVImpl.getValue(dtv.java:3773)
com.microsoft.sqlserver.jdbc.DTV.getValue(dtv.java:247)
com.microsoft.sqlserver.jdbc.Column.getValue(Column.java:190)
com.microsoft.sqlserver.jdbc.SQLServerResultSet.getValue(SQLServerResultSet.java:2041)
com.microsoft.sqlserver.jdbc.SQLServerResultSet.getValue(SQLServerResultSet.java:2027)
com.microsoft.sqlserver.jdbc.SQLServerResultSet.getString(SQLServerResultSet.java:2489)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.getDataForDapQuery(EDDTableFromDatabase.java:988)
gov.noaa.pfel.erddap.dataset.EDDTable.subsetVariablesDataTable(EDDTable.java:12453)
gov.noaa.pfel.erddap.dataset.EDDTable.distinctSubsetVariablesDataTable(EDDTable.java:12519)
gov.noaa.pfel.erddap.dataset.EDDTable.ensureValid(EDDTable.java:730)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.<init>(EDDTableFromDatabase.java:601)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.lowFromXml(EDDTableFromDatabase.java:248)
gov.noaa.pfel.erddap.dataset.EDDTableFromDatabase.fromXml(EDDTableFromDatabase.java:106)
gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:443)
gov.noaa.pfel.erddap.LoadDatasets.run(LoadDatasets.java:359)

#52 Thread[main,5,main] TIMED_WAITING
java.lang.Thread.sleep(Native Method)
org.apache.catalina.core.StandardServer.await(StandardServer.java:563)
org.apache.catalina.startup.Catalina.await(Catalina.java:864)
org.apache.catalina.startup.Catalina.start(Catalina.java:810)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345)
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476)

#53 Thread[Reference Handler,10,system] WAITING daemon
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
java.lang.ref.Reference.tryHandlePending(Reference.java:191)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

#54 Thread[RunLoadDatasets,5,main] TIMED_WAITING daemon
java.lang.Thread.sleep(Native Method)
gov.noaa.pfel.erddap.RunLoadDatasets.run(RunLoadDatasets.java:288)

#55 Thread[Signal Dispatcher,9,system] RUNNABLE daemon


 
ERDDAP, Version 2.14
Disclaimers | Privacy Policy | Contact