![]() |
Marine Institute ERDDAP Server
Serving data from the Marine Institute (Ireland) |
Brought to you by the Marine Institute(Ireland) |
Current time is 2023-02-08T22:38:32+00:00 Startup was at 2023-02-08T07:15:55+00:00 Last major LoadDatasets started 7m 29s ago and finished after 280 seconds. nGridDatasets = 10 nTableDatasets = 59 nTotalDatasets = 69 n Datasets Failed To Load (in the last major LoadDatasets) = 3 IMINewportBuoys, imifeeaghco2, ireland_marine_research_database_charts, (end) Unique users (since startup) n = 426 Response Failed Time (since last major LoadDatasets) n = 0 Response Failed Time (since last Daily Report) n = 810, median ~= 1348 ms Response Failed Time (since startup) n = 811, median ~= 1345 ms Response Succeeded Time (since last major LoadDatasets) n = 15, median ~= 28 ms Response Succeeded Time (since last Daily Report) n = 9175, median ~= 94 ms Response Succeeded Time (since startup) n = 9236, median ~= 95 ms TaskThread has finished 0 out of 0 tasks. Currently, no task is running. TaskThread Failed Time (since last Daily Report) n = 0 TaskThread Failed Time (since startup) n = 0 TaskThread Succeeded Time (since last Daily Report) n = 0 TaskThread Succeeded Time (since startup) n = 0 Number of threads: Tomcat-waiting=9, inotify=3, other=36 MemoryInUse= 762 MB (highWaterMark= 1714 MB) (Xmx ~= 5888 MB) Major LoadDatasets Time Series: MLD Datasets Loaded Requests (median times in ms) Number of Threads MB Open timestamp time nTry nFail nTotal nSuccess (median) nFail (median) memFail tooMany tomWait inotify other inUse Files ---------------------------- ----- ----------------- ------------------------------------------------ --------------------- ----- ----- 2023-02-08T22:35:42+00:00 278s 10 3 69 100 ( 76) 4 ( 10) 0 0 10 3 36 81 0% 2023-02-08T22:19:44+00:00 221s 11 3 69 117 ( 90) 4 ( 100) 0 0 10 3 36 82 0% 2023-02-08T22:04:47+00:00 224s 9 3 69 123 ( 81) 21 ( 3125) 0 0 10 3 36 83 0% 2023-02-08T21:49:57+00:00 234s 10 3 69 135 ( 82) 6 ( 500) 0 0 10 3 36 81 0% 2023-02-08T21:35:14+00:00 250s 10 3 69 132 ( 96) 6 ( 750) 0 0 14 3 36 82 0% 2023-02-08T21:19:42+00:00 219s 11 3 69 245 ( 784) 28 ( 1500) 0 0 13 3 36 69 0% 2023-02-08T21:04:42+00:00 219s 9 3 69 114 ( 71) 1 ( 35) 0 0 10 3 36 82 0% 2023-02-08T20:49:42+00:00 219s 10 3 69 110 ( 156) 6 ( 1500) 0 0 10 3 36 82 0% 2023-02-08T20:35:12+00:00 250s 10 3 69 147 ( 115) 23 ( 1786) 0 0 10 3 36 80 0% 2023-02-08T20:19:35+00:00 213s 11 3 69 147 ( 99) 14 ( 1500) 0 0 10 3 36 80 0% 2023-02-08T20:04:57+00:00 235s 9 3 69 94 ( 74) 5 ( 625) 0 0 9 3 37 205 0% 2023-02-08T19:49:39+00:00 216s 12 3 69 113 ( 69) 0 ( 0) 0 0 10 3 36 82 0% 2023-02-08T19:35:10+00:00 247s 10 3 69 110 ( 65) 1 ( 15) 0 0 10 3 36 82 0% 2023-02-08T19:19:37+00:00 215s 11 3 69 147 ( 82) 13 ( 625) 0 0 10 3 36 82 0% 2023-02-08T19:04:42+00:00 220s 9 3 69 132 ( 71) 12 ( 1000) 0 0 9 3 37 92 0% 2023-02-08T18:49:40+00:00 218s 10 3 69 86 ( 66) 0 ( 0) 0 0 10 3 36 82 0% 2023-02-08T18:35:10+00:00 248s 10 3 69 195 ( 182) 16 ( 1000) 0 0 10 3 36 82 0% 2023-02-08T18:19:35+00:00 213s 9 3 69 115 ( 74) 3 ( 28) 0 0 10 3 36 80 0% 2023-02-08T18:04:43+00:00 221s 15 3 69 134 ( 95) 17 ( 1357) 0 0 15 3 36 82 0% 2023-02-08T17:49:37+00:00 215s 10 3 69 85 ( 77) 2 ( 100) 0 0 10 3 36 80 0% 2023-02-08T17:35:37+00:00 275s 10 3 69 158 ( 130) 28 ( 1429) 0 0 8 3 38 248 0% 2023-02-08T17:19:30+00:00 209s 9 3 69 131 ( 92) 2 ( 35) 0 0 10 3 36 80 0% 2023-02-08T17:04:39+00:00 217s 11 3 69 130 ( 70) 4 ( 1000) 0 0 10 3 36 80 0% 2023-02-08T16:49:32+00:00 211s 10 3 69 214 ( 440) 35 ( 1864) 0 0 10 3 36 81 0% 2023-02-08T16:35:07+00:00 246s 10 3 69 153 ( 51) 6 ( 1) 0 0 10 3 36 68 0% 2023-02-08T16:19:34+00:00 212s 9 3 69 152 ( 78) 14 ( 400) 0 0 10 3 36 81 0% 2023-02-08T16:04:34+00:00 213s 9 3 69 177 ( 86) 23 ( 1583) 0 0 10 3 36 83 0% 2023-02-08T15:49:38+00:00 217s 13 3 69 148 ( 94) 12 ( 5000) 0 0 10 3 36 81 0% 2023-02-08T15:35:06+00:00 245s 10 3 69 200 ( 45) 20 ( 350) 0 0 10 3 36 81 0% 2023-02-08T15:19:30+00:00 209s 9 3 69 206 ( 128) 11 ( 1583) 0 0 13 3 36 80 0% 2023-02-08T15:04:31+00:00 210s 9 3 69 123 ( 68) 4 ( 500) 0 0 10 3 36 80 0% 2023-02-08T14:49:35+00:00 214s 13 3 69 173 ( 169) 30 ( 1167) 0 0 10 3 36 82 0% 2023-02-08T14:35:09+00:00 248s 10 3 69 180 ( 111) 18 ( 1000) 0 0 4 3 49 354 0% 2023-02-08T14:19:40+00:00 219s 10 3 69 148 ( 98) 13 ( 4500) 0 0 10 3 36 82 0% 2023-02-08T14:04:35+00:00 215s 9 3 69 164 ( 98) 23 ( 1929) 0 0 16 3 36 80 0% 2023-02-08T13:49:39+00:00 219s 14 3 69 189 ( 221) 23 ( 1875) 0 0 24 3 36 77 0% 2023-02-08T13:34:59+00:00 239s 11 4 68 166 ( 90) 8 ( 3500) 0 0 10 3 36 79 0% 2023-02-08T13:19:34+00:00 214s 10 4 68 112 ( 94) 5 ( 275) 0 0 10 3 36 81 0% 2023-02-08T13:04:32+00:00 212s 10 4 68 147 ( 88) 23 ( 1100) 0 0 10 3 36 80 0% 2023-02-08T12:49:29+00:00 209s 11 4 68 136 ( 91) 12 ( 500) 0 0 10 3 36 82 0% 2023-02-08T12:35:07+00:00 247s 13 4 68 161 ( 99) 24 ( 3000) 0 0 10 3 36 81 0% 2023-02-08T12:19:44+00:00 224s 10 4 68 205 ( 91) 28 ( 2600) 0 0 10 3 36 79 0% 2023-02-08T12:04:46+00:00 226s 10 4 68 167 ( 111) 8 ( 750) 0 0 11 3 36 73 0% 2023-02-08T11:49:41+00:00 222s 11 4 68 151 ( 156) 19 ( 1833) 0 0 10 3 36 80 0% 2023-02-08T11:35:24+00:00 264s 12 4 68 150 ( 97) 1 ( 15) 0 0 10 3 37 78 0% 2023-02-08T11:19:48+00:00 229s 12 4 68 167 ( 465) 16 ( 3500) 0 0 10 3 36 80 0% 2023-02-08T11:05:13+00:00 254s 11 4 68 242 ( 170) 58 ( 1444) 0 0 10 3 36 78 0% 2023-02-08T10:49:34+00:00 215s 12 5 67 238 ( 268) 31 ( 1250) 0 0 35 3 36 77 0% 2023-02-08T10:35:15+00:00 256s 12 5 67 138 ( 65) 12 ( 1500) 0 0 10 3 36 76 0% 2023-02-08T10:19:35+00:00 216s 13 5 67 234 ( 95) 14 ( 350) 0 0 10 3 36 79 0% 2023-02-08T10:04:36+00:00 217s 11 5 67 210 ( 167) 30 ( 1000) 0 0 14 3 36 78 0% 2023-02-08T09:49:55+00:00 236s 12 5 67 155 ( 37) 4 ( 50) 0 0 9 3 37 116 0% 2023-02-08T09:35:42+00:00 283s 11 5 67 130 ( 42) 0 ( 0) 0 0 10 3 36 77 0% 2023-02-08T09:22:40+00:00 401s 11 5 67 153 ( 153) 6 ( 200) 0 0 10 3 36 75 0% 2023-02-08T09:06:32+00:00 334s 14 5 67 141 ( 55) 0 ( 0) 0 0 10 3 36 71 0% 2023-02-08T08:49:39+00:00 220s 12 5 67 122 ( 65) 1 ( 15) 0 0 10 3 36 76 0% 2023-02-08T08:35:21+00:00 263s 14 6 66 137 ( 56) 11 ( 1375) 0 0 10 3 36 77 0% 2023-02-08T08:19:37+00:00 218s 12 6 66 152 ( 325) 25 ( 1083) 0 0 10 3 36 73 0% 2023-02-08T08:04:49+00:00 231s 14 6 66 121 ( 99) 13 ( 1500) 0 0 14 3 36 74 0% 2023-02-08T07:49:45+00:00 227s 16 6 66 129 ( 91) 12 ( 500) 0 0 10 3 36 75 0% 2023-02-08T07:34:42+00:00 224s 20 9 63 68 ( 38) 1 ( 350) 0 0 10 3 36 74 0% 2023-02-08T07:25:21+00:00 562s 71 15 57 61 ( 450) 1 ( 35) 0 0 10 3 39 73 0% Major LoadDatasets Times Distribution (since last Daily Report): n = 61, median ~= 213051 ms 0 ms: 0 1 ms: 0 2 ms: 0 <= 5 ms: 0 <= 10 ms: 0 <= 20 ms: 0 <= 50 ms: 0 <= 100 ms: 0 <= 200 ms: 0 <= 500 ms: 0 <= 1 s: 0 <= 2 s: 0 <= 5 s: 0 <= 10 s: 0 <= 20 s: 0 <= 1 min: 0 <= 2 min: 0 <= 5 min: 59 <= 10 min: 2 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Major LoadDatasets Times Distribution (since startup): n = 62, median ~= 214576 ms 0 ms: 0 1 ms: 0 2 ms: 0 <= 5 ms: 0 <= 10 ms: 0 <= 20 ms: 0 <= 50 ms: 0 <= 100 ms: 0 <= 200 ms: 0 <= 500 ms: 0 <= 1 s: 0 <= 2 s: 0 <= 5 s: 0 <= 10 s: 0 <= 20 s: 0 <= 1 min: 0 <= 2 min: 0 <= 5 min: 59 <= 10 min: 3 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Minor LoadDatasets Times Distribution (since last Daily Report): n = 32, median ~= 1467 ms 0 ms: 0 1 ms: 0 2 ms: 0 <= 5 ms: 0 <= 10 ms: 0 <= 20 ms: 0 <= 50 ms: 1 <= 100 ms: 0 <= 200 ms: 1 <= 500 ms: 0 <= 1 s: 0 <= 2 s: 30 <= 5 s: 0 <= 10 s: 0 <= 20 s: 0 <= 1 min: 0 <= 2 min: 0 <= 5 min: 0 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Minor LoadDatasets Times Distribution (since startup): n = 32, median ~= 1467 ms 0 ms: 0 1 ms: 0 2 ms: 0 <= 5 ms: 0 <= 10 ms: 0 <= 20 ms: 0 <= 50 ms: 1 <= 100 ms: 0 <= 200 ms: 1 <= 500 ms: 0 <= 1 s: 0 <= 2 s: 30 <= 5 s: 0 <= 10 s: 0 <= 20 s: 0 <= 1 min: 0 <= 2 min: 0 <= 5 min: 0 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Response Failed Time Distribution (since last major LoadDatasets): n = 0 Response Failed Time Distribution (since last Daily Report): n = 810, median ~= 1348 ms 0 ms: 9 1 ms: 9 2 ms: 3 <= 5 ms: 1 <= 10 ms: 5 <= 20 ms: 25 <= 50 ms: 72 <= 100 ms: 41 <= 200 ms: 21 <= 500 ms: 102 <= 1 s: 62 <= 2 s: 158 <= 5 s: 218 <= 10 s: 61 <= 20 s: 16 <= 1 min: 1 <= 2 min: 5 <= 5 min: 1 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Response Failed Time Distribution (since startup): n = 811, median ~= 1345 ms 0 ms: 9 1 ms: 9 2 ms: 3 <= 5 ms: 1 <= 10 ms: 5 <= 20 ms: 25 <= 50 ms: 73 <= 100 ms: 41 <= 200 ms: 21 <= 500 ms: 102 <= 1 s: 62 <= 2 s: 158 <= 5 s: 218 <= 10 s: 61 <= 20 s: 16 <= 1 min: 1 <= 2 min: 5 <= 5 min: 1 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Response Succeeded Time Distribution (since last major LoadDatasets): n = 15, median ~= 28 ms 0 ms: 1 1 ms: 2 2 ms: 0 <= 5 ms: 2 <= 10 ms: 0 <= 20 ms: 2 <= 50 ms: 2 <= 100 ms: 3 <= 200 ms: 0 <= 500 ms: 3 <= 1 s: 0 <= 2 s: 0 <= 5 s: 0 <= 10 s: 0 <= 20 s: 0 <= 1 min: 0 <= 2 min: 0 <= 5 min: 0 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Response Succeeded Time Distribution (since last Daily Report): n = 9175, median ~= 94 ms 0 ms: 466 1 ms: 575 2 ms: 336 <= 5 ms: 601 <= 10 ms: 69 <= 20 ms: 177 <= 50 ms: 1156 <= 100 ms: 1357 <= 200 ms: 672 <= 500 ms: 892 <= 1 s: 607 <= 2 s: 1015 <= 5 s: 1016 <= 10 s: 167 <= 20 s: 59 <= 1 min: 7 <= 2 min: 3 <= 5 min: 0 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 Response Succeeded Time Distribution (since startup): n = 9236, median ~= 95 ms 0 ms: 467 1 ms: 578 2 ms: 338 <= 5 ms: 605 <= 10 ms: 71 <= 20 ms: 180 <= 50 ms: 1156 <= 100 ms: 1364 <= 200 ms: 678 <= 500 ms: 895 <= 1 s: 610 <= 2 s: 1035 <= 5 s: 1023 <= 10 s: 167 <= 20 s: 59 <= 1 min: 7 <= 2 min: 3 <= 5 min: 0 <= 10 min: 0 <= 20 min: 0 <= 1 hr: 0 > 1 hr: 0 TaskThread Failed Time Distribution (since last Daily Report): n = 0 TaskThread Failed Time Distribution (since startup): n = 0 TaskThread Succeeded Time Distribution (since last Daily Report): n = 0 TaskThread Succeeded Time Distribution (since startup): n = 0 SgtMap topography nFromCache=0 nNotFromCache=2 GSHHS: nCached=6 of 100, nCoarse=36, nSuccesses=12, nTossed=0 NationalBoundaries: nCached=3 of 100, nCoarse=6, nSuccesses=3, nTossed=0 StateBoundaries: nCached=3 of 100, nCoarse=6, nSuccesses=3, nTossed=0 Rivers: nCached=3 of 100, nCoarse=6, nSuccesses=3, nTossed=0 bufferedImage isAccelerated=[unknown] canonical map sizes: 27 + 594 + 256 + 1544 + 990 + 1429 = 4840 canonicalStringHolder map sizes: 344 + 644 + 749 + 347 + 203 + 350 = 2637 Number of threads: Tomcat-waiting=9, inotify=3, other=36 (format: #threadNumber Thread[threadName,threadPriority,threadGroup] threadStatus) #1 Thread[AsyncFileHandlerWriter-159413332,5,main] TIMED_WAITING daemon sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:159) #2 Thread[Catalina-utility-1,1,main] WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:750) #3 Thread[Catalina-utility-2,1,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:750) #4 Thread[cluster1-connection-reaper-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #5 Thread[cluster1-nio-worker-0,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #6 Thread[cluster1-nio-worker-1,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #7 Thread[cluster1-nio-worker-2,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #8 Thread[cluster1-nio-worker-3,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #9 Thread[cluster1-scheduled-task-worker-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #10 Thread[cluster1-timeouter-0,5,main] TIMED_WAITING java.lang.Thread.sleep(Native Method) io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569) io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #11 Thread[cluster2-connection-reaper-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #12 Thread[cluster2-nio-worker-0,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #13 Thread[cluster2-nio-worker-1,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #14 Thread[cluster2-nio-worker-2,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #15 Thread[cluster2-nio-worker-3,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #16 Thread[cluster2-scheduled-task-worker-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #17 Thread[cluster2-timeouter-0,5,main] TIMED_WAITING java.lang.Thread.sleep(Native Method) io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569) io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #18 Thread[cluster3-connection-reaper-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #19 Thread[cluster3-nio-worker-0,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #20 Thread[cluster3-nio-worker-1,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #21 Thread[cluster3-nio-worker-2,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #22 Thread[cluster3-nio-worker-3,5,main] RUNNABLE io.netty.channel.epoll.Native.epollWait(Native Method) io.netty.channel.epoll.Native.epollWait(Native.java:192) io.netty.channel.epoll.Native.epollWait(Native.java:185) io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:290) io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:347) io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #23 Thread[cluster3-scheduled-task-worker-0,5,main] TIMED_WAITING sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #24 Thread[cluster3-timeouter-0,5,main] TIMED_WAITING java.lang.Thread.sleep(Native Method) io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:569) io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:465) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) #25 Thread[FileHandlerLogFilesCleaner-1,5,main] WAITING daemon sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) #26 Thread[Finalizer,8,system] WAITING daemon java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:188) #27 Thread[GC Daemon,2,system] TIMED_WAITING daemon java.lang.Object.wait(Native Method) sun.misc.GC$Daemon.run(GC.java:117) #28 Thread[http-nio-8080-Acceptor,5,main] RUNNABLE daemon sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:540) org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78) org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106) java.lang.Thread.run(Thread.java:750) #29 Thread[http-nio-8080-Poller,5,main] RUNNABLE daemon sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:787) java.lang.Thread.run(Thread.java:750) #30 Thread[https-jsse-nio-8443-Acceptor,5,main] RUNNABLE daemon sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:540) org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78) org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106) java.lang.Thread.run(Thread.java:750) #31 Thread[https-jsse-nio-8443-Poller,5,main] RUNNABLE daemon sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:787) java.lang.Thread.run(Thread.java:750) #32 Thread[Java2D Disposer,10,system] WAITING daemon java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) sun.java2d.Disposer.run(Disposer.java:148) java.lang.Thread.run(Thread.java:750) #33 Thread[main,5,main] TIMED_WAITING java.lang.Thread.sleep(Native Method) org.apache.catalina.core.StandardServer.await(StandardServer.java:563) org.apache.catalina.startup.Catalina.await(Catalina.java:864) org.apache.catalina.startup.Catalina.start(Catalina.java:810) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:345) org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:476) #34 Thread[Reference Handler,10,system] WAITING daemon java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) java.lang.ref.Reference.tryHandlePending(Reference.java:191) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) #35 Thread[RunLoadDatasets,5,main] TIMED_WAITING daemon java.lang.Thread.sleep(Native Method) gov.noaa.pfel.erddap.RunLoadDatasets.run(RunLoadDatasets.java:288) #36 Thread[Signal Dispatcher,9,system] RUNNABLE daemon