Search before asking
Apache SkyWalking Component
NodeJS Server Side Agent (apache/skywalking-nodejs)
What happened
Due to the temporary closure of the Skywalking server, the Node.js client is unable to connect to the corresponding service normally.
At this time, it will cause the "connect ECONNREFUSED error stack object" to be continuously held, not released, and not collected by GC, ultimately leading to system OOM.
Below is the heapdump I captured from the Node.js application:
| WritableState×16 |
11 |
3.8 kB0 % |
271,826 kB83 % |
|
| WritableState@460177 |
11 |
0.2 kB0 % |
231,086 kB71 % |
|
| bufferedRequest::{chunk, encoding, isBuf, callback, next}@310851 |
12 |
0.1 kB0 % |
231,084 kB71 % |
|
| next::{chunk, encoding, isBuf, callback, next}@308967 |
13 |
0.1 kB0 % |
231,082 kB71 % |
|
| chunk::{file, details, metadata, level, message, , , stack}@308941 |
13 |
0.1 kB0 % |
2.1 kB0 % |
|
| stack::"Error: 14 UNAVAILABLE: No connection established. Last error: Error: connect ECONNREFUSED 1.2.3.4:11800. Resolution note: at callErrorFromStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/call.js:32:19) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client.js:274:76) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:367:141) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:327:181) at /xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/resolving-call.js:135:78 at processTicksAndRejections (node:internal/process/task_queues:78:11) for call at at ServiceClientImpl.makeClientStreamRequest (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client.js:241:32) at ServiceClientImpl.collect (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at TraceReportClient.reportFunction (/xxxx/node_modules/.pnpm/skywalking-backend-js@0.8.0/node_modules/skywalking-backend-js/lib/agent/protocol/grpc/clients/TraceReportClient.js:58:46) at listOnTimeout (node:internal/timers:559:17) at processTimers (node:internal/timers:502:7)"@202057 |
14 |
0.0 kB0 % |
1.6 kB0 % |
|
| first::"Error: 14 UNAVAILABLE: No connection established. Last error: Error: connect ECONNREFUSED 1.2.3.4:11800. Resolution note: at callErrorFromStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/call.js:32:19) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client.js:274:76) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:367:141) at Object.onReceiveStatus (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:327:181) at /xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/resolving-call.js:135:78 at processTicksAndRejections (node:internal/process/task_queues:78:11) for call at "@307315 |
15 |
0.0 kB0 % |
1.0 kB0 % |
|
| second::" at ServiceClientImpl.makeClientStreamRequest (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/client.js:241:32) at ServiceClientImpl.collect (/xxxx/node_modules/.pnpm/@grpc+grpc-js@1.14.3/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at TraceReportClient.reportFunction (/xxxx/node_modules/.pnpm/skywalking-backend-js@0.8.0/node_modules/skywalking-backend-js/lib/agent/protocol/grpc/clients/TraceReportClient.js:58:46) at listOnTimeout (node:internal/timers:559:17) at processTimers (node:internal/timers:502:7)"@307497 |
15 |
0.6 kB0 % |
0.6 kB0 % |
|
| map::system / Map (ConsOneByteString)@213 |
5 |
0.1 kB0 % |
0.1 kB0 % |
|
| metadata::Metadata@201581 |
13 |
0.0 kB0 % |
0.5 kB0 % |
|
| map::system / Map@197299 |
12 |
0.1 kB0 % |
0.2 kB0 % |
|
| file::"/xxxx/node_modules/.pnpm/skywalking-backend-js@0.8.0/node_modules/skywalking-backend-js/lib/agent/protocol/grpc/clients/TraceReportClient.js"@27437 |
8 |
0.2 kB0 % |
0.2 kB0 % |
|
| ::Array@307493 |
14 |
0.1 kB0 % |
0.1 kB0 % |
|
| proto::{constructor, defineGetter, defineSetter, …, get proto, set proto, toLocaleString}@128651 |
3 |
0.1 kB0 % |
0.1 kB0 % |
|
| details::"No connection established. Last error: Error: connect ECONNREFUSED 1.2.3.4:11800. Resolution note: "@201579 |
13 |
0.1 kB0 % |
0.1 kB0 % |
|
| message::"Failed to report trace data 14 UNAVAILABLE: No connection established. Last error: Error: connect ECONNREFUSED 1.2.3.4:11800. Resolution note: "@308493 |
14 |
0.0 kB0 % |
0.1 kB0 % |
|
| ::"error"@8291 |
4 |
0.0 kB0 % |
0.0 kB0 % |
|
| level::"error"@8291 |
4 |
0.0 kB0 % |
0.0 kB0 % |
|
| properties::system / PropertyArray@308931 |
|
|
|
|
What you expected to happen
Do not keep resetting the link when the SkyWalking server is unavailable.
How to reproduce
SkyWalking server is unavailable
Anything else
No response
Are you willing to submit a pull request to fix on your own?
Code of Conduct
Search before asking
Apache SkyWalking Component
NodeJS Server Side Agent (apache/skywalking-nodejs)
What happened
Due to the temporary closure of the Skywalking server, the Node.js client is unable to connect to the corresponding service normally.
At this time, it will cause the "connect ECONNREFUSED error stack object" to be continuously held, not released, and not collected by GC, ultimately leading to system OOM.
Below is the heapdump I captured from the Node.js application:
What you expected to happen
Do not keep resetting the link when the SkyWalking server is unavailable.
How to reproduce
SkyWalking server is unavailable
Anything else
No response
Are you willing to submit a pull request to fix on your own?
Code of Conduct