Description
I am getting the following error message in my Lambda logs
{
"type": "system",
"errno": "EMFILE",
"code": "EMFILE",
"erroredSysCall": "connect"
}
Looking at everything from a bunch of different angles, I am having a problem with the Nuxt Generator. The code is opening files and websocket connections beyond the max limit set by Lambda. The AWS team is telling me that the code needs to initiate all connections, web socket connections etc outside the Lambda Handler. This is because these connections can be reused for future invokes. Reference Lambda best practices in the body of the AWS support advice below. Please see the reference they sent me.
[2] Best practices for working with AWS Lambda functions - https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
In addition to the connections being commingled with the files, the handler is leaving the files open and not closing them when it is done doing what it needs to do. So I am being told that I should follow best practices today and they are recommending that all connections are made outside the handler and then reused. Connections that can be reused between invokes and should be.
The code should instruct the handler to close each file when it finishes with what it is instructed to do. If you open a file inside of Lambda, you need to be sure that it is closed before you open a new file.
I am having problems with generator and how to make sure that I am sticking to the hard limits set by AWS Lambda policy.
Can someone help me fix the handler code to separate the connections from the files and open and close each of the files as they are being used?
[1] Lambda quotas - Function configuration, deployment, and execution - https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#function-configuration-deployment-and-execution
This is from AWS directly:
To briefly summaries our conversation, you are still receiving the EMFILE issue on your Lambda function. As explained, this is due to 1,024 limit of open files and connections on your Lambda function. We discussed best practices for your Lambda function where you would initiate connections outside of the handler and reuse them as well as closing each file after you open it. This will resolve your issue of reaching the 1,024 limit.
Also on the call I examined your log file and could see that the Lambda function seemed to have some error handling built into the code as it was able to continue executing even with the errors, however it was still receiving an error due to timeouts. To help try to resolve this we increased the timeout to 5 minutes. This timeout may need to be adjusted further depending on the time it takes for you to also close the files, however 5 minutes I would suspect is adequate for your function.
I did note that I had encountered a similar issue recently in a case and I could send you on the resources I had there. There is a useful 3rd party resource that covers both errors (EMFILE and /tmp storage) you have faced and talks about resolving them, it is also in javascript, which is useful as your Lambda is using node.js [3].
Finally, to summarize this case, I am recommending that you are obeying Lambda best practices [2] by closing every file on your Lambda function after it has been read. This will ensure you no longer receive the EMFILE error.