Gatsby DSG Build on Netlify fails due to function size

My gatsby build on netlify fails due to the size of the DSG/SSR functions which are created by netlify-plugin-gatsby. Error produced is Request must be smaller than 69905067 bytes for the CreateFunction operation.

Additionally, during the 'onPostbuild' section of the build, I get the following message:
The function zip ../../../tmp/zisi-6212bb95da69157a8c04d77d/__ssr.zip size is 266 MB, which is larger than the maximum supported size of 52.4 MB.

I'm able to get my build to function locally but when I serve the site, the DSG pages (which get successfully built as specified in build output log) don't work and return 404's.

Build is running @netlify/plugin-gatsby 2.0.2 & gatsby 4.5.3.

I've contacted Netlify support, and they suggested this is to do with the functions from this plugin and not Netlify itself - they're hitting AWS's size limitation.

Any help would be greatly appreciated!

[ascorbic]

Hey! The build logs should include details of what are the largest files in the zip. Can you share the logs?

[nolessafool]

This is related to ticket https://netlify.zendesk.com/agent/tickets/84013 from an Enterprise customer in our helpdesk, where there are things like:

• a link to build logs
• a link to our logging service showing that this happens occasionally to several folks but that several is literally 10 other customers out of millions.
• a mention that the same code deploys on gatsby's cloud a-ok despite the very large file that we seem to create

Thanks in advance for your help, @ascorbic !

[ascorbic]

Thanks, @fool. The file mentioned in the ticket (data.mdb) is the Gatsby data layer database. @scottgimblettnz This is created by Gatsby during the build process, and is required. That is a very large database though. How many pages does your site have? If you run gatsby build --verbose, it should also log the number of created nodes and pages. Which Gatsby source plugins are you using? You may be able to cut this down by restricting the amount of data being loaded, which is something lots of source plugins can do. This would also improve build times.

[ascorbic]

OK, I've realised I have access to your logs so am able to diagnose some of it. You appear to have several 150MB(!) json files in there, which are static query results. If you load the current index page for the site, you will see in the network panel that this is being downloaded for every page. This kind of thing happens when you are making static queries in Gatsby that return extremely large results. Are there static queries that are returning every page in the data store?

@ascorbic Thanks for the response. It appears we do have a static query which returns every page, yes. This particular build has multiple languages available, and in some cases is building a version of a page for each language we have. If my build output is accurate, we're building something like 15,000+ pages (hence the desire to use DSG). I suppose this would explain why the json files are so large!

[trevorblades]

I'm having a similar issue on my site, where I'm seeing mp4 files being bundled in all of my function zip files, causing them all to exceed the maximum function size. Deploy log

I ended up working around this by adding the following to my netlify.toml:

[functions]
  included_files = ["!.cache/page-ssr/routes/static/*.mp4"]

It seems like the entire .cache directory is being bundled with each function. Is that typical?

[ascorbic]

@trevorblades Not the whole .cache directory, but the page-ssr directory is, because it may be needed for SSR

[trevorblades]

I just ran up against this issue again for another site. Functions had been bundled and uploaded without issue for a long time, and suddenly they started failing because the function size was >50 MB. I added a line that excludes .cache/page-ssr/routes/static/** and they started working again.

However, this has me worried that as we continue adding pages and more content to the site, our functions will eventually become too large to upload, and there won't be any other globs that I can reasonably exclude to make them work again.

At that point, we'd be stuck. Reducing the amount of static content (MDX files rendered statically) isn't an option. It seems odd that all of those statically rendered MDX pages would be needed for our SSR pages, but I don't know what files I should or shouldn't exclude from the function bundles.

[joechoi-git]

I ran into this issue for a Gatsby site that uses Contentful as a source.

I fixed it by using a filter to reduce the function size by about 70% to fit under a 50 MB limit.

Inside gatsby-config.js,

{
      resolve: "gatsby-source-contentful",
      options: {
        localeFilter: locale => locale.code === "en-US",
      },
    },

[cesarvarela]

We are having the same issue:

https://app.netlify.com/sites/aiid/deploys/62d1ab99dec9430009a8a6a9

This is the project:

https://github.com/responsible-ai-collaborative/aiid

Looks like bundling the .cache folder will always end up in this issue as soon as the site grows enough. For example, this project is very text heavy, and there is not much that could be removed.

[AustinLeeGordon]

I'm having a similar issue with my __ssr.zip file being too large.

We implemented SSR on the careers section of our site and ran into the same error when we deployed it to Netlify: The function zip ../<path>/__ssr.zip size is 108 MB, which is larger than the maximum supported size of 52.4 MB.

The __ssr.zip file includes all pages and templates in the .cache/page-ssr/routes directory (even the ones that didn't implement SSR), which pushes it over the limit. I added the following settings to our netlify.toml file to exclude all files except the ones with SSR.

// netlify.toml
[functions]
  included_files = ["!.cache/page-ssr/routes/component---src-pages-!(careers)*.js", "!.cache/page-ssr/routes/component---templates-*.js"]

This fixed the size issue and seems to be working in our deploy preview, but I'm nervous about pushing it into production. I'm assuming the pages that aren't using SSR don't need to be in __ssr.zip, but I don't know for sure. I haven't dug deep enough to understand it. I also don't like the idea of modifying the neltify.toml file every time I implement SSR on another area of the site. I can see it causing issues down the road.

If that's the fix, could the plugin filter out files that don't export getServerData? Looks like it would have to modify the included_files to pass to zip-it-and-ship-it, but I'm sure it could be done.

The site I'm referring to is named hutson. Deploy id with the error is 636a73d3975e8d009cad5a66 and the working deploy after changing the included_files in netlify.toml is 63729fed93fdf800097e1e3b. Case number 117675. Thanks!