Run out of heap space when exporting even simple config

[skibu]

I tried creating a trivial agency with 1 route, 2 stops, and 2 trips. When I tried exporting the GTFS data I get the error shown below. There might be a problem due to my earlier getting an error message when I tried saving a route or it could be that the default config for gtfs-editor simply doesn't allocate enough heap space. If this is simply an issue of allocating enough heap space when running gtfs-editor perhaps the INSTALL.md instructions could explain how to increase heap size.

The error is:

@6loe2j7pd
Internal Server Error (500) for request GET /export/creategtfs?calendarFrom=1429211985864&calendarTo=1431803985865&agencySelect=9b46ae27-57f7-48d0-96b3-8d4c39d1b8e0

Execution exception (In /app/jobs/ProcessGtfsSnapshotExport.java around line 82)
OutOfMemoryError occured : Java heap space

[JWJoubert]

Indeed, you need quite a bit. I'm running it successfully with -Xmx4g ... I think it is excessive, but hey, that solves the problem.

How do you do it? Well my installation on the server is set up as a Ubuntu service, so instead of starting it with path-to-play/play run I simply start it with service gtfs-editor start ... but got an IT colleague to help set that up. And it is in that configuration setup that I'm passing the increased memory argument.

[mvanlaar]

This works for me before running play run wit openjdk 7:

It wil set the Java global parameters

export _JAVA_OPTIONS="-Xms800m -Xmx1500m -XX:PermSize=64m -XX:MaxPermSize=256m"

[cairosubway]

The above Java global parameters worked for me after I increased the RAM in the instance from 1GB to 2GB. Thanks from TransitScreen!

[mpschlickmann]

Hello,

I'm new on the gtfs-editor and having the same issue. Could you explain me better how I increase the RAM?

Thanks!

[JWJoubert]

@mpschlickmann, maybe give us a bit more details about your setup: Windows/Mac/Linux? Running on a server? The solution from @mvanlaar above will work... just type that line Martijn gives (I think the leading '_' should be dropped... just JAVA_OPTIONS ) in the terminal/command line. So,

export JAVA_OPTIONS="-Xms800m -Xmx1500m -XX:PermSize=64m -XX:MaxPermSize=256m"

And as indicated, it only started working for me when using -Xmx4g instead of Martijn's 1500m

[mpschlickmann]

@JWJoubert thanks for the quick answer. I'm using Ubuntu and @mvanlaar solution worked exactly as he posted:

export JAVA_OPTIONS="-Xms800m -Xmx1500m -XX:PermSize=64m -XX:MaxPermSize=256m"

Now it says "RuntimeException occured : java.io.FileNotFoundException: /home/schlickmann/gtfs-editor/public/data/gtfs_2.zip (No such file or directory)" but I'll check on that, thanks!

[mvanlaar]

@mpschlickmann check if the directory data exists and if not create it.

[mpschlickmann]

@mvanlaar thanks! it worked

[medwards]

I still think export needs to be optimized. I tried a VM /w those java options exported at 2, 3 and 5GB of memory assigned with no success. Finally I just gave the VM 10GB of RAM and no special options and it worked. This strikes me as kind of high.

[dcunited001]

I'm trying to import GTFS on dotcloud and I'm running out of memory on imports, regardless of how much memory I allocate. DotCloud with a 512MB max is already too expensive for us, which is $30/month. But i even tried a 1024MB container and even that did not work. That container works out to be about $60/month.

I'm launching with the following config options on Dotcloud

JAVA_OPTS="-Xmx1024m -Xss512k -XX:+UseCompressedOops"

Also, is it possible to run the software with SQL? If that was the case, i could easily bump up the container size during import/export and then turn it down once that's done, saving my client hundreds per year.

It may still be possible to import/export application data on DotCloud by uploading it to disk on the cloud instance. Is there any way to do that?

[sw2279]

I am running ubuntu on digital ocean. I typed the following in command line. I am still getting out of memory error. Am I writing the command line incorrectly?

export JAVA_OPTIONS="-Xms800m -Xmx4g -XX:PermSize=64m -XX:MaxPermSize=256m"

[dcunited001]

@sw2279 most likely, yes, the command is correct. i found that i couldn't import successfully without 1024MB ram!

One option is to comb through the GTFS files you are importing and clean them up.

This arbitrary memory requirement made this otherwise well-engineered software application useless because we couldn't deploy it. And because SQL was removed from the project a year ago, we couldn't simply transfer data from one instance of GTFS-Editor to another. Why a 1MB flat file inflates to require > 1024MB ram will never cease to blow my mind. I haven't gotten a response on these repo's, so I'm not about to make changes and try to submit a pull request.

[cascafico]

Heap space message is totally misleading: I solved only when I manually created public/data folder