Move tests to a separate repository

[seisman]

Originally posted by @joa-quim in #3344 (comment)

I worry about the fact that we are always updating the PS's. That keeps adding to the repo size.

Either we should make a repo just for testings or store the png versions. I know I brought that before and that github is not meant to store binaries but we really don't care about past versions of the test images. So having just the current good version is enough.

---

Originally posted by @PaulWessel in #3344 (comment)

I agree with @joa-quim that we don't care about past PS files at all. What is a good solution for maintaining the current originals. Perhaps a separate github repo is the best solution since it can be integrated with our workflows. We have talked about this before and a problem is that there is no good way to put the current gmt repo on a diet - i.e., remote old PS info.
Maybe discussing just this issue - separating tests from the rest of the repo - is a good topic for upcoming group meeting?

[seisman]

It's easy to move the tests directory to another repository. We can still run all the tests locally or in the CI jobs by cloning the tests repository into a gmt subdirectory.

The big trouble is the tests in the doc/examples and doc/scripts directory. We need the PS files when building the documentation.

[PaulWessel]

Some thoughts here.

Also, a fresh full git clone takes 72 seconds for me and gives me a 1.1 Gb tree, while a --depth 1 clone takes 17 seconds and gives me a 278 Mb tree. This is from home, not at the faster network at UH.

I wonder who we are trying to help here by adding extra work for ourselves. Any user who just wants to build the very latest can do a shallow clone. A developer can do the full dump. I know we have this discussion from time to time, @joa-quim, but a 1-2 Gb footprint is no longer defined as large. Seems to me any scheme to make separate repos for test and then somehow build documentation across repos is extra work for no clear reason.

Perhaps we could just be better at announcing the shallow clone option?

[joa-quim]

Not long ago it was 800 MB, now 1.1 GB. I have several github repos. If everybody starts thinking a coupe GB is not a big deal soon not even the 1 TB are enough in laptops. But let me get again to the png versions. What is the issue with them? Right, git won't save their history but we don't care with it. And tests would run faster because no need to rasterize the orig PS.

[PaulWessel]

Where does it say that git does not version and store history of images? My googling takes me to git-lfs only.

[PaulWessel]

Just to be clear, I agree there is no real need for history here. You can always recreate a figure as it was by checking out an earlier GMT version.

[PaulWessel]

There is also this from Antlassian using the git filter branch. Maybe this is something we should do from time to time to prune the PS files.

[joa-quim]

Yes, that seems the way to go. Also found this post that says storing binary files is a bad idea and mentions the filtering operation too.

[seisman]

git-filter-repo is an alternative better than git filter-branch.

[seisman]

The following command clones the repository with all commits since 2019-10-01 (GMT 6.0.0 was released on 2019-11-01). The repository size is ~360 MB.

git clone --shallow-since=2019-10-01 git@github.com:GenericMappingTools/gmt.git

[PaulWessel]

Here are updated plots of test duration and cumulative impact. First, a histogram of the log10(time) of each test. This forms soft of a normal distribution, meaning most tests take 1-2 seconds:

The next plot shows the cumulative time arranged from the quickest to the slowest tests, with test number versus the relative execution time of the entire test suite. We see that the last 30-40 tests take 20% of the execution time.