Fix flaky tests

[decentral1se]

How to help

• Pick a test from the list below 🗒️
• Run the test over and over again until you see a failure (see below for HOWTO) 🏃‍♀️
• If it fails, open up an issue to document your digging (e.g. TestMigrate test flaky #257) 🤓
• Try to submit a fix 😄
• If it doesn't fail after $a_lot_of_runs, let's re-enable it (e.g. Re-enable TestStatusCalls test #256) 🌞

Good Luck 🧑‍🚒 👷‍♀️

Testing the tests

cd graph
LIBRARIAN_WRITEALL=0 go test -v ./ -run TestBadger | tee -a test.log # wait $a_while
cat test.log | grep FAIL # check for failures
cat test.log | grep PASS$ | wc -l # count passes

TODO

• TestStatusCalls Re-enable TestStatusCalls test #256
• TestPublishUnicode test: TestPublishUnicode works again #259
• TestPersistence test: TestPersistence works again #263
• TestMigrate Fix TestMigrate #264
• TestDontStartTwiceOnTheSameRepo test: TestDontStartTwiceOnTheSameRepo is back #265
• TestIndexFixtures TestIndexFixtures test is failing #279 Fix TestIndexFixtures #280 Probably ought to update TestIndexFixtures #281 Update TestIndexFixtures to ssb-fixtures 3.0.2 #283
• TestSpec TestSpec is not really flaky #275 Re-enable TestSpec #276
• TestInvalidFuzzed TestInvalidFuzzed is flaky #284 Fix TestInvalidFuzzed #285 Fix TestInvalidFuzzed (Simplified) #286
• TestLegacyInviteJSAccept Fix TestLegacyInviteJSAccept #295
• TestNames TestNames test flaky #250 Attempting to fix TestNames #255 Fix multiple index concurrency issues #308
• TestAboutNames TestAboutNames is flaky #282 Fix TestAboutNames #309
• TestSubsetQueryPlanExecution TestSubsetQueryPlanExecution is flaky #277 Fix TestSubsetQueryPlanExecution #310
• TestSignMessages TestSignMessages is flaky #289 Fix multiple index concurrency issues #308
• TestBadger TestBadger still flaking #266 Fix TestBadger #313
• TestStartup TestStartup is flaky #261 Fix TestStartup #315
• TestMetafeedIndexes TestMetafeedIndexes is flaky #299 Fix TestMetafeedIndexes #316
• TestPeople TestPeople is really, really broken #274 Fix TestPeople #322
• TestLegacyInviteJSCreate TestLegacyInviteJSCreate failed #249
• TestReplicate TestReplicate is flaky #260
• TestNullFeed TestNullFeed is flaky #268 Attempting to fix TestNullFeed #269
• TestBlocking TestBlocking is flaky #273
• TestFSCK TestFSCK flaking #296
• TestNullFetched TestNullFetched is flaky #298
• TestPrivateGroupsManualDecrypt TestPrivateGroupsManualDecrypt is flaky #300
• TestPrivMsgsFromGo TestPrivMsgsFromGo failing #328

[KyleMaas]

List of skipped tests currently (searching for SkipOnCI):

client/client_test.go-func TestStatusCalls(t *testing.T)

cmd/gossb-migrate-mf/migrate/migrate_test.go-func TestMigrate(t *testing.T) {

cmd/go-sbot/crashrecovery_test.go-func TestDontStartTwiceOnTheSameRepo(t *testing.T) {

graph/builder_test.go-func TestBadger(t *testing.T) {

graph/people_test.go-func TestPeople(t *testing.T) {
(This one's a little tricky, because it doesn't have a skip itself but it calls the run() function which has a skip)

message/legacy/verify_invalid_test.go-func TestInvalidFuzzed(t *testing.T) {

message/publish_test.go-func TestSignMessages(t *testing.T) {

Looks like a lot of this file:
multilogs/indexspeed_test.go:	if testutils.SkipOnCI(t) {

private/box2/spec_test.go-func TestSpec(t *testing.T) {

sbot/names_test.go-func TestNames(t *testing.T) {

sbot/persistence_test.go-func TestPersistence(t *testing.T) {

sbot/metafeed_test.go-func TestMetafeedIndexes(t *testing.T) {

sbot/unipub_test.go-func TestPublishUnicode(t *testing.T) {

sbot/groups_test.go-func TestPrivateGroupsManualDecrypt(t *testing.T) {

sbot/null_feed_test.go-func TestNullFeed(t *testing.T) 

sbot/null_feed_test.go-func TestNullFetched(t *testing.T) {

sbot/fsck_test.go-func TestFSCK(t *testing.T) {

tests/invite_legacy_test.go-func TestLegacyInviteJSCreate(t *testing.T) {

tests/invite_legacy_test.go-func TestLegacyInviteJSAccept(t *testing.T) {

tests/blocking_test.go-func TestBlocking(t *testing.T) {