[s3inbox] Send remove message on overwrite

[MalinAhlberg]

Related issue(s) and PR(s)

This PR closes #1480 .

Description

When uploading a file to s3inbox, a check is done to see if the file is already there. If it is, a delete message is sent before the upload message, so that the FEGA portal will know that the new version of the file exists.

Note: The remove message is formatted according to this spec. However, our messages will have two extra fields, "encrypted_checksums":null and "filesize":0, since it shares the go struct with upload messages. Hopefully, that won't be any problem. Please let me know if you think it will.

How to test

Run the integration tests, and check that after the tests in .github/integration/tests/sda/10_upload_test.sh are run, there are 6 messages for upload and one for remove in rabbitmq messages in the inbox queue.

You can also manually upload a file twice, and check the queue and the logs to make sure the remove message is only sent the second time:
s3cmd -c /tmp/shared/s3cfg put dummyfile.c4gh s3://test_dummy.org/

[jbygdell]

Note: The remove message is formatted according to this spec. However, our messages will have two extra fields, "encrypted_checksums":null and "filesize":0, since it shares the go struct with upload messages. Hopefully, that won't be any problem. Please let me know if you think it will.

We have the schema for the inbox-remove event here. As well in internal/schemas´ as InboxRemove`

[jbygdell]

We have all the JSON schemas available so validation of the outgoing message can be done.

We should also not only check for the existence of the file but also handle any errors in a way so that the user gets the proper notification.

[jbygdell]

The checkFileExists should have a static test so we can validate the two expected results.

[MalinAhlberg]

After a lot of rebasing, bug fixing and improved test setup (mainly from @jbygdell) this is now ready for review again!
Sorry for the messy history here on this comment page. The commits are less messy!

[MalinAhlberg]

I have fixed the things you comment on @jbygdell . Please have a look and resolve the comments.

Personally, I think our linters should be responsible for things like newlines. They should complain on everything we want to enforce. If they do not enforce a standard, it should be optional :) .

[jbygdell]

Personally, I think our linters should be responsible for things like newlines. They should complain on everything we want to enforce. If they do not enforce a standard, it should be optional :) .

The new version of the linter (that caused all the failures) will catch the empty lines among many other things in the codbase. So I'm just being preemptive here.

[nanjiangshu]

Works well when I tested. Great job!