Limit the amount of mails Mailhog saves

[oers]

I have a long running Installation for a test environment.
To speed up the reaction times of the gui I would like to:

• limit the maximum age of a mail kept (e.g. 24 hours)
  and/or
  • limit the amaount of files kept (LIFO) like 50

[ian-kent]

thanks @oers 👍 good idea!

[matglas]

Is there any known progress on this?

[savinov]

As a workaround: call periodically HTTP DELETE /api/v1/messages to clean the storage.

[allangood]

I've started to work on this item as it's seems to be very critical for me.
I'm using MongoDB because "inmemorry" storage uses too much memory to store embedded content and attachments, and MongoDB tends to be very slow after 20k~50k messages (MongoDB running with 1.5GB limit).

For now, to make thinks easy, I'm implementing limit based on number of messages for InMemory and MongoDB storage. Maildir needs more work.

I will submit the PR in a few days.

Regards.

[allangood]

Addressed on PR #244
Hope it helps!

Regards.

[allangood]

I would like to share some tests results using this feature enabled.

Before and after the patch
The big drop is when I started to use the --storage-limit 2000 switch

Messages sent during that period:

Regards.

[fabiencytadel]

Nice work @allangood !

[trajano]

Is this merged yet?

[allangood]

No, isn't. But I've forked this project and published a version with this change on my account.
https://github.com/allangood/MailHog/releases/tag/v1.0.1

[trajano]

@allangood is there a docker image for this?

[allangood]

Hi @trajano , you can use the Docker file available on my repository to build it. It uses the latest version of Alpine.

[trajano]

@allangood what's the URL for the repository? I can't find it in docker hub. Oh I just read it carefully, you didn't publish it to Hub then.

[acky001]

This needs to be merged, because without it, automatic periodical tests are not possible with MailHog. The memory consumption will kill it sooner or later. It's just a matter of time and amount of mails sent.

[allangood]

Sorry guys, but I never publish it on Docker Hub.
For now, you have to git clone/download and build the image locally.

I didn't want to "compete" with the original repository...

[mookkiah]

I see there is lot of value in automatic clean up of emails as a feature in mailhog.
Even though HTTP DELETE /api/v1/messages helps, it is wiping all.

We use mailhog as a service for development and automated functional test practice.
Also we use for load testing email sending workflows. This avoids stress in our actual email server.

Wiping all mails periodically causing tests to fail as our test script checks back to mailhog if mails received as expected.

So I would like the clean up procedure as configurable like
when environment property CLEAN_EMAILS_OLDER_THAN_MINUTES is set has value then run a clean up activity which acts accordingly.

There are options how to implement
Option 1: Make the clean up activity as part of the product
Option 2: Write shell script call it from the container run time platform (docker swamr, kubernetes etc). For example have a cron job/side car for in kubernetes which calls the script instead of starting mailhog
Option 3: Make the script available in documentation so that users can decide however they want to run.
Option 4: Introduce a filtering option on messages api like /api/v2/messages?older_than_in_minutes=60. Let the user call the REST api with DELETE method using their platform offered scheduling option.

Any opinion to add/eliminate these options to come to a better decision?

[sgohl]

I would not expect mailhog to implement a whole scheduler just for this one task, if not for something more useful anyway.
For me, it would be absolutely acceptable to have a callable HTTP route with a GET argument to provide the OLDER_THAN amount, be it minutes, hours or days. It's not important how long the task needs to run.
To prevent Mailhog from needing to run a possible costly query, I can also think of a task which just marks/labels ALL mails "to_be_deleted" with one HTTP-route, while extending the already existing endpoint to optionally only delete those mails which have been marked/labeled

[mookkiah]

@sgohl, I hope you are referring to add a query parameter older_than to GET /api/v2/messages api. Right? Or you meant to say DELETE?

By saying "not expect mailhog to implement a whole scheduler", I guess you are preferring to introduce the option 4 which provides API to delete older_than (in minutes) emails. Let the user decide how they want to call the delete rest api.

Let me know if I did not write option 4?

[sgohl]

I hope you are referring to add a query parameter older_than to GET /api/v2/messages api. Right? Or you meant to say DELETE?

yea ok sry, I think I got your 4th option wrong at first. Reading again, it's exactly what I meant

[mookkiah]

@sgohl @ian-kent I am thinking about implementing it. But I do not see any recent PR got accepted.
Even the contributing section needs update.
May be I am not looking properly. Please help me understand. Is this project accepting contribution?

[vvp1983]

hello! 2 y.ago... Any news? Script or crontab with api or other - very bad idea.

[jeremysimmons-ent]

Very interested in this PR what do we need to do to make it happen? I'd be happy to update the PR to resolve conflicts if necessary.

[petertwise]

Interested to see @allangood 's PR make it in eventually.

Our needs are much more basic, so for now I'm using @savinov 's DELETE method overnight daily just with this in my crontab

# purge the MailHog inbox at 4am nightly
0 4 * * * curl  --header "Content-type: application/json" --request DELETE http://0.0.0.0:8025/api/v1/messages

(placed here for any other n00bs like me, using everything out of the box default, who might need to google a few things to figure out how to "just run DELETE on the API periodically")

[dhlavac]

Any update on this feature? Limit on messages/memory would be helpful

[mabihan]

A workaround :

1. Use file-based storage (use MH_STORAGE and MH_MAILDIR_PATH)
2. Add a cron that keeps only last n files

docker-compose.yml :

version: '3'

services:
  mailhog:
    restart: always
    image: mailhog/mailhog
    container_name: mailhog
    environment:
      MH_STORAGE: maildir
      MH_MAILDIR_PATH: /home/mailhog/mails
      MH_API_BIND_ADDR: 0.0.0.0:8026
      MH_UI_BIND_ADDR: 0.0.0.0:8026
      MH_SMTP_BIND_ADDR: 0.0.0.0:8025
    ports:
      - "8025:8025"
      - "8026:8026"

crontab (test only with macOs)

# delete mailhog data
* * * * *       ~/.scripts/clean-mailhog.sh

~/.scripts/clean-mailhog.sh

#!/bin/bash

PATH=/usr/local/bin:/usr/local/sbin:~/bin:/usr/bin:/bin:/usr/sbin:/sbin

CONTAINER_NAME="mailhog"
KEPT_FILE_COUNT=100

MH_MAILDIR_PATH=$(docker exec $CONTAINER_NAME sh -c 'echo $MH_MAILDIR_PATH')
MH_HOME=$(docker exec $CONTAINER_NAME sh -c 'echo $HOME')

((KEPT_FILE_COUNT+=1))

docker exec $CONTAINER_NAME sh -c "ls -t $MH_MAILDIR_PATH/|tail -n +$KEPT_FILE_COUNT|xargs -I @ rm  $MH_MAILDIR_PATH/@"
docker exec $CONTAINER_NAME sh -c "echo 'last cleanup at $(date)' >  $MH_HOME/cleanup.log"