Feature delete expired threads in mongo aio checkpointer

[XenoAmess]

add some utility to auto-delete expired threads in mongo aio checkpointer
that be useful when in real-world production usage

[blink1073]

Hi @XenoAmess! Can you describe your use case for this feature? I'm not sure it is in scope for this library, is it something you could maintain separately?

[Pesokrava]

I would like to have this feature. The use case is pretty straightforward, after a while the DB will be populated with no longer relevant checkpointers. One option would be to have some sort of a cleanup CRON job but since mongo does support TTL, it would be much more convenient to have an option to use that built-in functionality. But for mongo TTL one needs a index on a timestamp / datetime field, which is currently not possible.

[XenoAmess]

Hi @XenoAmess! Can you describe your use case for this feature?

we need this feature (at least in my work situation) to make sure the middleware cost would not be accumulate infinitally.

I'm not sure it is in scope for this library, is it something you could maintain separately?

Yes, currently we maintained a class in our repo for this thing, and that can work.

But I think maybe quite some people would need something like this.

If used in production usually we need to think about thinks like this IMO...

[XenoAmess]

I would like to have this feature. The use case is pretty straightforward, after a while the DB will be populated with no longer relevant checkpointers. One option would be to have some sort of a cleanup CRON job but since mongo does support TTL, it would be much more convenient to have an option to use that built-in functionality. But for mongo TTL one needs a index on a timestamp / datetime field, which is currently not possible.

yes and notice that we need to delete a whole chat session when delete it, and we use a multiple document way to store it, it means if we use mongo TTL we might have possibility that a thread is half-deleted, that be dangerous

[jpfcabral]

Any updates on it?

[jpfcabral]

hey, @XenoAmess .

Let me know if could be easier if you just add a new index...

https://www.mongodb.com/docs/manual/core/index-ttl/

[XenoAmess]

hey, @XenoAmess .

Let me know if could be easier if you just add a new index...

https://www.mongodb.com/docs/manual/core/index-ttl/

well if you implement it this way, you might take care about situation that parent node die before&after child node...

for example, ttl configuration change during an update, thus makes parent node ttl after child node

and please make sure a half-deleted session would not cause trouble, like, during this time of run of graph between 2 nodes, a very old message deleted due to ttl.

[caseyclements]

I prefer to proceed with #108 . TTL indexes are a great built-in feature of mongodb, and very low-touch. That said, I am currently implementing the BaseStore for long-term which deals explicitly with TTL.We'll learn a lot if I can get that that to work with TTL Indexes, and the LangGraph maintainers are happy with the solution.

[jpfcabral]

Good to hear, @caseyclements. Let me know if you need help with this. I'm currently using this implementation (#108) in my work and it will be very helpful to have these implementations thought out in the long run.