feat: Payment settlement in PDP [DO NOT MERGE]

[LexLuthr]

1. Settle every 48 hours (adjust after review)
2. Delete dataSet

• Pull in FWSS v1.0.0 for latest ABI

Pending Decisions

• Redo IPNI to announce removal? Current implementation lacks too many things
  1. No way to do rm ads
  2. Same piece will be announced multiple times to same conextID
  3. We are building contextID with pieceCid v1 for some reason

[ZenGround0]

About halfway through. Settlement watcher looks pretty good. Next up I need to understand what exactly is going on with the terminate service watcher and how its distinct from dataset deletion watcher.

[ZenGround0]

0x09a0fDc2723fAd1A7b8e3e00eE5DF73841df55a0

[ZenGround0]

FYI getting these directlty from FWSS contract:

cast call --rpc-url http://api.calibration.node.glif.io/rpc/v1   0x02925630df557F957f70E112bA06e50965417CA0 "paymentsContractAddress()"

[ZenGround0]

0x2321fB2A346E3dA71f9310FB477384fF727924ba

[ZenGround0]

I am wondering if multicall is really necessary for whatever we're doing here?

It ends up useful in client facing applications where latency matters a lot but this background settlement process should not be latency sensitive.

What's the main motivation? Gas savings?

[LexLuthr]

Gas saving + reducing the number of messages. Each settlement requires its own message. We don't have settlerails() func.

[ZenGround0]

I'm not against it necessarily but I'd like to understand why we're doing this. I'd be surprised if this helps much with gas since multicall afaik iterates through a bunch of sends and sends them one by one to the contract. So we're not getting any batching locality in the payments contract. And we're not saving much on message overhead since we need to specify all call parameters. I don't think latency matters very much or that one big multicall makes mpool resource use all that better.

The cost is that we have yet another contract dependency. For example all devnet testing will need to deploy a new multicall. And if something goes wrong with our multicall contract we need to be on top of switching to a new one.

Something @rvagg noticed is that we could have used multicall to make our lives easier with the pdp verifier create and add call so this could have helped there. But that ship has sailed.

[LexLuthr]

Gas saving + reducing the number of messages. Each settlement requires its own message. We don't have settlerails() func. Multicall was suggested by @wjmelements as a workaround for now. IIRC, @rvagg also ran into gas issues with settling, I am not sure if Multicall with help or just make things worse for that.

My PoV - I want to send least amount of message to settle.

[ZenGround0]

The as issues @rvagg ran into with settling were essentially contract logic bugs which we know how to fix (and hopefully did?) These are orthogonal to this issue.

My understanding of the gas model as I outline above makes me suspect that multicall savings are very small. I won't hold up this change but just FYI we are adding medium complexity and a serious dependency for an amount of gas savings that I don't think we've measured and I don't think is very big. It feels like a mistake.

[ZenGround0]

But yeah no argument on number of messages. If there are good reasons to reduce those then this is a no brainer.

[wjmelements]

Gas issue should be resolved by two optimizations that landed about two weeks ago.

[LexLuthr]

About halfway through. Settlement watcher looks pretty good. Next up I need to understand what exactly is going on with the terminate service watcher and how its distinct from dataset deletion watcher.

This mostly comes down to some constraints in settle. When a service it terminated, we set endEpoch. We can get paid till endEpoch if we provide service.

Below is the Gist.

DataSet deletion: (SP side deletion)

1. LockUpSettle falls sorts
2. Terminate service. Can't terminate rail directly
3. New table to track dataSet deletion. Add entry to this table.
4. Track the terminate rail message and mark success in table once executed.
5. Check that FWSS now shows an epoch for termination in dataSet. If not, wait till it show up.
6. If termination epoch is set and head > epoch. Then, we send termination message to PDPVerifier.
7. Track the message and wait for success. Then double check that dataSet is not live anymore
8. Delete all local pieces, clean up tables

I wanted to terminate the DataSet immediately after terminating the service but unfortunately, lockup settle seems weird enough to not allow me a clean exit.

1. User stops paying day 1.
2. Day 3 I try to settle and find out user hasn't paid.
3. Terminate service - this frees up lockup to be claimed by SP. EndEpoch is set to lastSettledAt+lockUp period.
4. If I try to delete the dataSet now, https://github.com/FilOzone/filecoin-services/blob/8e162d676f3e83c495f104989b3014b3961e2f05/service_contracts/src/FilecoinWarmStorageService.sol#L750-L753 will prevent it. IIRC, this is to allow client time to find another provider.
5. So, I keep providing service for 8 more days and keep trying to settle this rail. There is a nuance if I am correct. I must settle before or at EndEpoch to get all the payment. Otherwise, it will be unlocked and user can withdraw it.
6. Even if I somehow do that, then I still have to wait till after endEpoch to terminate the dataSet.

This different task doing different things. I still need to figure out how to do 5 reliably.

Dealing directly with payment contract bring in too many constraints and edge cases. This is the reason why I asked why doesn't operator do this cleanly?

I am open to less convoluted ideas if you can think of any.