Probing/exchange rates are not a STREAM concern

[kincaidoneil]

Proposal: Allow logic enforcing minimum exchange rates to be completely disabled, including:

1. Sending test packets before the stream is established
2. Requiring packets to meet the minimum exchange rate that STREAM calculates

---

1. Supporting receive-only mode.
   • Case 1: credit with connector is 0, which currently causes all test packets to fail and stream to be closed, even if the endpoint won't be sending money.
   • Case 2: there exists credit with the connector, but the probing temporarily exhausts it, which is unnecessary and can cause disruptions for other applications.
2. Insecure. Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors, so for the foreseeable future, most clients necessitate their own exchange rate oracle to ensure the rate is reasonable.
   • In practice, the current behavior only really ensures the exchange rate stays consistent/within the slippage margin throughout the stream, which isn't desirable since it should respond to exchange rate fluctuations. (By contrast, the price oracle approach can respond to fluctuations, if necessary).
   • Instead, a callback function could be passed to STREAM clients & servers, that given a source amount and a destination amount, could return true/false if that particular packet should be fulfilled. (If it returned false, STREAM would reject that packet). This would allow consumers of the module to integrate with their preferred price oracle or use their own method to enforce a minimum exchange rate on a per-packet basis.
3. Longer connection establishment. Probing wastes time while establishing the connection (Send money before determining the exchange rate? #44), when alternatively it could be used to start sending money immediately, since the test packets can trivially be identified and gamed.
4. No support for large asset scale differences. Probing with no knowledge of the relative exchange rate can introduce problems with vastly different exchange rate scales. (And requires sending massive test packets, which as noted earlier, could temporarily exhaust bandwidth that other applications are using, if they're forwarded).

To determine the initial packet size to send, STREAM could send a single very large packet (e.g., maybe the maximum uint64 an ILP packet allows the total amount remaining to send). F08s (which connectors should be configured to trigger before T04s) should quickly reduce it to a reasonable amount. (Alternatively, STREAM could be more aware of settlement/how much credit exists with the upstream peer, but I'm guessing that'd be more contentious).

[emschwartz]

Having a way to disable the probing and set the prepareAmount to 0 sounds good to me.

Instead, a callback function could be passed to STREAM clients & servers

The client doesn't know the amount that will actually arrive at the destination beforehand and the server would need another frame added to the protocol to know the source amount the sender sent. The original idea was to put this logic in the hands of the sender (so they could probe, use a price feed like you describe, or any other method) and make the receiver respect their wishes independent of the logic they use (with a simple check of the prepare amount against the desired amount configured in the stream packet).

[kincaidoneil]

Instead, a callback function could be passed to STREAM clients & servers

The client doesn't know the amount that will actually arrive at the destination beforehand and the server would need another frame added to the protocol to know the source amount the sender sent. The original idea was to put this logic in the hands of the sender (so they could probe, use a price feed like you describe, or any other method) and make the receiver respect their wishes independent of the logic they use (with a simple check of the prepare amount against the desired amount configured in the stream packet).

Oh, right, right. Instead, the callback could return that minimum acceptable amount that the sender of the payment sets, e.g. minDestinationAmount: (sourceAmount: BigNumber) => BigNumber (it'd just allow the consumer to set that on a per-packet basis rather than Stream on a per-connection basis).

[traviscrist]

To determine the initial packet size to send, STREAM could send a single very large packet (e.g. the total amount remaining to send). F08s (which connectors should be configured to trigger before T04s) should quickly reduce it to a reasonable amount. (Alternatively, STREAM could be more aware of settlement/how much credit exists with the upstream peer, but I'm guessing that'd be more contentious).

Won't this just result in the same issue we have with #44. Starting with a large real amount and then waiting for T04s to back off could lead to an even longer time till money is sent. It would just half each packet until it gets a small enough amount. This could take many round trips to establish.

If asset scales are different enough I will have to get to sending 1 so the receiver gets 100 or 1000.

What do you propose as the start size of this packet?

If that amount is too small say 1,000,000,000 and only 1 gets there how do you deal with that? (there is very little precision)

[kincaidoneil]

Won't this just result in the same issue we have with #44. Starting with a large real amount and then waiting for T04s to back off could lead to an even longer time till money is sent. It would just half each packet until it gets a small enough amount. This could take many round trips to establish.

Good point. I guess I'd be fine keeping the volley of packets and just making them all fulfillable (and then just ensuring they total less than the amount remaining to send). However, that might require refactoring to support multiple packets in flight (?).

If that amount is too small say 1,000,000,000 and only 1 gets there how do you deal with that? (there is very little precision)

The nice thing about this approach is it lets the consumer deal with how much precision they're willing to accept. If the callback is called with 1,000,000,000 as the source amount, and the sender calculates 1.4056 as the minimum destination amount (for example), they could just round that up to 2. (If Stream could send a packet for larger than 10^9, presumably it would've already tried to send that, right?). In this case, the Stream would likely fail, because the receiver would probably receive packets of amount 1, to be rejected, and not 2 -- which should emulate the current behavior.

The one downside is it might take the entire timeout to fail, rather than fail pretty quickly.

[sentientwaffle]

Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors ...

Can you elaborate on this?

---

Also, how do you imagine the price oracle function would work if not with probe packets? Would it effectively be like a connector's rate backend with some slippage tacked on to account for the route?

[kincaidoneil]

Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors ...

Can you elaborate on this?

As a connector, I look for packets with an amount of 1,000,000,000,000 (the first test packet). If I see one, I know that it's more than likely a probing packet for a STREAM connection. I also know the specific connection tag from the destination address. If any subsequent packets are sent with that connection tag, I'll just take a much larger slippage margin than I typically would. If the sender & receiver have no other knowledge of the exchange rate other than the probing packets sent through me, they'll start fulfilling packets over it, enabling me to keep taking that higher slippage margin. Other packets would be unaffected, since I'm only doing it for their connection tag.

But, if the sender had multiple uplink plugins, Stream could probe using several of them to ensure the exchange rate was not manipulated, and then just use the uplink connector with the most competitive rate. (However, the receiver's direct downlink connector could still be manipulating the rate).

To make probing alone secure, I think it'd require:

1. Enough liquidity and connectors so there are several wholly independent paths the packet could take from sender to receiver
2. The ability for the receiver to have multiple downlink plugins (and therefore multiple destination addresses) linked to the same shared secret, so they could correlate packets for a single Stream connection received from different plugins

Also, how do you imagine the price oracle function would work if not with probe packets? Would it effectively be like a connector's rate backend with some slippage tacked on to account for the route?

Yep! Just like the connector's rate backend, minus slippage. In fact, in Switch/ilp-sdk, we're using the very same rate backend on the client as we do on our connector (using crypto-rate-utils, which uses CoinCap as its oracle).

[sentientwaffle]

I'm not a fan of requiring a rate backend in the sender or receiver.
From interledger's design goals:

Interoperability - ILP should be usable across any type of ledger, even those that were not built for interoperability.

Requiring that the destination currency be known by the sender (in the rate backend) hinders interoperability.

btw, the stream Connection has the minimumAcceptableExchangeRate method that seems relevant. If the sender is aware of the destination currency, they could sanity-check the actual exchange rate against an expected one.

[kincaidoneil]

Requiring that the destination currency be known by the sender (in the rate backend) hinders interoperability.

I kinda agree, but in any case, this wouldn't be the default. I like the probing model theoretically, but at this stage in the network, I just don't think it's secure in practice. (Also, didn't the ConnectionAssetDetails frame already kinda break that principle?)

btw, the stream Connection has the minimumAcceptableExchangeRate method that seems relevant. If the sender is aware of the destination currency, they could sanity-check the actual exchange rate against an expected one.

Yep. We've used that in the past, but what I'm proposing would give you control over that on a per-packet basis, rather than per-connection -- it exposes much more control to the consumer. And since probing has introduced issues for us with receive only mode (among the other grievances mentioned), it'd just be nice to have the ability to disable that entirely.

[traviscrist]

The main purpose of the probing is to establish a connection where the path has a minimum precision before sending fulfillable packets. It also helps discover large asset scale differences without risking funds. I'd like to get rid of the probing but I don't have a better idea on establishment if you care about the exchange rate and precision.

1. Supporting a receive only mode makes sense, I see the use case for allowing this and disabling the current connection establishment process and need for an exchange rate.

2. Allowing a callback to be passed into STREAM which can have custom exchange rate logic seems like a good idea. I fail to see how this can do things on the fly very well though. This would not know the asset classes unless the optional ConnectionAssetDetails frame is sent. In general ILP does not care about the asset class by design.

3. I don't think that is the case unless you get lucky and the first packet is fulfilled. I see 2-3 packets needed at minimum even if fullfillable. First gets rejected with an F08, the second is then likely a T04 and then the third finally makes it.

Also how is this first packet amount determined? It is a random number between some values so it cannot be guessed?

4. This proposal would also not have support for large scale asset differences. If the first packet fails to fulfill since the receivers scale is magnitudes larger. How does the STREAM client know to increase their fulfill packet amount? There's no amount too small code...

[kincaidoneil]

Update to my thinking on this:

Using an exchange rate API is still requisite for security, but sending probing packets (both initially, and maybe even throughout the payment) is probably useful, for two reasons I overlooked:

1. Accurately estimating the exchange rate

• Wallets probably want to the present the user with a more accurate estimated destination amount before the user approves the payment (though should still be verified against external sources)
• Improves the UX if the wallet knows whether the payment will succeed (is the exchange rate good enough?) before attempting the payment, to fail faster if the rate turned out to be really bad or there wasn't a way to route the payment

2. Protecting against the recipient claiming the slippage

One issue with STREAM is if the recipient receives more than minimum prepare amount, they can lie to the sender and say they only received the minimum amount, effectively claiming whatever amount was allocated for slippage and get paid (slightly) more than they should. The problem is the sender is revealing their minimum before the recipient commits to how much they received.

One way to address this is using a Loopback transport (initially proposed by Michiel de Jong, and recently explored by Adrian, Matt & Don): instead of giving the recipient the ability to derive the fulfillment, they forward the packet back to the sender at the link layer (e.g. using ILP-over-HTTP) with the amount they received. They're committing to the amount they received first, and then the sender chooses to fulfill or reject without revealing their minimum amount that should be received. This probably isn't something we'll adopt, but is interesting in that it addresses this problem.

But, STREAM probing using unfulfillable test packets achieves the same objective, since the recipient is committing to how much they received without the sender revealing any information about their minimum acceptable exchange rate. If these packets are sent before the payment begins and it turns out to be better than the the external exchange rate minus slippage, then the sender should just sets minimum exchange rate higher to minimize the margin the recipient can lie about! Which is essentially the behavior JS STREAM already has 😄