Skip to content

Update swaps average savings calculation#9662

Closed
rekmarks wants to merge 3 commits intodevelopfrom
swaps-average-savings-calculation-fix
Closed

Update swaps average savings calculation#9662
rekmarks wants to merge 3 commits intodevelopfrom
swaps-average-savings-calculation-fix

Conversation

@rekmarks
Copy link
Copy Markdown
Member

@danjm noticed that we were effectively doubling the size of the fee savings (whether negative or positive) by calculating performance savings using ethValueOfTrade, which already has the fees subtracted.

This PR fixes our average savings calculation by using the existing intermediate value, ethValueReceived, and its median to calculate performance savings.

In addition, we add the following properties to all quotes, to avoid recalculating their values in the UI:

  • ethFee
  • ethValueReceived
  • ethValueOfTrade

@rekmarks rekmarks requested a review from danjm October 20, 2020 19:19
@rekmarks rekmarks requested a review from a team as a code owner October 20, 2020 19:19
@rekmarks rekmarks force-pushed the swaps-average-savings-calculation-fix branch from 706cb17 to 94f1d16 Compare October 20, 2020 19:43
@metamaskbot
Copy link
Copy Markdown
Collaborator

Builds ready [94f1d16]
Page Load Metrics (432 ± 58 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint3097492211
domContentLoaded30168543012158
load30369143212258
domInteractive30168543012158

brad-decker
brad-decker previously approved these changes Oct 20, 2020
Copy link
Copy Markdown
Member

@Gudahtt Gudahtt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danjm noticed that we were effectively doubling the size of the fee savings (whether negative or positive) by calculating performance savings using ethValueOfTrade, which already has the fees subtracted.

I don't understand the problem here. Subtracting fees was intended, right? So that we account for the network fee in determining savings. How else are network fees accounted for? If we solely focus on the received value, we might end up in a place where the best quote has a lower received value than the median due to the median having a higher network fee.

Edit: Oh I see now. The part where it's double-counted is in calculating savings.total. I understand now.

async _findTopQuoteAndCalculateSavings (quotes = {}) {
/**
* Calculates trade data in order to identify the best quotes and its savings.
* Modifies the passed-in quotes in place, adding the following decimal string
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why the quotes are modified in-place here, rather than being returned. I find it easier to understand what a function does if it doesn't rely upon mutating the parameters, and the types become much more informative as well.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't think any of these new values should be attached to the quote object before they're used either. This strategy of keeping state around for later has left us with a lot of unused state that we had forgotten to clean up. Best to add it only when we being actually using it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my understanding, problems with mutating in place arise when the mutated data is accessed by many different contexts over its lifetime. In this instance, the mutation occurs at a single time and place, namely when _addTopQuoteAndTradeData is called in fetchAndSetQuotes.

I'd suggest that we return the modified quotes object from _addTopQuoteAndTradeData, while still modifying it in place. This would grant us the type benefits without having to reinstantiate the quotes object.

If we want to also reinstantiate the quotes object in _addTopQuoteAndTradeData as a matter of principle, I'm also fine with that, although I don't think it does us any good in this case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

problems with mutating in place arise when the mutated data is accessed by many different contexts over its lifetime.

Nah I wasn't referring to anything like this. I meant it was hard to understand. Mutation makes the code hard to read.

I expect a function's type signature to give some hint as to what it's doing - stuff goes in, stuff comes out. It makes tracing the code easier as well. I would never have expected this to mutate the quotes before reading this doc comment. This is particularly problematic when reading where this function is used, as it wouldn't occur to me to read the definition of every function invoked just to follow what was going on with the quotes. I would see that it returned the ID of the top quote, and assume that's all it did.

We also lose any opportunity to describe the operation with types if we rely on mutation. With proper parameter and return types, we can express using the type system that these properties were not present before, but may be present afterwards.

Copy link
Copy Markdown
Member Author

@rekmarks rekmarks Oct 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also lose any opportunity to describe the operation with types if we rely on mutation.

I was suggesting that we can mutate the quotes in place and return them (altering the function's return type) to get this benefit:

we can express using the type system that these properties were not present before, but may be present afterwards.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest that we return the modified quotes object from _addTopQuoteAndTradeData, while still modifying it in place. This would grant us the type benefits without having to reinstantiate the quotes object.

Ah, right, this sounds fine. Sorry I only meant to take issue with the function signature, not the mutating of the parameter itself. I guess I don't feel strongly one way or the other about whether the parameter is mutated in this case, since it's a private function used in one place - I explained that poorly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we are in agreement, I shall alter the function signature!

Copy link
Copy Markdown
Member

@Gudahtt Gudahtt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still seems incorrect 🤔

We're now comparing the best quote against the median fee and the median received value, even if those two numbers are from different quotes. We should be comparing the best quote to the median quote when considering the total value. Once that quote is identified, we can extract the fee v.s. performance savings.

@rekmarks
Copy link
Copy Markdown
Member Author

rekmarks commented Oct 21, 2020

@Gudahtt, subtracting fees is necessary to get the ETH value of the trade, but in our performance savings calculations, we were subtracting them twice. Referring to the equations in the description of #9611:

performance_savings = best_quote_trade_value - median(all_trade_values)
fee_savings = median(all_trade_fees) - best_quote_trade_fee
total_savings = performance_savings + fee_savings

The problem is with the trade_value for performance_savings. Consider:

trade_fee = total_eth_cost - (source_token_is_eth ? amount_of_eth_swapped : 0)
trade_value = destination_token_eth_value - total_eth_cost

trade_fee is a component of total_eth_cost. So, before this PR, we were doing this (breaking best_quote_trade_value and best_quote_trade_fee into their components):

performance_savings = (
  destination_token_eth_value - 
  (best_quote_trade_fee + best_quote_amount_of_eth_swapped)
) - median(all_trade_values)

fee_savings = median(all_trade_fees) - best_quote_trade_fee

total_savings = (
  (
    destination_token_eth_value -
    (best_quote_trade_fee + best_quote_amount_of_eth_swapped)
  ) - median(all_trade_values)
) + (median(all_trade_fees) - best_quote_trade_fee)

You'll see that best_quote_trade_fee is subtracted twice. This PR corrects these equations to this:

performance_savings = best_received_value - median(all_received_values)

fee_savings = median(all_trade_fees) - best_quote_trade_fee

total_savings = (
  best_received_value - median(all_received_values
) + (median(all_trade_fees) - best_quote_trade_fee)

We still calculate the trade_value and use it for the same purposes as we did prior to #9611.

@rekmarks
Copy link
Copy Markdown
Member Author

rekmarks commented Oct 21, 2020

We're now comparing the best quote against the median fee and the median received value, even if those two numbers are from different quotes.

@Gudahtt what do you mean "even if those two numbers are from different quotes"? We first identify the best quote using the ethTradeValue as we did before, then we compare the ethValueReceived and ethFee of the best quote to the median for each of all quotes.

@Gudahtt
Copy link
Copy Markdown
Member

Gudahtt commented Oct 21, 2020

then we compare the ethValueReceived and ethFee of the best quote to the median for each of all quotes.

That last part is the problem - the quote with the median ethFee isn't necessarily the same quote as the one with the median ethValueReceived. We're comparing the best quote with an imaginary one composed of the fee from one quote and the received value from another.

@rekmarks
Copy link
Copy Markdown
Member Author

@Gudahtt is right: we have to identify the quote or pair of quotes that comprise the median ethTradeValue, and then compare the average of their ethFee and ethValueReceived values to those of the best quote. I'll do this tomorrow morning.

Great catch!

@metamaskbot
Copy link
Copy Markdown
Collaborator

Builds ready [02d2fa2]
Page Load Metrics (413 ± 50 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint28523663
domContentLoaded25558841110550
load25659041310550
domInteractive25458841110550

@danjm
Copy link
Copy Markdown
Contributor

danjm commented Oct 21, 2020

An alternative to this PR that avoids the need to calculate ethValueReceived: https://github.com/MetaMask/metamask-extension/compare/alternative-savings-fix

@rekmarks
Copy link
Copy Markdown
Member Author

This will be superseded by a PR from @danjm some time today.

@rekmarks rekmarks closed this Oct 21, 2020
@github-actions github-actions bot locked and limited conversation to collaborators Oct 21, 2020
@rekmarks rekmarks deleted the swaps-average-savings-calculation-fix branch March 12, 2021 19:32
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants