@patrickhulce given the range of devices people are likely to run LH on, do we need more data provided? I can more than likely help there

This looks really good so far @patrickhulce - The only thing that I'm 🤔 about is how people will remember what very-slow actually means? Could we class them in some other way? (I thought of years, "-1 year" -- like how people use autoprefixer etc… but that doesn't feel right, I think)

given the range of devices people are likely to run LH on, do we need more data provided? I can more than likely help there

Sure the more the better! The most important thing IMO would be selecting cutoffs that will affect the fewest people. i.e. there's the fewest devices on the border of two categories.

how people will remember what very-slow actually means? Could we class them in some other way?

In the original throttling doc I wrote I had used something more like "Premium Mobile", "Budget Mobile", etc, but given the disparity between premium android and premium iOS it's not exactly accurate or fair. Year is also tricky because new budget android devices (for example Alcatel 1X) are much slower than phones from 5 years ago. Definitely on board with finding something as descriptive/accurate as possible but I'm at a loss for what that actually is haha 🤷‍♀️

Definitely on board with finding something as descriptive/accurate as possible but I'm at a loss for what that actually is haha 🤷‍♀️

I've the same feels atm. Let's sit on it for a day or two and see if anything falls out?

Let's sit on it for a day or two and see if anything falls out?

So now that it's 7 days, has anything fallen out of anyone? ;)

I think from what we've seen, professional performance advocates will do the research into the devices and choose something that works for the circumstances that that are being tested for, whereas "performance interested" folks are more likely to take our guidance on what the test devices are.

Given that, I think the best naming convention that comes to mind is exactly what you've come up with.

I'd love to see a table of the kinds of devices that each setting represents, something that can be referenced and updated in (likely a year or so?). Maybe with an accompanying benchmarkIndex for completeness:

@brendankenny @paulirish any opinions here?

Does anyone use cpuSlowdownMultiplier except DevTools? We could get rid of it with v4

just to be clear, I mean get rid of it and replace it with cpuTargetDeviceClass selection instead :)

I definitely like all the things you're saying. Treating this as a computed artifact for the real multiplier SGTM assuming the other pieces look good to folks to proceed.

I'm not convinced we can eliminate cpuSlowdownMultiplier though :/ I think the strongest use case for keeping it is in LR/DAZL-like situations when you have a fleet of servers you want to run lots of tests with exact same settings and not waste time running the benchmark every time (yet to be implemented :D). I see a future world in which we only run the benchmark when cpuTargetDeviceClass is set.

The other of the stagnant PRs I mentioned today :)

This would be nice to clear up before v4 if our decision involves eliminating cpuSlowdownMultiplier (I'm not sure we want to do that), but otherwise can wait for post-CDS.

Yeah, your point above is a good one, and even if do get more confident that we can eliminate cpuSlowdownMultiplier, we'll likely want a decent transition time anyways as we tune the benchmark, interpretation of the results, etc.

Makes sense to start with it as just additive.

bump :)

Feel more like... `core(throttling): add option to slowdown to a device class" or sumpthin.

done!

The plan is to try this out and see how it goes before setting the multiplier dynamically for everyone, right?

yep

I was thinking about how we might surface this to the user to let them know that the benchmark index has been adjusted, some ideas:

Oooh I like those. I was also thinking it might be part of a much larger rework of the environment settings. Something that communicates more clearly how it was run and what we were targeting.

I know this is a preliminary PR and isn't really applied yet, but I was thinking about how we might surface this to the user to let them know that the benchmark index has been adjusted, some ideas:

I prefer your first option @exterkamp — It's important that people can clearly see what the machine was rated at, as well as the slowing applied to it 👍

This isn't ready for primetime, and we've decided to revisit this with perhaps a new benchmark in the future.