Description

When Infura goes down, we want to automatically and invisibly redirect requests from Infura RPC endpoints to Quicknode endpoints.

• There is not one central Quicknode RPC endpoint that all Infura requests will be forwarded to; rather, each Infura-supported chain gets its own Quicknode endpoint. These URLs contain an API key, so we need to add an environment variable for each endpoint.
• All of the Infura RPC endpoints that have an associated Quicknode endpoint are:
  • Ethereum Mainnet
  • Linea Mainnet
  • Arbitrum Mainnet
  • Avalanche
  • Optimism (OP) Mainnet
  • Polygon Mainnet
  • Base Mainnet
• The failover behavior itself is already present in @metamask/network-controller, where a new failoverRpcUrls property has been added to the RpcEndpoint type. This commit bumps @metamask/network-controller to bring in those changes, and then it not only adds a migration to go through each RPC endpoint in NetworkController state and populate the failoverRpcUrls property, it also updates the code responsible for the "additional networks" feature to ensure that Infura RPC endpoints get added with failoverRpcUrls set correctly. In addition, any other place that is responsible for adding a network, we need to make sure we pass failoverRpcUrls as an empty array (since it's a required configuration property).
• Finally, this commit adds some minimal UI to allow users to see that Infura endpoints have a failover URL assigned. Since all of the Quicknode URLs are long, this ends up getting cut off, but that doesn't matter.

Related issues

Fixes #14120.

Manual testing steps

There are 7 environment variables that you need to have set in .js.env before you begin. Please contact me in Slack and I will give them to you.

Testing the UI changes

1. Run yarn setup / yarn setup:expo and then run yarn watch.
2. Open the app.
3. Set up an account if necessary
4. Open the network switcher
5. Click on "Edit" next to "Ethereum Mainnet"
6. Under RPC Url, you should see a truncated version of the Quicknode URL below mainnet.infura.io
7. Open the network switcher again
8. Add "Avalanche C-Chain" (which is an Infura-supported "featured" network)
9. Under RPC Url, you should see a truncated version of the Quicknode URL below the Avalanche Infura URL
10. Open the network switcher again
11. Tap "Add a custom network"
12. Add a new network, such as "Flare Mainnet": https://chainlist.wtf/chain/14/. Type in a network name, add a new RPC URL, then type in the chain ID and the symbol.
    • Make sure to put the RPC URL first and then the RPC name, or else you won't be able to save.
13. Try switching to the new network. You should not see any errors doing so.
14. Click on the network switcher and edit Flare. Under Default RPC URL you should not see a failover RPC URL listed like you did for Ethereum Mainnet or Avalanche.
15. Open the browser, go to docs.metamask.io, tap the hamburger menu, go to Wallet API, tap the menu again, go to JSON-RPC API, tap the menu again, go to wallet_addEthereumChain.
16. Click on "Connect MetaMask".
17. Scroll down and run the request (which should add Gnosis). Accept the approval.
18. You should get a modal saying that Gnosis has been switched to.
19. Tap back on the wallet view and open the network switcher.
20. Go to edit Gnosis. Under RPC Url you should not see a failover RPC URL listed like you did for Ethereum Mainnet or Avalanche.

Testing the failover logic

1. Open this branch in your editor.
2. We need to lower the time that an unavailable endpoint is ignored while the failover is active. Open app/core/Engine/Engine.ts. Look for const networkControllerOpts = and make the following changes:

     const networkControllerOpts = {
       infuraProjectId: process.env.MM_INFURA_PROJECT_ID || NON_EMPTY,
       state: initialNetworkControllerState,
       messenger: networkControllerMessenger,
       getRpcServiceOptions: () => ({
         fetch,
         btoa,
   +     policyOptions: {
   +       maxRetries: 1,
   +       maxConsecutiveFailures: 10,
   +       circuitBreakDuration: 10000,
   +     },
       }),
     };

3. Open node_modules/@metamask/network-controller/dist/rpc-service/rpc-service.cjs.
4. We now need to know what URL we are requesting and also force Infura to be "down" for a little while. Find exports.RpcService = RpcService. Make the following changes:

     exports.RpcService = RpcService;
   + const startDate = new Date().getTime();
     _RpcService_fetch = new WeakMap(), _RpcService_fetchOptions = new WeakMap(), _RpcService_failoverService = new WeakMap(), _RpcService_policy = new WeakMap(), _RpcService_instances = new WeakSet(), _RpcService_getDefaultFetchOptions = function _RpcService_getDefaultFetchOptions(endpointUrl, fetchOptions, givenBtoa) {
         if (endpointUrl.username && endpointUrl.password) {
             const authString = `${endpointUrl.username}:${endpointUrl.password}`;
             const encodedCredentials = givenBtoa(authString);
             return (0, deepmerge_1.default)(fetchOptions, {
                 headers: { Authorization: `Basic ${encodedCredentials}` },
             });
         }
         return fetchOptions;
     }, _RpcService_getCompleteFetchOptions = function _RpcService_getCompleteFetchOptions(jsonRpcRequest, fetchOptions) {
         const defaultOptions = {
             method: 'POST',
             headers: {
                 Accept: 'application/json',
                 'Content-Type': 'application/json',
             },
         };
         const mergedOptions = (0, deepmerge_1.default)(defaultOptions, (0, deepmerge_1.default)(__classPrivateFieldGet(this, _RpcService_fetchOptions, "f"), fetchOptions));
         const { id, jsonrpc, method, params } = jsonRpcRequest;
         const body = JSON.stringify({
             id,
             jsonrpc,
             method,
             params,
         });
         return { ...mergedOptions, body };
     }, _RpcService_executePolicy =
     /**
      * Makes the request using the Cockatiel policy that this service creates.
      *
      * @param jsonRpcRequest - The JSON-RPC request to send to the endpoint.
      * @param fetchOptions - The options for `fetch`; will be combined with the
      * fetch options passed to the constructor
      * @returns The decoded JSON-RPC response from the endpoint.
      * @throws A "method not found" error if the response status is 405.
      * @throws A rate limiting error if the response HTTP status is 429.
      * @throws A timeout error if the response HTTP status is 503 or 504.
      * @throws A generic error if the response HTTP status is not 2xx but also not
      * 405, 429, 503, or 504.
      */
     async function _RpcService_executePolicy(jsonRpcRequest, fetchOptions) {
         return await __classPrivateFieldGet(this, _RpcService_policy, "f").execute(async () => {
             const response = await __classPrivateFieldGet(this, _RpcService_fetch, "f").call(this, this.endpointUrl, fetchOptions);
   +         console.log(`=== Fetching ${this.endpointUrl.toString()} ===`);
   +         const now = new Date().getTime();
   +         if (this.endpointUrl.host.includes('infura.io') && (now - startDate) < 120000) {
   +           throw new Error("Infura is down");
   +         }
             if (response.status === 405) {

5. Shake the device, reload the app, and wait for the dev server to rebundle.
6. In your terminal, press Cmd-F and start a search for === Fetching.
7. You'll see a bunch of messages flying by. Some of these are Sentry logging messages, which you can ignore. You'll start seeing === Fetching <URL> === messages where it's an Infura URL, followed by the error we just created ("Infura is down").
8. After about 10-20 seconds, you'll start seeing === Fetching <URL> === messages where instead of an Infura URL, it's a Quicknode URL (https://<descriptive-random-string>.quicknode.pro/<api-key>). You have to look closely, because right after that you will still continue to see the "Infura is down" message.
9. After about 10-20 more seconds you'll start to see errors like "Execution prevented because the circuit breaker is down".
10. Pretty instantly after that, Infura should "recover" and you should start seeing no more errors.

Screenshots/Recordings

Before

After

This is what it now looks like when editing an Infura network:

This is what it looks like when editing a custom network:

Here is a video that goes through all Quicknode-suppported Infura networks and then shows selecting a custom RPC endpoint vs. Infura RPC endpoint:

Pre-merge author checklist

• I’ve followed MetaMask Contributor Docs and MetaMask Mobile Coding Standards.
• I've completed the PR template to the best of my ability
• I’ve included tests if applicable
• I’ve documented my code using JSDoc format if applicable
• I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

• I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
• I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.