Dolphin

Dolphin Network – The Distributed Inference Network

Dolphin Network is a distributed inference network for hosting of various AI models.

The goal of Dolphin Network is to allow gamers & other GPU owners to repurpose their GPUs during idle periods by powering a portion of our inference network.

By processing a portion of the network's requests, they are able to earn $POD tokens which can later be used for inference or sold on the market to recoup the cost of the GPU.

We think this is one of the best applications of AI x dePIN as:

1. Demand for AI inference is very high.

2. Supply of idle consumer GPUs is also very high as many people have high power GPUs sitting idle > 95% of the time as they only use them while gaming.

   Many of these GPUs can provide inference speeds & quality levels that would be almost indistinguishable from centralized servers.

   We believe we take advantage of this to provide cheaper inference than centralised providers given that many node operators already own their GPU outright & only need to pay for the electricity cost making them less price sensitive compared to commercial operations.

3. The connection between demand & supply does not depend on location. Unlike other dePIN networks that depend on the user being in close proximity to the provider, an AI inference consumer can access compute from across the globe, with ping time having a minimal impact on usability.

Inference Integrity Verification

The main problem faced by a Decentralized Inference Network is being able to prevent malicious nodes from modifying the node software to cheat the system & gain an unfair advantage over other operators in the network.

This could include modification of the node software to…

• Not actually run inference for each request & instead send out pre-generated text
• Use smaller models or higher levels of quantization to increase processing speed which decreases response quality & predictability for inference users

Dolphin Network uses software integrity checks, validator nodes & cryptoeconomic bond deposits to prevent fraud in the network so inference consumers can be confident that they are interfacing with the AI model they have selected at the quality level advertised.

1. Node Software Integrity Verification Methods

Encrypted Binaries

All nodes on Dolphin Network run a binary that is encoded using a robust encryption algorithm, ensuring that its contents remain secure and inaccessible to unauthorized parties.

This encryption process involves converting the binary data into an unreadable format, which can only be decoded by entities that possess the corresponding private key.

This significantly reduces the risk of reverse engineering, unauthorized modifications, and tampering with the binary code.

Signed Binaries

After the encrypted binary has been created, the binary is signed by our private key.

If the binary has been altered since it was signed, the signature check will fail & the node running the modified binary will not be allowed to join the network.

Code Obfuscation

Code Obfuscation techniques are employed in the Dolphi binary to further enhance the security of the network.

This method involves transforming the executable code into a format that is challenging for humans to understand or reverse engineer, while still being fully functional.

This process helps protect against unauthorized analysis and modification of the code by malicious actors who may attempt to modify the software to gain an unfair advantage over other nodes.

Model Integrity Verification

Each model is assigned a unique code, known as a checksum which is generated through an algorithm.

Checksums act like a digital signature for the model and are utilized to confirm the integrity and authenticity of each model in the network.

When a node receives a model, it recalculates and compares this checksum to the original to verify that the model has not been altered or corrupted in transit.

This mechanism not only confirms the model's integrity at deployment but also continuously ensures that each node is running the correct model.

Hardware Identification & Utilization Metrics

Hardware identifiers & utilization metrics relating to the machine powering each node instance are monitored to verify inference is being run exclusively on GPUs & to prevent spoofing or duplication of node instances.

2. Inference Validation

The quality of our network is upheld by Validators that verify all inference requests to ensure that they pass our various benchmarks & that inference responses are being generated as expected.

Validators perform multiple verification checks on each response:

• Tokenized Output Verification: The validator tokenizes the response text and compares the token count against what the worker claims, ensuring the model uses the same tokenizer as the expected model. Token counts must match within a 25% threshold to account for network latency.
• Performance Verification: We enforce minimum tokens per second bounds that are expected for each model based on size and typical speeds. Workers that fall below the configured thresholds fail validation.
• Logprobs Verification via Trusted Validator: Workers return top 5 logprobs for each generated token. The validator sends these claimed logprobs along with the input and output token IDs to a trusted validator worker, which computes logprobs independently using the same model and verifies they match within a tolerance threshold (0.01 logprob difference to account for different hardware). This ensures workers are actually running the expected model, as different models produce different probability distributions over tokens.

While the outputs for each model are not perfectly deterministic, these verification checks allow us to effectively assess whether the expected model is being run correctly across all nodes in the network.

3. Cryptoeconomic Verification Methods

Node operators have the option to bond staked $POD tokens at the account level for nodes they are running.

The amount needed scales with the account's recent base earnings, so larger operators need more bonded POD to keep the same boost.

In order to create a bond, users will deposit staked $POD tokens to our bonding smart contract and this becomes bonded POD attached to the account.

In return for this, they will get a boost in the $POD rewards generated from their contribution to the network.

Bond deposits are time-locked, with a 3 month cooldown needed before the collateral can be withdrawn.

This strongly aligns node operators with the value of the $POD token while also making it economically irrational to cheat given that they would be liable to have a meaningful account-level bond slashed in any confirmed case of malicious activity.

This mechanism creates a class of nodes owned by bonded accounts that have a very strong layer of economic security on top of all the technical verification that is already in place in the network.

Inference consumers have the option to route requests to all nodes in the network or exclusively to nodes owned by bonded accounts depending on what level of verification they would like for their generation requests.

Governance has the ability to set the boost multiplier ratio through on-chain voting.

We believe this ratio should remain relatively low if no cheating is detected. However, if any cheaters manage to bypass the security measures outlined earlier, this multiplier can be increased to a point where running nodes from a bonded account becomes significantly more attractive than running unbonded nodes. This approach will propagate an additional layer of security throughout the majority of the network.

Malicious Node Reporting & Dolphin Anti-cheat (DAC)

The Dolphin Network has validators that sample requests to verify that inference is being ran as expected.

If any issues are found, they may remove & or ban nodes based on a number of 'strikes' that depend on how many times the node has failed different verification checks & the severity of those failures

Users of our Web UI & Telegram bot also have the ability to report responses that they think may be generated by malicious node operators.

These nodes will be flagged for validators to further analyse.

In the case where a user successfully flags a node that is not operating correctly, they will be rewarded in staked $POD tokens for their contributions to the security & quality of the network.

If the node has a bond posted as collateral, the user will receive a portion of the slashed deposit for their role in the case.

If the node has no bond deposit posted, the user will be rewarded with tokens from the treasury.

Confirmed malicious nodes will be banned from the network to prevent them rejoining under another alias.