Skip to content

fix smoothL1Loss#209

Merged
davidkoski merged 2 commits intoml-explore:mainfrom
jiyzhang:fix_smooth_l1_loss
Apr 10, 2025
Merged

fix smoothL1Loss#209
davidkoski merged 2 commits intoml-explore:mainfrom
jiyzhang:fix_smooth_l1_loss

Conversation

@jiyzhang
Copy link
Contributor

file: Source/MLXNN/Losses.swift

public func smoothL1Loss(
    predictions: MLXArray, targets: MLXArray, beta: Float = 1, reduction: LossReduction = .mean
) -> MLXArray {
    precondition(predictions.shape == targets.shape)

    let diff = abs(predictions - targets)
    let loss = which(diff .< beta, 0.5 * square(diff) / beta, abs(diff) - 0.5 * beta)

    return reduction.reduce(loss: loss)
}

According to the definition of smooth l1 loss

$$l = \begin{cases} 0.5(x - y)^2 / \beta, & \text{if } |x - y| < \beta \\ |x - y| - 0.5\beta, & \text{otherwise} \end{cases}$$

    let diff = predictions - targets

should be

    let diff = abs(predictions - targets)

Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
Copy link
Member

@awni awni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

@davidkoski davidkoski merged commit e18c758 into ml-explore:main Apr 10, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants