Skip to content

cv::dnn::ClassificationModel::classify() should run SoftMax when model doesn't have SoftMax layer #21689

@UnaNancyOwen

Description

@UnaNancyOwen
System information (version)
  • OpenCV => 4.5.5
Detailed description

cv::dnn::ClassificationModel::classify() returns the maximum value in output layer of model as confidence.
In the case of model that contains SoftMax layer, confidence will be [0.0-1.0] range value.
In the case of model that not contains SoftMax layer, confidence will be varies range value from model to model.
I think SoftMax should be run in cv::dnn::ClassificationModel::classify(), if model doesn't have SoftMax layer.

  std::pair<int, float> ClassificationModel::classify(InputArray frame)
  {
      std::vector<Mat> outs;
      impl->processFrame(frame, outs);
      CV_Assert(outs.size() == 1);

+     Mat out = outs[0].reshape(1, 1);

+     const bool has_softmax = getNetwork_().getLayersCount("Softmax") ? true : false;
+     if (!has_softmax)
+     {
+         cv::Mat exp;
+         const float max = *std::max_element(out.begin<float>(), out.end<float>());
+         cv::exp((out - max), exp);
+         out = exp / cv::sum(exp)[0];
+     }

      double conf;
      cv::Point maxLoc;
-     minMaxLoc(outs[0].reshape(1, 1), nullptr, &conf, nullptr, &maxLoc);
+     minMaxLoc(out, nullptr, &conf, nullptr, &maxLoc);
      return {maxLoc.x, static_cast<float>(conf)};
  }

High Level API is used by beginners. I think it should be friendly.

Steps to reproduce
// e.g. classification model without SoftMax layer
const std::string model = "DenseNet_121.caffemodel";
const std::string config = "DenseNet_121.prototxt";
cv::dnn::ClassificationModel classification_model= cv::dnn::ClassificationModel( model, config );

auto [classid, confidence] = model.classify( image );
std::cout << classid << " (" << confidence << ")" << std::endl; // confidence range is [???-???]
// e.g. classification model with SoftMax layer
const std::string model = "squeezenet_v1.1.caffemodel";
const std::string config = "squeezenet_v1.1.prototxt";
cv::dnn::ClassificationModel classification_model= cv::dnn::ClassificationModel( model, config );

auto [classid, confidence] = model.classify( image );
std::cout << classid << " (" << confidence << ")" << std::endl; // confidence range is [0.0-1.0]
Issue submission checklist
  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues,
    forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files: videos, images, onnx, etc

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions