Skip to content

Unable to map torch_upsample_nearest_neighbor to core upsample, using flexible input shapes during conversion #1754

@scirop

Description

@scirop

🐞Describing the bug

I get the error Unable to map torch_upsample_nearest_neighbor to core upsample when I try to convert the DETR PyTorch model. I tried to go deep into the package to see that the issue is arising from the _try_get_upsample_factor function where the op.op_type is gather but the conditional checks for cast.

Stack Trace

Traceback (most recent call last):
File "test.py", line 19, in
model = ct.convert(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/_converters_entry.py", line 444, in convert
mlmodel = mil_convert(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 190, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 217, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 282, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 112, in call
return load(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/load.py", line 57, in load
return _perform_torch_convert(converter, debug)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/load.py", line 96, in _perform_torch_convert
prog = converter.convert()
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/converter.py", line 300, in convert
self.torch_passes(prog)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_passes.py", line 24, in torch_passes
PASS_REGISTRYp
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/mil/passes/graph_pass.py", line 14, in call
self.apply(prog)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_upsample_to_core_upsample.py", line 35, in apply
_torch_upsample_to_core_upsample_block(f)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/mil/passes/helper.py", line 42, in wrapper
return func(*args)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_upsample_to_core_upsample.py", line 47, in _torch_upsample_to_core_upsample_block
raise ValueError("Unable to map {} to core upsample".format(op.op_type))
ValueError: Unable to map torch_upsample_nearest_neighbor to core upsample

Python code snippet

from transformers import DetrFeatureExtractor, DetrForObjectDetection
import torch
from PIL import Image
import requests
import coremltools as ct

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

image_processor = DetrFeatureExtractor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50", return_dict=False)

inputs = image_processor(images=image, return_tensors="pt")
outputs = model(**inputs)


traced_model = torch.jit.trace(model, example_inputs=inputs["pixel_values"])

model = ct.convert(
    traced_model,
    convert_to="mlprogram",
    inputs=[ct.ImageType(shape=(1, 3, ct.RangeDim(256, 3072), ct.RangeDim(256, 3072)))]
)

System environment (please complete the following information):

  • coremltools version: 6.1
  • OS: Ubuntu 20.04.4 LTS
  • PyTorch version: 1.10.0
  • transformers version: 4.19.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugUnexpected behaviour that should be corrected (type)triagedReviewed and examined, release as been assigned if applicable (status)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions