-
Notifications
You must be signed in to change notification settings - Fork 770
Unable to map torch_upsample_nearest_neighbor to core upsample, using flexible input shapes during conversion #1754
Description
🐞Describing the bug
I get the error Unable to map torch_upsample_nearest_neighbor to core upsample when I try to convert the DETR PyTorch model. I tried to go deep into the package to see that the issue is arising from the _try_get_upsample_factor function where the op.op_type is gather but the conditional checks for cast.
Stack Trace
Traceback (most recent call last):
File "test.py", line 19, in
model = ct.convert(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/_converters_entry.py", line 444, in convert
mlmodel = mil_convert(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 190, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 217, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 282, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/converter.py", line 112, in call
return load(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/load.py", line 57, in load
return _perform_torch_convert(converter, debug)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/load.py", line 96, in _perform_torch_convert
prog = converter.convert()
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/converter.py", line 300, in convert
self.torch_passes(prog)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_passes.py", line 24, in torch_passes
PASS_REGISTRYp
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/mil/passes/graph_pass.py", line 14, in call
self.apply(prog)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_upsample_to_core_upsample.py", line 35, in apply
_torch_upsample_to_core_upsample_block(f)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/mil/passes/helper.py", line 42, in wrapper
return func(*args)
File "/usr/local/lib/python3.8/dist-packages/coremltools/converters/mil/frontend/torch/ssa_passes/torch_upsample_to_core_upsample.py", line 47, in _torch_upsample_to_core_upsample_block
raise ValueError("Unable to map {} to core upsample".format(op.op_type))
ValueError: Unable to map torch_upsample_nearest_neighbor to core upsample
Python code snippet
from transformers import DetrFeatureExtractor, DetrForObjectDetection
import torch
from PIL import Image
import requests
import coremltools as ct
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
image_processor = DetrFeatureExtractor.from_pretrained("facebook/detr-resnet-50")
model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50", return_dict=False)
inputs = image_processor(images=image, return_tensors="pt")
outputs = model(**inputs)
traced_model = torch.jit.trace(model, example_inputs=inputs["pixel_values"])
model = ct.convert(
traced_model,
convert_to="mlprogram",
inputs=[ct.ImageType(shape=(1, 3, ct.RangeDim(256, 3072), ct.RangeDim(256, 3072)))]
)
System environment (please complete the following information):
- coremltools version: 6.1
- OS: Ubuntu 20.04.4 LTS
- PyTorch version: 1.10.0
- transformers version: 4.19.3