[tflite] export SetNumThreads to TFLite Python API #25748

freedomtan · 2019-02-14T07:59:01Z

export SetNumThreads() and add an option to set the number of threads in the label_image.py for TFLite. On PYNQ-Z1, a board with 2xCA9,

$ python tensorflow/lite/examples/python/label_image.py --num_threads 1
0.415686: 653:military uniform
0.352941: 907:Windsor tie
0.058824: 668:mortarboard
0.035294: 458:bow tie, bow-tie, bowtie
0.035294: 835:suit, suit of clothes
time:  0.691580057144

$ python tensorflow/lite/examples/python/label_image.py --num_threads 2
0.415686: 653:military uniform
0.352941: 907:Windsor tie
0.058824: 668:mortarboard
0.035294: 458:bow tie, bow-tie, bowtie
0.035294: 835:suit, suit of clothes
time:  0.412344932556

jianlijianli · 2019-02-15T21:49:36Z

added @miaout17 to the reviewer list since it involves interpreter change.

jdinkel88 · 2019-02-18T02:57:46Z

Mine built well, but I get an object has no attribute
Traceback (most recent call last):
File "./label_image.py", line 74, in
interpreter.set_num_threads(int(args.num_threads))
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter.py", line 260, in set_num_threads
return self._interpreter.SetNumThreads(i)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 97, in
getattr = lambda self, name: _swig_getattr(self, InterpreterWrapper, name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 74, in _swig_getattr
return _swig_getattr_nondynamic(self, class_type, name, 0)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 69, in _swig_getattr_nondynamic
return object.getattr(self, name)
AttributeError: type object 'object' has no attribute 'getattr'

freedomtan · 2019-02-18T03:07:01Z

@jdinkel88 maybe you forgot to install the newly built pip wheel (remember to pip3 install -U YOUR_NEWLY_BUILT_PIP_WHEEL) or ran a wrong python interpreter (make sure that the python interpreter uses the newly installed one)?

dpsm · 2019-03-08T16:18:43Z

@petewarden can you review this? RPI right now runs the apps using the python binding on single core :(

dpsm · 2019-03-08T17:57:03Z

👍

freedomtan · 2019-03-09T01:26:35Z

@miaout17 can you help review this? it seems some guys using RPI really want this.

dpsm · 2019-03-12T03:50:19Z

@miaout17 is there anyone else you can nominate as reviewer in case you don't have cycles?

dpsm

Verified on rpi that it indeed uses more than a single core

tensorflow/lite/python/interpreter.py

export SetNumThreadS() and add an option to set the number of threads in the `label_image.py` for TFLite.

jdduke · 2020-05-26T17:10:31Z

tensorflow/lite/python/interpreter.py

        [TfLiteDelegate](https://www.tensorflow.org/lite/performance/delegates)
          objects returned by lite.load_delegate().
+      num_threads: Set the number of threads used by TFLite kernels.
+        If not set, kernels are running single-threaded. Note that currently,


This should really be:

"Sets the number of threads used by the interpreter and available to CPU kernels. If not set, the interpreter will use an implementation-dependent default number of threads. Currently, only a subset of kernels, such as conv, support multi-threading."

jdduke · 2020-05-26T17:11:16Z

tensorflow/lite/examples/python/label_image.py

      help='input standard deviation')
+  parser.add_argument(
+      '--num_threads',
+      default=1, type=int,


Would prefer that the default be None, if possible?

rthadur · 2020-06-05T17:21:54Z

@jdduke gentle ping if we need more changes here.

jdduke

Can you add a test or two in interpreter_test.py? Validate that it throws an error for invalid input, and that 2 threads works as expected?

srjoglekar246 · 2020-06-11T21:41:13Z

I can add the tests internally, so that we don't have to reimport the changes. Will do it by tomorrow and get this in.

metanav · 2020-06-13T01:56:22Z

How can I make a pip package to test it on RPI 4?

teijeong · 2020-06-15T13:27:02Z

@metanav Would you try this? https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/pip_package

googlebot added the cla: yes label Feb 14, 2019

pragyaak self-assigned this Feb 14, 2019

pragyaak added the size:S CL Change Size: Small label Feb 14, 2019

pragyaak requested a review from jianlijianli February 14, 2019 18:19

pragyaak added the awaiting review Pull request awaiting review label Feb 14, 2019

jianlijianli requested a review from miaout17 February 15, 2019 21:47

tensorflowbutler removed the awaiting review Pull request awaiting review label Feb 16, 2019

freedomtan mentioned this pull request Feb 16, 2019

Tensorflow Lite, python API does not work #21574

Closed

This was referenced Feb 16, 2019

it hangs after a while (RPI3B) PINTO0309/Tensorflow-bin#2

Closed

TF lite has link error when I try to build with build_rpi_lib.sh #25120

Closed

pragyaak added the awaiting review Pull request awaiting review label Feb 19, 2019

pragyaak assigned rthadur and unassigned pragyaak Feb 23, 2019

dpsm previously approved these changes Mar 18, 2019

View reviewed changes

rthadur requested review from jdduke and removed request for jianlijianli and miaout17 March 18, 2019 20:31

tensorflowbutler removed the awaiting review Pull request awaiting review label Mar 19, 2019

jdduke requested changes Mar 19, 2019

View reviewed changes

tensorflow/lite/python/interpreter.py Show resolved Hide resolved

jdduke requested a review from miaout17 March 19, 2019 18:32

freedomtan and others added 2 commits March 21, 2019 14:47

[tflite] export SetNumThreads to TFLite Python API

61d8cb0

export SetNumThreadS() and add an option to set the number of threads in the `label_image.py` for TFLite.

add simple description of set_num_threads()

7dba4b1

freedomtan dismissed dpsm’s stale review via 7dba4b1 March 21, 2019 06:49

kokoro-team removed the kokoro:force-run Tests on submitted change label May 22, 2020

jdduke requested changes May 26, 2020

View reviewed changes

address review comments

fd546da

freedomtan dismissed stale reviews from gbaned and teijeong via fd546da May 27, 2020 07:33

google-ml-butler bot removed the ready to pull PR ready for merge process label May 27, 2020

rthadur approved these changes May 27, 2020

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels May 27, 2020

kokoro-team removed the kokoro:force-run Tests on submitted change label May 27, 2020

gbaned added ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels May 28, 2020

rthadur removed the ready to pull PR ready for merge process label Jun 1, 2020

rthadur approved these changes Jun 1, 2020

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jun 1, 2020

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 1, 2020

jdduke approved these changes Jun 1, 2020

View reviewed changes

google-ml-butler bot added the kokoro:force-run Tests on submitted change label Jun 1, 2020

kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 1, 2020

gbaned added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jun 9, 2020

jdduke requested changes Jun 11, 2020

View reviewed changes

tensorflow-copybara merged commit ad4363b into tensorflow:master Jun 12, 2020

ArktinenKarpalo mentioned this pull request Jun 22, 2020

tensorflow lite käyttää vain 1 thread, thread määrä konfiguroitavaksi CACDAR-Project/Ohtuprojekti-kesa2020#98

Open

bafu mentioned this pull request Jul 19, 2020

Failed to manually run bnpipeline on x86_64 DT42/AIKEA#4

Open

TylerADavis mentioned this pull request Nov 3, 2020

Add tflite_runtime 2.3.1 wheels to Python quickstart #44100

Closed

[tflite] export SetNumThreads to TFLite Python API #25748

[tflite] export SetNumThreads to TFLite Python API #25748

Uh oh!

Conversation

freedomtan commented Feb 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jianlijianli commented Feb 15, 2019

Uh oh!

jdinkel88 commented Feb 18, 2019

Uh oh!

freedomtan commented Feb 18, 2019

Uh oh!

dpsm commented Mar 8, 2019

Uh oh!

dpsm commented Mar 8, 2019

Uh oh!

freedomtan commented Mar 9, 2019

Uh oh!

dpsm commented Mar 12, 2019

Uh oh!

dpsm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jdduke May 26, 2020

Choose a reason for hiding this comment

Uh oh!

freedomtan May 27, 2020

Choose a reason for hiding this comment

Uh oh!

jdduke May 26, 2020

Choose a reason for hiding this comment

Uh oh!

freedomtan May 27, 2020

Choose a reason for hiding this comment

Uh oh!

rthadur commented Jun 5, 2020

Uh oh!

jdduke left a comment

Choose a reason for hiding this comment

Uh oh!

srjoglekar246 commented Jun 11, 2020

Uh oh!

metanav commented Jun 13, 2020

Uh oh!

teijeong commented Jun 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

freedomtan commented Feb 14, 2019 •

edited

Loading