[docs] Update CPU/GPU inference docs by stevhliu · Pull Request #26881 · huggingface/transformers

stevhliu · 2023-10-17T20:19:34Z

Implements the proposal suggested in #26723, namely:

consolidating the inference on one/many GPUs docs into one page
removing the inference on specialized hardware page
cleanups and updates to the inference docs to provide more context about the how/what of the optimization techniques, code examples so users don't have to skip around to other pages, etc.

HuggingFaceDocBuilderDev · 2023-10-20T23:09:48Z

The documentation is not available anymore as the PR was closed or merged.

MKhalusova

Great work on making the inference docs easier to follow. I found a few typos, but otherwise looks great to me.

docs/source/en/perf_infer_cpu.md

docs/source/en/perf_infer_gpu_one.md

docs/source/en/perf_infer_cpu.md

LysandreJik · 2023-10-24T08:59:55Z

@lvwerra did you participate in writing the original document as well? Do you have feedback on this refactor?

younesbelkada

Thanks for updating CPU/GPU inference docs! I left few comments, what do you think?

docs/source/en/perf_infer_cpu.md

docs/source/en/perf_infer_gpu_one.md

regisss · 2023-10-24T09:53:59Z

Nice overhaul! I think it's also an opportunity to mention Optimum as this falls right into what we are doing:

I think the GPU inference page should briefly mention inference with ORT and link to: https://huggingface.co/docs/optimum/onnxruntime/usage_guides/gpu
Same for CPU with a link to: https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models
And for Intel CPUs I would also refer to Optimum Intel: https://huggingface.co/docs/optimum/intel/index

WDYT?

Cc @echarlaix @mfuntowicz

lvwerra · 2023-10-24T12:36:40Z

@lvwerra did you participate in writing the original document as well? Do you have feedback on this refactor?

Only did the general structure of that part so leave it to others to comment on content :)

echarlaix · 2023-10-24T13:14:12Z

Nice overhaul! I think it's also an opportunity to mention Optimum as this falls right into what we are doing:

I think the GPU inference page should briefly mention inference with ORT and link to: https://huggingface.co/docs/optimum/onnxruntime/usage_guides/gpu

Same for CPU with a link to: https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models

And for Intel CPUs I would also refer to Optimum Intel: https://huggingface.co/docs/optimum/intel/index

WDYT?

Cc @echarlaix @mfuntowicz

Totally aligned with your suggestion @regisss, let us know if we can help @stevhliu

stevhliu · 2023-10-24T21:20:48Z

Thanks for the feedback y'all!

Let me know what you think of the new Optimum sections @regisss @echarlaix :)

regisss

It looks great, thanks a lot @stevhliu 🙂

docs/source/en/perf_infer_cpu.md

echarlaix

It's really good, thanks a lot @stevhliu 🔥

docs/source/en/perf_infer_gpu_one.md

Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>

LysandreJik

Looks good! Only left one comment related to redirection

LysandreJik · 2023-10-26T10:07:41Z

docs/source/en/perf_infer_gpu_many.md

If this file is deleted, it should redirect to another page with similar information. These pages are linked elsewhere so we should make sure following the previous links don't lead to 404s

Should be fixed, it was only linked in the performance.md doc!

I meant that it could have been linked in tweets/blogs/things out of our control. I had a feeling we once redirected deleted pages to their updated counterparts; would it be feasible here? Maybe a question to @mishig25 regarding the doc-builder

Yes, it's possible writing a _redirects.yml (at the same level as _toctree.yml), here is an example in Optimum: https://github.com/huggingface/optimum/blob/main/docs/source/_redirects.yml
It's quite simple, the old link is to the left of the colon and the new one to the right.

That's really cool, I didn't know that was possible! I added a _redirects.yml for Transformers, let me know if there is anything else I need to add for it to work 🤗

It looks good to me, that should work!

LysandreJik

Thanks for your work @stevhliu!

* first draft * remove non-existent paths * edits * feedback * feedback and optimum * Apply suggestions from code review Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> * redirect to correct doc * _redirects.yml --------- Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>

stevhliu added 2 commits October 17, 2023 13:11

first draft

de0fcbe

remove non-existent paths

e8f317c

stevhliu requested review from LysandreJik, MKhalusova and younesbelkada October 17, 2023 21:50

edits

3cfd800

MKhalusova approved these changes Oct 23, 2023

View reviewed changes

feedback

3a928b2

younesbelkada approved these changes Oct 24, 2023

View reviewed changes

feedback and optimum

36ff545

regisss approved these changes Oct 25, 2023

View reviewed changes

docs/source/en/perf_infer_cpu.md Outdated Show resolved Hide resolved

echarlaix approved these changes Oct 25, 2023

View reviewed changes

docs/source/en/perf_infer_gpu_one.md Outdated Show resolved Hide resolved

docs/source/en/perf_infer_gpu_one.md Outdated Show resolved Hide resolved

Apply suggestions from code review

5e335b9

Co-authored-by: regisss <15324346+regisss@users.noreply.github.com> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>

stevhliu requested a review from ArthurZucker October 25, 2023 16:59

LysandreJik reviewed Oct 26, 2023

View reviewed changes

stevhliu added 2 commits October 26, 2023 10:57

redirect to correct doc

c3ceffd

_redirects.yml

49d3c80

LysandreJik approved these changes Oct 31, 2023

View reviewed changes

Merge branch 'main' into optimize-inference-docs

40a1b18

stevhliu merged commit 77930f8 into huggingface:main Oct 31, 2023

stevhliu deleted the optimize-inference-docs branch October 31, 2023 16:44

Conversation

stevhliu commented Oct 17, 2023

Uh oh!

HuggingFaceDocBuilderDev commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MKhalusova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LysandreJik commented Oct 24, 2023

Uh oh!

younesbelkada left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

regisss commented Oct 24, 2023

Uh oh!

lvwerra commented Oct 24, 2023

Uh oh!

echarlaix commented Oct 24, 2023

Uh oh!

stevhliu commented Oct 24, 2023

Uh oh!

regisss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

echarlaix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LysandreJik left a comment

Choose a reason for hiding this comment

Uh oh!

LysandreJik Oct 26, 2023

Choose a reason for hiding this comment

Uh oh!

stevhliu Oct 26, 2023

Choose a reason for hiding this comment

Uh oh!

LysandreJik Oct 27, 2023

Choose a reason for hiding this comment

Uh oh!

regisss Oct 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stevhliu Oct 30, 2023

Choose a reason for hiding this comment

Uh oh!

regisss Oct 30, 2023

Choose a reason for hiding this comment

Uh oh!

LysandreJik left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

HuggingFaceDocBuilderDev commented Oct 20, 2023 •

edited

Loading

younesbelkada left a comment •

edited

Loading

regisss Oct 27, 2023 •

edited

Loading