Add tutorials, examples, and guides for the Transformers Flavor#10360
Add tutorials, examples, and guides for the Transformers Flavor#10360BenWilson2 merged 9 commits intomlflow:masterfrom
Conversation
|
Documentation preview for 5575c89 will be available here when this CircleCI job completes successfully. More info
|
16d66da to
f60152a
Compare
|
This font is a bit difficult to read (for me). Can we use a different font? 
|
| "\n", | ||
| "generation_pipeline = transformers.pipeline(\n", | ||
| " task=task,\n", | ||
| " model=\"declare-lab/flan-alpaca-large\",\n", |
There was a problem hiding this comment.
Unfortunately, the smaller version of the flan-alpaca model is only suitable for CI testing. The results that are returned from the 2 smaller models are so bad that it makes the tutorial look like it's broken. There are 4 larger models than this one which provide dramatically improved results, but they're so large that the runtime on CPU for executing the tutorial is measured in tens of minutes. This version of the model is a fairly good compromise for size vs. ability
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Formatting Predictions for Tutorial Readability\n", | ||
| "Please note that the following function, `format_predictions`, is designed specifically for enhancing the readability of model predictions within this Jupyter Notebook environment. It **is not a standard component** of the model's inference pipeline and is **not necessary for the actual usage** of the model in production or other applications.\n", | ||
| "\n", | ||
| "The function takes a list of prediction strings and formats each one by:\n", | ||
| "\n", | ||
| "- Splitting the prediction text into individual sentences.\n", | ||
| "- Ensuring that each sentence is properly stripped of leading/trailing whitespace and ends with a period.\n", | ||
| "- Joining the sentences back together with newline characters for clear visual separation.\n", | ||
| "\n", | ||
| "This formatting is particularly useful in a notebook setting where we want to present the output in a more readable and presentable manner for demonstration purposes." | ||
| ] | ||
| }, |
There was a problem hiding this comment.
Do we need this explanation? It seems like what format_predictions does is simple string manipulation. I don't think we need cover details for that. IMO, just stating "Format the output nicely" should be fine.
There was a problem hiding this comment.
Good call. Removed everything but the first 2 sentences (and shortened them).
| "\n", | ||
| "Welcome to our first tutorial on leveraging the power of **Transformers** with **MLflow**. This tutorial is designed for beginners who are just getting started with machine learning workflows and model management. We will walk you through a simple example that demonstrates the integration of the Transformers library with MLflow, a platform that simplifies the machine learning lifecycle.\n", | ||
| "\n", | ||
| "### What are Transformers?\n", |
There was a problem hiding this comment.
do we need to explain what transformers are?
There was a problem hiding this comment.
I'm making no assumptions about users knowledge with any introduction notebook flavor. That being said, that's the only place such a high-level explanation should happen within a collection of tutorials (I made sure that the only "what is this thing that we're learning about in this tutorial" for the more advanced tutorials in transformers don't have a 'what is transformers' entry). The other tutorials focus on things like "What is Whisper?" in this spot.
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Setting the tracking server\n", |
There was a problem hiding this comment.
Can we remove this section? For demonstrating purposes, I think we can just use the default local file store.
There was a problem hiding this comment.
We have a follow-on PR to replace this cell in all notebooks with:
- Explain how to connect to hosted MLflow (Databricks CE)
- Explain how to connect to self-hosted tracking server
- Explain how to use local mode if you don't want to start a tracking server.
I'll be updating all notebook cells with the same descriptive cell to maintain consistency next sprint.
There was a problem hiding this comment.
Can we explain these in a different section because they are unrelated to transformers?
| "\n", | ||
| "In this tutorial, we delve into the world of language translation by leveraging the power of [Transformers](https://huggingface.co/docs/transformers/model_doc/whisper) and MLflow. This guide is crafted for practitioners with a grasp of machine learning concepts who seek to streamline their translation model workflows. We will showcase the use of MLflow to log, manage, and serve a cutting-edge translation model - the `google/flan-t5-base` from the [🤗 Hugging Face](https://huggingface.co/) library.\n", | ||
| "\n", | ||
| "### What is `flan-t5-base`?\n", |
There was a problem hiding this comment.
Do we need to explain this? Can you just provide a link to the huggingface hub?
There was a problem hiding this comment.
Good idea. With an arbitrary model like this, the external link is better. I'll remove this section and insert a link
B-Step62
left a comment
There was a problem hiding this comment.
Thanks a lot for really ellaborate work for the refactoring and tutorials! I love the contents and detailed navigation, just added some personal suggestions to make it visually consice:)
| <article class="simple-grid"> | ||
| <div class="simple-card"> | ||
| <a href="transformers/index.html"> | ||
| <div class="header"> |
There was a problem hiding this comment.
super-nit: On my browser, the card is displayed w/ wide space between header and description. Is it intended?
There was a problem hiding this comment.
That was a design request from UX to ensure that all cards have their <p> content start at the same height with an offset from the border of the <h2> at the top.
I think we'll be refining the CSS for this, though. (In a later PR :) )
| - **Framework Interoperability**: Flexibility to train models in one framework and deploy in another, supporting PyTorch, TensorFlow, and JAX, with additional export options for ONNX and TorchScript. | ||
| - **Community**: A highly active and engaged community for collaboration and support through forums and the Hugging Face Hub. | ||
|
|
||
| MLflow's Transformers flavor is designed to harness the power of the Transformers library within the MLflow ecosystem, providing tools for: |
There was a problem hiding this comment.
Reading down from here might be a bit verbose, as (1) this bullet points (2) "Features" section (3) intro of "MLflow Flavor for Transformers" below, appears to repeat similar things. Can we consolidate these into one?
There was a problem hiding this comment.
Good call. Dramatically simplifed these sections and condensed the information for a faster read.
| ^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Once a model is trained, it needs to be deployed for inference. MLflow's integration with Transformers simplifies this by providing | ||
| functions such as :py:func:`mlflow.transformers.load_model` and :py:func:`mlflow.pyfunc.load_model`, which allow for easy model serving. |
There was a problem hiding this comment.
Do you think it's nice to have a link to specific section in the detailed guide, or/and a list of steps for serving saved transformer model with code snippet? At this point some ppl may wonder "it says easy but how to do that?"
There was a problem hiding this comment.
good idea! Added one for loading as a pyfunc (to the guide) and one for inference configurations for scalable serving.
| </article> | ||
| </section> | ||
|
|
||
| Download the Tutorial Notebooks to try them locally |
There was a problem hiding this comment.
nit: Can we move these links to each notebook? I doublt ppl use these buttons w/o opening each tutorial.
There was a problem hiding this comment.
Unfortunately, we can't render these or properly trigger a local notebook download from within the Jupyter notebook render. nbsphinx (the plugin for sphinx that renders the .ipynb as .html) does not allow for custom JS or CSS to propagate to the rendered html (I tried). In order for us to actually have a notebook download to local on all browsers, we have to specify some custom JS to override the settings that the server (s3 website hosting) specifies for how to handle specific file types (otherwise, it will, by browser default, render the raw text of the notebook content directly in the browser).
We might be able to come up with something different with a change to a different platform for html generation, but for now, we have to have some external to the notebook rendering mechanism that generates a link / button to download the files.
| To see an example of how to fine-tune a model and log the results in MLflow in an end-to-end example of creating a spam classifier pipeline, | ||
| see the `fine-tuning tutorial <tutorials/fine-tuning/transformers-fine-tuning.html>`_. | ||
|
|
||
| For a frame of reference of what the experience is within MLflow for fine-tuning a transformers foundation model, the image below shows the |
There was a problem hiding this comment.
Could you clarify why we need two MLflow Runs for a single fine-tuning task? If I understand correctly, the green one is produced by # Log the pipeline part of the tutorial, but I'm not fully sure why we have to start a separate run for it, instead of within the training run which I think is more intuitive...
If having two runs is our recommended practice, we can show training run alone here and move the other run to deployment section and explain the reason behind that, cuz it's not directly relevant to experiment tracking, rather more productionization purpose. WDYT?
There was a problem hiding this comment.
It's not really a recommended task, per se. The original intention was to reduce the complexity. Splitting the runs up like this is fairly confusing, though. I unified them and updated the screen capture visualization as well :) Good idea
| "- **Reproducibility**: Record everything needed to reproduce a model prediction, from data preprocessing to model parameters.\n", | ||
| "- **Deployment**: Streamline the process of deploying your Transformers models to production.\n", | ||
| "\n", | ||
| "### What Will You Learn?\n", |
There was a problem hiding this comment.
nit: I love this kind of "objective" section, but would be even nicer if it's shown on top of the page, because I can quickly decide whether or not the tutorial is for me or not.
There was a problem hiding this comment.
Great call out. Updated all of the notebooks.
| "- For the purposes of generating a signature later on, as well as having a visual indicator of the expected input data to our model when loading as a `pyfunc`, we next set up an input_example that contains sample prompts.\n", | ||
| "- Finally, we define parameters that will be used to control the behavior of the model during inference, such as the maximum length of the generated text and whether to sample multiple times.\n", | ||
| "\n", | ||
| "### Understanding Pipelines\n", |
There was a problem hiding this comment.
What about collapse those introductory sections? I think we should have these description for Pipelines, Signatures, etc, for ppl who directly jumped into the tutorial, but it takes larger space than the main task.
<h1>Understanding Pipelines</h1>
<span>Pipelines are a high-level abstraction provided by the Transformers library that simplifies the process of using models for inference. They encapsulate the complexity of the underlying code, offering a straightforward API for a variety of tasks, such as text classification, question answering, and in our case, text generation. </span><br/><br/>
<details>
<summary>
<b>Expand to learn about Transformer Pipelines.</b>
</summary>
<h2>The pipeline() function</h2>
<span>The pipeline() function is a versatile tool that can be used to create a pipeline for any supported task. When you specify a task, the function returns a pipeline object tailored for that task, constructing the required calls to sub-components (a tokenizer, encoder, generative model, etc.) in the order needed to fulfill the needs of the specified task. This abstraction dramatically simplifies the code required to use these models and their respective components.</span>
</details>
There was a problem hiding this comment.
This is an AWESOME IDEA! thank you so much for this. Updating the tutorials to make the notebooks cleaner :)
| @@ -0,0 +1,1096 @@ | |||
| { | |||
There was a problem hiding this comment.
It seems the notebook layout becomes a bit noisy on browser, due to random spacing makes it hard to visually capture the section structure. Is there any easy way we can clean this up?
There was a problem hiding this comment.
I think it's the styling of our template with how it formats the different elements. I'm going to only use h2 at the top of the notebook and increment the header sections to try and make it more visually appealing
| .. toctree:: | ||
| :maxdepth: 2 | ||
| :hidden: | ||
|
|
||
| tutorials/text-generation/text-generation.ipynb | ||
| tutorials/audio-transcription/whisper.ipynb | ||
| tutorials/translation/component-translation.ipynb | ||
| tutorials/conversational/conversational-model.ipynb | ||
| tutorials/fine-tuning/transformers-fine-tuning.ipynb |
There was a problem hiding this comment.
Any notebooks that can be reviewed in separate PRs to make it easier to review them?
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
41b6167 to
ef2d45f
Compare
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
| " <ul>\n", | ||
| " <li><strong>Import</strong>: We import the necessary libraries: transformers for building our NLP model and mlflow for model tracking and management.</li>\n", | ||
| " <li><strong>Task Definition</strong>: We then define the task for our pipeline, which in this case is <code>text2text-generation`</code> This task involves generating new text based on the input text.</li>\n", | ||
| " <li><strong>Pipeline Declaration</strong>: Next, we create a generation_pipeline using the <code>pipeline</code> function from the Transformers library. This pipeline is configured to use the <code>declare-lab/flan-alpaca-base model</code>, which is a pre-trained model suitable for text generation.</li>\n", |
There was a problem hiding this comment.
| " <li><strong>Pipeline Declaration</strong>: Next, we create a generation_pipeline using the <code>pipeline</code> function from the Transformers library. This pipeline is configured to use the <code>declare-lab/flan-alpaca-base model</code>, which is a pre-trained model suitable for text generation.</li>\n", | |
| " <li><strong>Pipeline Declaration</strong>: Next, we create a generation_pipeline using the <code>pipeline</code> function from the Transformers library. This pipeline is configured to use the <code>declare-lab/flan-alpaca-large model</code>, which is a pre-trained model suitable for text generation.</li>\n", |
B-Step62
left a comment
There was a problem hiding this comment.
LGTM!
QQ: Can we re-run notebooks and deploy the changes after MLflow 2.9.0 release? Not a blocker, but those annoying pydantic warnings should be gone in the new version.
| " input_example,\n", | ||
| " mlflow.transformers.generate_signature_output(generation_pipeline, input_example),\n", | ||
| " parameters,\n", | ||
| ")" |
There was a problem hiding this comment.
| ")" | |
| ")\n" | |
| "signature" |
| " \"Please tell me a joke about hiking.\",\n", | ||
| " ],\n", | ||
| " params={\"temperature\": 0.7},\n", | ||
| ")" |
There was a problem hiding this comment.
| ")" | |
| ")\n" | |
| "\n" | |
| "# Format each prediction for notebook readability\n", | |
| "formatted_predictions = format_predictions(predictions)\n", | |
| "\n", | |
| "for i, formatted_text in enumerate(formatted_predictions):\n", | |
| " print(f\"Response to prompt {i+1}:\\n{formatted_text}\\n\")" |
we explained formatting in previous cell already, so I think it's ok to just show it here (like we do in the advanced tutorials)
There was a problem hiding this comment.
great point. Removed that extra markdown cell and concatenated the code cells :)
| "name": "stderr", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n", |
There was a problem hiding this comment.
nit: Can we remove this warning? It's shown many times across the notebooks.
There was a problem hiding this comment.
Yep! I have added warnings silencing for all UserWarnings (pydantic, setuptools) and added the fix for the tokenizer parallelization warning.
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
The warnings have been removed :) |
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>
🛠 DevTools 🛠
Install mlflow from this PR
Checkout with GitHub CLI
Related Issues/PRs
#xxxWhat changes are proposed in this pull request?
Refactor the content in models.rst to move the transformers information to an "in-depth" guide.
Migrate the existing examples to notebook-based tutorials
Provide an overview landing page for the flavor that explains the flavor at a high level
How is this PR tested?
Does this PR require documentation update?
Release Notes
Is this a user-facing change?
Overhaul the transformers flavor docs and add use-case tutorials
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/artifacts: Artifact stores and artifact loggingarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesarea/examples: Example codearea/gateway: AI Gateway service, Gateway client APIs, third-party Gateway integrationsarea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models: MLmodel format, model serialization/deserialization, flavorsarea/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templatesarea/projects: MLproject format, project running backendsarea/scoring: MLflow Model server, model deployment tools, Spark UDFsarea/server-infra: MLflow Tracking server backendarea/tracking: Tracking Service, tracking client APIs, autologgingInterface
area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows: Windows supportLanguage
language/r: R APIs and clientslanguage/java: Java APIs and clientslanguage/new: Proposals for new client languagesIntegrations
integrations/azure: Azure and Azure ML integrationsintegrations/sagemaker: SageMaker integrationsintegrations/databricks: Databricks integrationsHow should the PR be classified in the release notes? Choose one:
rn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notes