{"id":53097,"date":"2020-06-17T08:00:29","date_gmt":"2020-06-17T15:00:29","guid":{"rendered":"https:\/\/github.blog\/?p=53097"},"modified":"2021-12-20T14:49:07","modified_gmt":"2021-12-20T22:49:07","slug":"using-github-actions-for-mlops-data-science","status":"publish","type":"post","link":"https:\/\/github.blog\/ai-and-ml\/machine-learning\/using-github-actions-for-mlops-data-science\/","title":{"rendered":"Using GitHub Actions for MLOps & Data Science"},"content":{"rendered":"

Background<\/span><\/a><\/h2>\n

Machine Learning Operations<\/a> (or MLOps) enables Data Scientists to work in a more collaborative fashion, by providing testing, lineage, versioning, and historical information in an automated way.\u00a0 Because the landscape of MLOps is nascent, data scientists are often forced to implement these tools from scratch. The closely related discipline of DevOps<\/a> offers some help, however many DevOps<\/a> tools are generic and require the implementation of \u201cML awareness\u201d through custom code. Furthermore, these platforms often require disparate tools that are decoupled from your code leading to poor debugging and reproducibility.<\/p>\n

To mitigate these concerns, we have created a series of GitHub Actions that integrate parts of the data science and machine learning workflow with a software development workflow. Furthermore, we provide components and examples that automate common tasks.<\/p>\n

An Example Of MLOps Using GitHub Actions<\/span><\/a><\/h2>\n

Consider the below example of how an experiment tracking system can be integrated with GitHub Actions<\/a> to enable MLOps. In the below example, we demonstrate how you can orchestrate a machine learning pipeline to run on the infrastructure of your choice, collect metrics using an experiment tracking system, and report the results back to a pull request.<\/p>\n

\"Screenshot<\/p>\n

A screenshot of this pull request<\/a>.<\/p>\n

For a live demonstration of the above example, please see this talk<\/a>.<\/p>\n

MLOps is not limited to the above example. Due to the composability of GitHub Actions, you can stack workflows in many ways that can help data scientists. Below is a concrete example of a very simple workflow that adds links to mybinder.org<\/a> on pull requests:<\/p>\n

name: Binder\non: \n  pull_request:\n    types: [opened, reopened]\n\njobs:\n  Create-Binder-Badge:\n    runs-on: ubuntu-latest\n    steps:\n\n    - name: checkout pull request branch\n      uses: actions\/checkout@v2\n      with:\n        ref: ${{ github.event.pull_request.head.sha }}\n\n    - name: comment on PR with Binder link\n      uses: actions\/github-script@v1\n      with:\n        github-token: ${{secrets.GITHUB_TOKEN}}\n        script: |\n          var BRANCH_NAME = process.env.BRANCH_NAME;\n          github.issues.createComment({\n            issue_number: context.issue.number,\n            owner: context.repo.owner,\n            repo: context.repo.repo,\n            body: `[![Binder](https:\/\/mybinder.org\/badge_logo.svg)](https:\/\/mybinder.org\/v2\/gh\/${context.repo.owner}\/${context.repo.repo}\/${BRANCH_NAME}) 👈<\/g-emoji> Launch a binder notebook on this branch`\n          }) \n      env:\n        BRANCH_NAME: ${{ github.event.pull_request.head.ref }}\n<\/code><\/pre>\n
When the above YAML file is added to a repository\u2019s .github\/workflow<\/code> directory, pull requests can be annotated with a useful link as illustrated below [1]:<\/p>\n
\"Screenshot<\/p>\n
A Growing Ecosystem of MLOps & Data Science Actions<\/span><\/a><\/h2>\n
There is a growing number of Actions available for machine learning ops and data science. Below are some concrete examples that are in use today, categorized by topic.<\/p>\n
Orchestrating Machine Learning Pipelines<\/b>:<\/p>\n