Refactoring part1- flags modification by nzarif · Pull Request #543 · NVIDIA-Merlin/Transformers4Rec

nzarif · 2022-11-19T01:57:10Z

Fixes #542

Remove ambiguous flags:ignore_masking and hf_format.
Introduce the testing flag to differentiate between evaluation (=True) and inference (=False) modes.

Goals ⚽

Set the model to the inference mode by default + ensure training script is not impacted + update failing unit-tests
Remove the hf-format from Next Item Prediction task + ensure training / inference are not impacted + update failing unit-tests
Replace ignore_masking with testing flag + ensure training / inference are not impacted + update failing unit-tests

Implementation Details 🚧

These changes help with standardizing T4Rec API and enable using HuggingFace Trainer for training all PredictionTasks within T4Rec.

Testing Details 🔍

Run the torch unit tests and you will see they pass. You can also try with integration tests.

nvidia-merlin-bot · 2022-11-19T01:58:29Z

Click to view CI Results

GitHub pull request #543 of commit 867e7c0a7beaa686ee06d346dd1ede298193ae6f, no merge conflicts.
Running as SYSTEM
Setting status of 867e7c0a7beaa686ee06d346dd1ede298193ae6f to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/302/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse 867e7c0a7beaa686ee06d346dd1ede298193ae6f^{commit} # timeout=10
Checking out Revision 867e7c0a7beaa686ee06d346dd1ede298193ae6f (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 867e7c0a7beaa686ee06d346dd1ede298193ae6f # timeout=10
Commit message: "fixed formatting with black"
 > git rev-list --no-walk 2bcaec66c4b1943fcfebd1e1cb7e8dd0fcbd361c # timeout=10
[transformers4rec_tests] $ /bin/bash /tmp/jenkins2822451489368058426.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/NVTabular.git
  Cloning https://github.com/NVIDIA-Merlin/NVTabular.git to /tmp/pip-req-build-11ki1k0a
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/NVTabular.git /tmp/pip-req-build-11ki1k0a
  Resolved https://github.com/NVIDIA-Merlin/NVTabular.git to commit e5b7351deb9e4885c4038aa0bbc9f146d8477a0e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-dataloader>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.0.2)
Requirement already satisfied: merlin-core>=0.2.0 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.6.0+1.g5926fcf)
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (1.8.1)
Requirement already satisfied: distributed>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.5)
Requirement already satisfied: numba>=0.54 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.56.2)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (7.0.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.64.1)
Requirement already satisfied: dask>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.19.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.5)
Requirement already satisfied: fsspec==2022.5.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.10.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (21.3)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /usr/local/lib/python3.8/dist-packages (from scipy->nvtabular==1.6.0+6.ge5b7351d) (1.22.4)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.12.0)
Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.0)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.7.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.9.2)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.26.12)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.1.2)
Requirement already satisfied: locket>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (8.1.3)
Requirement already satisfied: tornado>=6.0.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.2)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.4)
Requirement already satisfied: setuptools<60 in /usr/lib/python3/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (45.2.0)
Requirement already satisfied: llvmlite<0.40,>=0.39.0dev0 in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.39.1)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.12.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.52.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.15.0)
Requirement already satisfied: heapdict in /usr/local/lib/python3.8/dist-packages (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.1.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata->numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.8.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.1.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.1)
Building wheels for collected packages: nvtabular
  Building wheel for nvtabular (pyproject.toml): started
  Building wheel for nvtabular (pyproject.toml): finished with status 'done'
  Created wheel for nvtabular: filename=nvtabular-1.6.0+6.ge5b7351d-cp38-cp38-linux_x86_64.whl size=257597 sha256=9e36c9086b3174b3b39965f2456b0d480d73788bd60d710d03a6c13b012bab53
  Stored in directory: /tmp/pip-ephem-wheel-cache-8h8s8zpr/wheels/c2/16/76/39994bff39d812513de5b5572bff0903b9eb8f6c645b44cedc
Successfully built nvtabular
Installing collected packages: nvtabular
  Attempting uninstall: nvtabular
    Found existing installation: nvtabular 1.6.0+5.g21117cfc
    Uninstalling nvtabular-1.6.0+5.g21117cfc:
      Successfully uninstalled nvtabular-1.6.0+5.g21117cfc
Successfully installed nvtabular-1.6.0+6.ge5b7351d
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 1 item
tests/unit/test_notebooks.py F                                           [100%]
=================================== FAILURES ===================================

_________________________________ test_session _________________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-51/test_session0')
@pytest.mark.skipif(importlib.util.find_spec("cudf") is None, reason="needs cudf")
def test_session(tmpdir):
    BASE_PATH = os.path.join(dirname(TEST_PATH), SESSION_PATH)
    os.environ["INPUT_DATA_DIR"] = "/tmp/data/"
    # Run ETL
    nb_path = os.path.join(BASE_PATH, "01-ETL-with-NVTabular.ipynb")
    _run_notebook(tmpdir, nb_path)

    # Run session based
    torch = importlib.util.find_spec("torch")
    if torch is not None:
        os.environ["INPUT_SCHEMA_PATH"] = BASE_PATH + "schema.pb"
        nb_path = os.path.join(BASE_PATH, "02-session-based-XLNet-with-PyT.ipynb")


      _run_notebook(tmpdir, nb_path)


tests/unit/test_notebooks.py:44:

tests/unit/test_notebooks.py:66: in _run_notebook

subprocess.check_output([sys.executable, script_path])

/usr/lib/python3.8/subprocess.py:415: in check_output

return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

input = None, capture_output = False, timeout = None, check = True

popenargs = (['/usr/bin/python', '/tmp/pytest-of-jenkins/pytest-51/test_session0/notebook.py'],)

kwargs = {'stdout': -1}, process = <subprocess.Popen object at 0x7f5d854c3be0>

stdout = b'', stderr = None, retcode = 1
def run(*popenargs,
        input=None, capture_output=False, timeout=None, check=False, **kwargs):
    """Run command with arguments and return a CompletedProcess instance.

    The returned instance will have attributes args, returncode, stdout and
    stderr. By default, stdout and stderr are not captured, and those attributes
    will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

    If check is True and the exit code was non-zero, it raises a
    CalledProcessError. The CalledProcessError object will have the return code
    in the returncode attribute, and output & stderr attributes if those streams
    were captured.

    If timeout is given, and the process takes too long, a TimeoutExpired
    exception will be raised.

    There is an optional argument "input", allowing you to
    pass bytes or a string to the subprocess's stdin.  If you use this argument
    you may not also use the Popen constructor's "stdin" argument, as
    it will be used internally.

    By default, all communication is in bytes, and therefore any "input" should
    be bytes, and the stdout and stderr will be bytes. If in text mode, any
    "input" should be a string, and stdout and stderr will be strings decoded
    according to locale encoding, or by "encoding" if set. Text mode is
    triggered by setting any of text, encoding, errors or universal_newlines.

    The other arguments are the same as for the Popen constructor.
    """
    if input is not None:
        if kwargs.get('stdin') is not None:
            raise ValueError('stdin and input arguments may not both be used.')
        kwargs['stdin'] = PIPE

    if capture_output:
        if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
            raise ValueError('stdout and stderr arguments may not be used '
                             'with capture_output.')
        kwargs['stdout'] = PIPE
        kwargs['stderr'] = PIPE

    with Popen(*popenargs, **kwargs) as process:
        try:
            stdout, stderr = process.communicate(input, timeout=timeout)
        except TimeoutExpired as exc:
            process.kill()
            if _mswindows:
                # Windows accumulates the output in a single blocking
                # read() call run on child threads, with the timeout
                # being done in a join() on those threads.  communicate()
                # _after_ kill() is required to collect that and add it
                # to the exception.
                exc.stdout, exc.stderr = process.communicate()
            else:
                # POSIX _communicate already populated the output so
                # far into the TimeoutExpired exception.
                process.wait()
            raise
        except:  # Including KeyboardInterrupt, communicate handled that.
            process.kill()
            # We don't call process.wait() as .__exit__ does that for us.
            raise
        retcode = process.poll()
        if check and retcode:


          raise CalledProcessError(retcode, process.args,


                                     output=stdout, stderr=stderr)

E               subprocess.CalledProcessError: Command '['/usr/bin/python', '/tmp/pytest-of-jenkins/pytest-51/test_session0/notebook.py']' returned non-zero exit status 1.
/usr/lib/python3.8/subprocess.py:516: CalledProcessError

----------------------------- Captured stderr call -----------------------------

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(
Creating time-based splits:   0%|          | 0/9 [00:00<?, ?it/s]

Creating time-based splits:  11%|█         | 1/9 [00:00<00:01,  6.03it/s]

Creating time-based splits:  22%|██▏       | 2/9 [00:00<00:01,  6.98it/s]

Creating time-based splits:  33%|███▎      | 3/9 [00:00<00:00,  6.64it/s]

Creating time-based splits:  44%|████▍     | 4/9 [00:00<00:00,  7.00it/s]

Creating time-based splits:  56%|█████▌    | 5/9 [00:00<00:00,  7.44it/s]

Creating time-based splits:  67%|██████▋   | 6/9 [00:00<00:00,  7.14it/s]

Creating time-based splits:  78%|███████▊  | 7/9 [00:00<00:00,  7.52it/s]

Creating time-based splits:  89%|████████▉ | 8/9 [00:01<00:00,  7.71it/s]

Creating time-based splits: 100%|██████████| 9/9 [00:01<00:00,  7.92it/s]

Creating time-based splits: 100%|██████████| 9/9 [00:01<00:00,  7.41it/s]

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

Traceback (most recent call last):

File "/tmp/pytest-of-jenkins/pytest-51/test_session0/notebook.py", line 106, in 

tr.NextItemPredictionTask(weight_tying=True, hf_format=True,

TypeError: init() got an unexpected keyword argument 'hf_format'

============================== 1 failed in 18.95s ==============================

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[transformers4rec_tests] $ /bin/bash /tmp/jenkins14103017125413985841.sh

nvidia-merlin-bot · 2022-11-19T04:05:22Z

Click to view CI Results

GitHub pull request #543 of commit 68b8c7341750aeab9b3ba79e2414a223a192f2fa, no merge conflicts.
Running as SYSTEM
Setting status of 68b8c7341750aeab9b3ba79e2414a223a192f2fa to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/303/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse 68b8c7341750aeab9b3ba79e2414a223a192f2fa^{commit} # timeout=10
Checking out Revision 68b8c7341750aeab9b3ba79e2414a223a192f2fa (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 68b8c7341750aeab9b3ba79e2414a223a192f2fa # timeout=10
Commit message: "remove hf_format from paper script"
 > git rev-list --no-walk 867e7c0a7beaa686ee06d346dd1ede298193ae6f # timeout=10
[transformers4rec_tests] $ /bin/bash /tmp/jenkins5478061728273408644.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/NVTabular.git
  Cloning https://github.com/NVIDIA-Merlin/NVTabular.git to /tmp/pip-req-build-jkv06zka
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/NVTabular.git /tmp/pip-req-build-jkv06zka
  Resolved https://github.com/NVIDIA-Merlin/NVTabular.git to commit e5b7351deb9e4885c4038aa0bbc9f146d8477a0e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.2.0 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.6.0+1.g5926fcf)
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (1.8.1)
Requirement already satisfied: merlin-dataloader>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.0.2)
Requirement already satisfied: fsspec==2022.5.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.0)
Requirement already satisfied: distributed>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.19.5)
Requirement already satisfied: dask>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (7.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (21.3)
Requirement already satisfied: numba>=0.54 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.56.2)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.10.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.64.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.5)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /usr/local/lib/python3.8/dist-packages (from scipy->nvtabular==1.6.0+6.ge5b7351d) (1.22.4)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.12.0)
Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.4.1)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.26.12)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (8.1.3)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.1.2)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.7.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.4)
Requirement already satisfied: psutil>=5.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.9.2)
Requirement already satisfied: locket>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.0)
Requirement already satisfied: tornado>=6.0.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.2)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.4.0)
Requirement already satisfied: setuptools<60 in /usr/lib/python3/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (45.2.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.12.0)
Requirement already satisfied: llvmlite<0.40,>=0.39.0dev0 in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.39.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.15.0)
Requirement already satisfied: heapdict in /usr/local/lib/python3.8/dist-packages (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.1.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata->numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.8.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.0.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 1 item
tests/unit/test_notebooks.py F                                           [100%]
=================================== FAILURES ===================================

_________________________________ test_session _________________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-53/test_session0')
@pytest.mark.skipif(importlib.util.find_spec("cudf") is None, reason="needs cudf")
def test_session(tmpdir):
    BASE_PATH = os.path.join(dirname(TEST_PATH), SESSION_PATH)
    os.environ["INPUT_DATA_DIR"] = "/tmp/data/"
    # Run ETL
    nb_path = os.path.join(BASE_PATH, "01-ETL-with-NVTabular.ipynb")
    _run_notebook(tmpdir, nb_path)

    # Run session based
    torch = importlib.util.find_spec("torch")
    if torch is not None:
        os.environ["INPUT_SCHEMA_PATH"] = BASE_PATH + "schema.pb"
        nb_path = os.path.join(BASE_PATH, "02-session-based-XLNet-with-PyT.ipynb")


      _run_notebook(tmpdir, nb_path)


tests/unit/test_notebooks.py:44:

tests/unit/test_notebooks.py:66: in _run_notebook

subprocess.check_output([sys.executable, script_path])

/usr/lib/python3.8/subprocess.py:415: in check_output

return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

input = None, capture_output = False, timeout = None, check = True

popenargs = (['/usr/bin/python', '/tmp/pytest-of-jenkins/pytest-53/test_session0/notebook.py'],)

kwargs = {'stdout': -1}, process = <subprocess.Popen object at 0x7efb4fbee340>

stdout = b'', stderr = None, retcode = 1
def run(*popenargs,
        input=None, capture_output=False, timeout=None, check=False, **kwargs):
    """Run command with arguments and return a CompletedProcess instance.

    The returned instance will have attributes args, returncode, stdout and
    stderr. By default, stdout and stderr are not captured, and those attributes
    will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

    If check is True and the exit code was non-zero, it raises a
    CalledProcessError. The CalledProcessError object will have the return code
    in the returncode attribute, and output & stderr attributes if those streams
    were captured.

    If timeout is given, and the process takes too long, a TimeoutExpired
    exception will be raised.

    There is an optional argument "input", allowing you to
    pass bytes or a string to the subprocess's stdin.  If you use this argument
    you may not also use the Popen constructor's "stdin" argument, as
    it will be used internally.

    By default, all communication is in bytes, and therefore any "input" should
    be bytes, and the stdout and stderr will be bytes. If in text mode, any
    "input" should be a string, and stdout and stderr will be strings decoded
    according to locale encoding, or by "encoding" if set. Text mode is
    triggered by setting any of text, encoding, errors or universal_newlines.

    The other arguments are the same as for the Popen constructor.
    """
    if input is not None:
        if kwargs.get('stdin') is not None:
            raise ValueError('stdin and input arguments may not both be used.')
        kwargs['stdin'] = PIPE

    if capture_output:
        if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
            raise ValueError('stdout and stderr arguments may not be used '
                             'with capture_output.')
        kwargs['stdout'] = PIPE
        kwargs['stderr'] = PIPE

    with Popen(*popenargs, **kwargs) as process:
        try:
            stdout, stderr = process.communicate(input, timeout=timeout)
        except TimeoutExpired as exc:
            process.kill()
            if _mswindows:
                # Windows accumulates the output in a single blocking
                # read() call run on child threads, with the timeout
                # being done in a join() on those threads.  communicate()
                # _after_ kill() is required to collect that and add it
                # to the exception.
                exc.stdout, exc.stderr = process.communicate()
            else:
                # POSIX _communicate already populated the output so
                # far into the TimeoutExpired exception.
                process.wait()
            raise
        except:  # Including KeyboardInterrupt, communicate handled that.
            process.kill()
            # We don't call process.wait() as .__exit__ does that for us.
            raise
        retcode = process.poll()
        if check and retcode:


          raise CalledProcessError(retcode, process.args,


                                     output=stdout, stderr=stderr)

E               subprocess.CalledProcessError: Command '['/usr/bin/python', '/tmp/pytest-of-jenkins/pytest-53/test_session0/notebook.py']' returned non-zero exit status 1.
/usr/lib/python3.8/subprocess.py:516: CalledProcessError

----------------------------- Captured stderr call -----------------------------

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(

/usr/local/lib/python3.8/dist-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(
Creating time-based splits:   0%|          | 0/9 [00:00<?, ?it/s]

Creating time-based splits:  11%|█         | 1/9 [00:00<00:01,  6.62it/s]

Creating time-based splits:  22%|██▏       | 2/9 [00:00<00:00,  7.76it/s]

Creating time-based splits:  33%|███▎      | 3/9 [00:00<00:00,  8.15it/s]

Creating time-based splits:  44%|████▍     | 4/9 [00:00<00:00,  7.86it/s]

Creating time-based splits:  56%|█████▌    | 5/9 [00:00<00:00,  8.18it/s]

Creating time-based splits:  67%|██████▋   | 6/9 [00:00<00:00,  8.06it/s]

Creating time-based splits:  78%|███████▊  | 7/9 [00:00<00:00,  8.23it/s]

Creating time-based splits:  89%|████████▉ | 8/9 [00:01<00:00,  7.73it/s]

Creating time-based splits: 100%|██████████| 9/9 [00:01<00:00,  7.70it/s]

Creating time-based splits: 100%|██████████| 9/9 [00:01<00:00,  7.84it/s]

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

Traceback (most recent call last):

File "/tmp/pytest-of-jenkins/pytest-53/test_session0/notebook.py", line 106, in 

tr.NextItemPredictionTask(weight_tying=True, hf_format=True,

TypeError: init() got an unexpected keyword argument 'hf_format'

============================== 1 failed in 18.74s ==============================

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[transformers4rec_tests] $ /bin/bash /tmp/jenkins6450324221702761511.sh

…s4Rec into refactor_part1

review-notebook-app · 2022-11-21T20:29:03Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

nvidia-merlin-bot · 2022-11-21T20:30:25Z

Click to view CI Results

GitHub pull request #543 of commit 6eab668d82d4fb10b8570c26b03933d553844dc3, no merge conflicts.
Running as SYSTEM
Setting status of 6eab668d82d4fb10b8570c26b03933d553844dc3 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/314/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse 6eab668d82d4fb10b8570c26b03933d553844dc3^{commit} # timeout=10
Checking out Revision 6eab668d82d4fb10b8570c26b03933d553844dc3 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6eab668d82d4fb10b8570c26b03933d553844dc3 # timeout=10
Commit message: "rebased with latest main"
 > git rev-list --no-walk a560f2ea11392f7c6faa3f6e913fa6c5adfe5fd5 # timeout=10
First time build. Skipping changelog.
[transformers4rec_tests] $ /bin/bash /tmp/jenkins12489377285219681698.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/NVTabular.git
  Cloning https://github.com/NVIDIA-Merlin/NVTabular.git to /tmp/pip-req-build-8ahtblgq
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/NVTabular.git /tmp/pip-req-build-8ahtblgq
  Resolved https://github.com/NVIDIA-Merlin/NVTabular.git to commit e5b7351deb9e4885c4038aa0bbc9f146d8477a0e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.2.0 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.6.0+1.g5926fcf)
Requirement already satisfied: merlin-dataloader>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.0.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (1.8.1)
Requirement already satisfied: dask>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.10.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (21.3)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.19.5)
Requirement already satisfied: distributed>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.0)
Requirement already satisfied: numba>=0.54 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.56.2)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (7.0.0)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /usr/local/lib/python3.8/dist-packages (from scipy->nvtabular==1.6.0+6.ge5b7351d) (1.22.4)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.12.0)
Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.4.1)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.4)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.26.12)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.1.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (8.1.3)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.9.2)
Requirement already satisfied: tornado>=6.0.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.2)
Requirement already satisfied: locket>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.0)
Requirement already satisfied: setuptools<60 in /usr/lib/python3/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (45.2.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.12.0)
Requirement already satisfied: llvmlite<0.40,>=0.39.0dev0 in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.39.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.2.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.52.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.15.0)
Requirement already satisfied: heapdict in /usr/local/lib/python3.8/dist-packages (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.2)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata->numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.8.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.0.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 1 item
tests/unit/test_notebooks.py .                                           [100%]
============================== 1 passed in 36.69s ==============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[transformers4rec_tests] $ /bin/bash /tmp/jenkins715229324499677446.sh

nvidia-merlin-bot · 2022-11-21T21:12:03Z

Click to view CI Results

GitHub pull request #543 of commit f7547ba84909ec17afba852a05dcabca38d6ac02, no merge conflicts.
Running as SYSTEM
Setting status of f7547ba84909ec17afba852a05dcabca38d6ac02 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/315/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse f7547ba84909ec17afba852a05dcabca38d6ac02^{commit} # timeout=10
Checking out Revision f7547ba84909ec17afba852a05dcabca38d6ac02 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f7547ba84909ec17afba852a05dcabca38d6ac02 # timeout=10
Commit message: "make flake8 ignore line 230 of prediction_task.py"
 > git rev-list --no-walk 6eab668d82d4fb10b8570c26b03933d553844dc3 # timeout=10
[transformers4rec_tests] $ /bin/bash /tmp/jenkins9269994383968406136.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/NVTabular.git
  Cloning https://github.com/NVIDIA-Merlin/NVTabular.git to /tmp/pip-req-build-aumtvh3c
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/NVTabular.git /tmp/pip-req-build-aumtvh3c
  Resolved https://github.com/NVIDIA-Merlin/NVTabular.git to commit e5b7351deb9e4885c4038aa0bbc9f146d8477a0e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (1.8.1)
Requirement already satisfied: merlin-dataloader>=0.0.2 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.0.2)
Requirement already satisfied: merlin-core>=0.2.0 in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.6.0+1.g5926fcf)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.19.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (21.3)
Requirement already satisfied: dask>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.64.1)
Requirement already satisfied: distributed>=2022.3.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.1)
Requirement already satisfied: numba>=0.54 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.56.2)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (7.0.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.5)
Requirement already satisfied: fsspec==2022.5.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.10.0)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /usr/local/lib/python3.8/dist-packages (from scipy->nvtabular==1.6.0+6.ge5b7351d) (1.22.4)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.0)
Requirement already satisfied: pyyaml>=5.3.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.12.0)
Requirement already satisfied: locket>=1.0.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.0)
Requirement already satisfied: psutil>=5.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.9.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.1.2)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.4.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (8.1.3)
Requirement already satisfied: tornado>=6.0.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.2)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.26.12)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.4)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.7.0)
Requirement already satisfied: llvmlite<0.40,>=0.39.0dev0 in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.39.1)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.8/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.12.0)
Requirement already satisfied: setuptools<60 in /usr/lib/python3/dist-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (45.2.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.15.0)
Requirement already satisfied: heapdict in /usr/local/lib/python3.8/dist-packages (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.1.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.8/dist-packages (from importlib-metadata->numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.8.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.0.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
plugins: anyio-3.6.1, xdist-3.0.2, cov-4.0.0
collected 1 item
tests/unit/test_notebooks.py .                                           [100%]
============================== 1 passed in 36.33s ==============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[transformers4rec_tests] $ /bin/bash /tmp/jenkins9623690368324846583.sh

github-actions · 2022-11-21T21:19:45Z

Documentation preview

https://nvidia-merlin.github.io/Transformers4Rec/review/pr-543

sararb

The PR looks good to me! I also pulled it into my local machine and ran the ci tests all the scores are matching the ones we get from the main branch.

I have just left one comment to remove setting the hf_format in the base Model class.

sararb · 2022-11-22T01:11:33Z

@@ -545,12 +554,19 @@ def __init__(
        self.optimizer = optimizer
        self.hf_format = hf_format


Can we remove setting the hf_format from the model's init method? (from the model's forward method and the docstring as well) This was used to set the hf_format of NextItemPredictionTask to False after training to prepare the model for inference. But with the new refactoring introduced in this PR, the model will always return a tensor at inference (which is also the default mode when calling model(inputs))

nvidia-merlin-bot · 2022-11-22T17:13:03Z

Click to view CI Results

GitHub pull request #543 of commit ebf4bec6caa086e7492676656ec0cc86b160fef9, no merge conflicts.
Running as SYSTEM
Setting status of ebf4bec6caa086e7492676656ec0cc86b160fef9 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/327/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse ebf4bec6caa086e7492676656ec0cc86b160fef9^{commit} # timeout=10
Checking out Revision ebf4bec6caa086e7492676656ec0cc86b160fef9 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ebf4bec6caa086e7492676656ec0cc86b160fef9 # timeout=10
Commit message: "removed hf_format from all classes"
 > git rev-list --no-walk 268891f3384aa13e2468245ba7e350409e8dc3d4 # timeout=10
First time build. Skipping changelog.
[transformers4rec_tests] $ /bin/bash /tmp/jenkins16265547424821876049.sh
GLOB sdist-make: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/setup.py
py38-gpu create: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/py38-gpu
py38-gpu inst: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/.tmp/package/1/transformers4rec-0.1.14+27.gebf4bec6.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.14,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,bleach==5.0.1,boto3==1.24.75,botocore==1.29.14,Brotli==1.0.9,cachetools==5.2.0,certifi==2022.9.24,cffi==1.15.1,charset-normalizer==2.1.1,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,docker-pycreds==0.4.0,docutils==0.16,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==3.4,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,nvtabular==1.4.0+8.g95e12d347,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pbr==5.11.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.28.1,requests-oauthlib==1.3.1,rsa==4.7.2,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.16.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.44,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,tabulate==0.8.10,tblib==1.7.0,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.14+27.gebf4bec6,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='2047557956'
py38-gpu run-test: commands[0] | pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in ./.tox/py38-gpu/lib/python3.8/site-packages (22.3.1)
py38-gpu run-test: commands[1] | pip install .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tqdm>=4.27 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (4.64.1)
Requirement already satisfied: betterproto<2.0.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (1.2.5)
Requirement already satisfied: tensorflow-metadata in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (1.11.0)
Requirement already satisfied: pyarrow>=1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (10.0.0)
Requirement already satisfied: numpy>=1.17.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (1.23.5)
Requirement already satisfied: transformers<4.19 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+27.gebf4bec6) (4.18.0)
Requirement already satisfied: stringcase in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (1.2.0)
Requirement already satisfied: grpclib in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (0.4.3)
Requirement already satisfied: regex!=2019.12.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (2022.10.31)
Requirement already satisfied: filelock in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (3.8.0)
Requirement already satisfied: requests in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (2.28.1)
Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (0.12.1)
Requirement already satisfied: pyyaml>=5.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (6.0)
Requirement already satisfied: sacremoses in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (0.0.53)
Requirement already satisfied: packaging>=20.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (21.3)
Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (0.11.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+27.gebf4bec6) (1.3.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+27.gebf4bec6) (1.57.0)
Requirement already satisfied: protobuf<4,>=3.13 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+27.gebf4bec6) (3.20.3)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (4.4.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.tox/py38-gpu/lib/python3.8/site-packages (from packaging>=20.0->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (3.0.9)
Requirement already satisfied: multidict in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (4.1.0)
Requirement already satisfied: charset-normalizer<3,>=2 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (1.26.12)
Requirement already satisfied: certifi>=2017.4.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (2022.9.24)
Requirement already satisfied: click in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (8.1.3)
Requirement already satisfied: six in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (1.16.0)
Requirement already satisfied: joblib in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+27.gebf4bec6) (1.2.0)
Requirement already satisfied: hyperframe<7,>=6.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+27.gebf4bec6) (4.0.0)
Building wheels for collected packages: transformers4rec
  Building wheel for transformers4rec (pyproject.toml): started
  Building wheel for transformers4rec (pyproject.toml): finished with status 'done'
  Created wheel for transformers4rec: filename=transformers4rec-0.1.14+27.gebf4bec6-py3-none-any.whl size=216748 sha256=c70775b0a3a289f97a2af2dd417a097cb7a091c65488eeb14f85e03708f05488
  Stored in directory: /tmp/pip-ephem-wheel-cache-kn3uxg5o/wheels/cb/5d/b4/e081835ae498194a418e957657f998bdff0fa2bd103855a861
Successfully built transformers4rec
Installing collected packages: transformers4rec
  Attempting uninstall: transformers4rec
    Found existing installation: transformers4rec 0.1.14+27.gebf4bec6
    Uninstalling transformers4rec-0.1.14+27.gebf4bec6:
      Successfully uninstalled transformers4rec-0.1.14+27.gebf4bec6
Successfully installed transformers4rec-0.1.14+27.gebf4bec6
___________________________________ summary ____________________________________
  py38-gpu: commands succeeded
  congratulations :)
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[transformers4rec_tests] $ /bin/bash /tmp/jenkins2106501837987259067.sh

nvidia-merlin-bot · 2022-11-22T17:46:48Z

Click to view CI Results

GitHub pull request #543 of commit 25a15500d897c819c8173db5f02d9bb32f179969, no merge conflicts.
Running as SYSTEM
Setting status of 25a15500d897c819c8173db5f02d9bb32f179969 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/328/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse 25a15500d897c819c8173db5f02d9bb32f179969^{commit} # timeout=10
Checking out Revision 25a15500d897c819c8173db5f02d9bb32f179969 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 25a15500d897c819c8173db5f02d9bb32f179969 # timeout=10
Commit message: "Merge branch 'main' into refactor_part1"
 > git rev-list --no-walk ebf4bec6caa086e7492676656ec0cc86b160fef9 # timeout=10
[transformers4rec_tests] $ /bin/bash /tmp/jenkins7773265032755853784.sh
GLOB sdist-make: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/setup.py
py38-gpu create: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/py38-gpu
py38-gpu inst: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/.tmp/package/1/transformers4rec-0.1.14+29.g25a15500.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.14,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,bleach==5.0.1,boto3==1.24.75,botocore==1.29.14,Brotli==1.0.9,cachetools==5.2.0,certifi==2022.9.24,cffi==1.15.1,charset-normalizer==2.1.1,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,docker-pycreds==0.4.0,docutils==0.16,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==3.4,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,nvtabular==1.4.0+8.g95e12d347,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pbr==5.11.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.28.1,requests-oauthlib==1.3.1,rsa==4.7.2,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.16.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.44,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,tabulate==0.8.10,tblib==1.7.0,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.14+29.g25a15500,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='3066332432'
py38-gpu run-test: commands[0] | pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in ./.tox/py38-gpu/lib/python3.8/site-packages (22.3.1)
py38-gpu run-test: commands[1] | pip install .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: betterproto<2.0.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.2.5)
Requirement already satisfied: transformers<4.19 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (4.18.0)
Requirement already satisfied: tqdm>=4.27 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (4.64.1)
Requirement already satisfied: numpy>=1.17.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.23.5)
Requirement already satisfied: tensorflow-metadata in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.11.0)
Requirement already satisfied: pyarrow>=1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (10.0.0)
Requirement already satisfied: grpclib in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (0.4.3)
Requirement already satisfied: stringcase in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (1.2.0)
Requirement already satisfied: filelock in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (3.8.0)
Requirement already satisfied: regex!=2019.12.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2022.10.31)
Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.11.0)
Requirement already satisfied: packaging>=20.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (21.3)
Requirement already satisfied: sacremoses in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.0.53)
Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.12.1)
Requirement already satisfied: requests in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2.28.1)
Requirement already satisfied: pyyaml>=5.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (6.0)
Requirement already satisfied: protobuf<4,>=3.13 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (3.20.3)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (1.3.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (1.57.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (4.4.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.tox/py38-gpu/lib/python3.8/site-packages (from packaging>=20.0->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (3.0.9)
Requirement already satisfied: multidict in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (4.1.0)
Requirement already satisfied: certifi>=2017.4.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2022.9.24)
Requirement already satisfied: idna<4,>=2.5 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.26.12)
Requirement already satisfied: charset-normalizer<3,>=2 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2.1.1)
Requirement already satisfied: joblib in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.2.0)
Requirement already satisfied: six in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.16.0)
Requirement already satisfied: click in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (8.1.3)
Requirement already satisfied: hyperframe<7,>=6.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (4.0.0)
Building wheels for collected packages: transformers4rec
  Building wheel for transformers4rec (pyproject.toml): started
  Building wheel for transformers4rec (pyproject.toml): finished with status 'done'
  Created wheel for transformers4rec: filename=transformers4rec-0.1.14+29.g25a15500-py3-none-any.whl size=216672 sha256=7214b6bf9938df2a1402b2006b7a0fd8ccfa7c1d23f2bcbc84f0c2b0d001abf7
  Stored in directory: /tmp/pip-ephem-wheel-cache-knu1acad/wheels/cb/5d/b4/e081835ae498194a418e957657f998bdff0fa2bd103855a861
Successfully built transformers4rec
Installing collected packages: transformers4rec
  Attempting uninstall: transformers4rec
    Found existing installation: transformers4rec 0.1.14+29.g25a15500
    Uninstalling transformers4rec-0.1.14+29.g25a15500:
      Successfully uninstalled transformers4rec-0.1.14+29.g25a15500
Successfully installed transformers4rec-0.1.14+29.g25a15500
___________________________________ summary ____________________________________
  py38-gpu: commands succeeded
  congratulations :)
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[transformers4rec_tests] $ /bin/bash /tmp/jenkins12621012214390116787.sh

nvidia-merlin-bot · 2022-12-15T16:27:14Z

Click to view CI Results

GitHub pull request #543 of commit 25a15500d897c819c8173db5f02d9bb32f179969, has merge conflicts.
Running as SYSTEM
PR has already been merged, builds using the merged sha1 will fail!!!
Setting status of 25a15500d897c819c8173db5f02d9bb32f179969 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/transformers4rec_tests/407/ and message: 'Build started for original commit.'
Using context: Jenkins Unit Test Run
Building on the built-in node in workspace /var/jenkins_home/jobs/transformers4rec_tests/workspace
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/543/*:refs/remotes/origin/pr/543/* # timeout=10
 > git rev-parse 25a15500d897c819c8173db5f02d9bb32f179969^{commit} # timeout=10
Checking out Revision 25a15500d897c819c8173db5f02d9bb32f179969 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 25a15500d897c819c8173db5f02d9bb32f179969 # timeout=10
Commit message: "Merge branch 'main' into refactor_part1"
 > git rev-list --no-walk 95d58567c1d22ecf94480f39b692d07df3b04288 # timeout=10
[workspace] $ /bin/bash /tmp/jenkins13429669360682688155.sh
GLOB sdist-make: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/setup.py
py38-gpu recreate: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/py38-gpu
py38-gpu inst: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec/.tox/.tmp/package/1/transformers4rec-0.1.14+29.g25a15500.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.30,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,bleach==5.0.1,boto3==1.24.75,botocore==1.29.30,Brotli==1.0.9,cachetools==5.2.0,certifi==2022.12.7,cffi==1.15.1,charset-normalizer==2.1.1,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,docker-pycreds==0.4.0,docutils==0.16,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==3.4,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,nvtabular==1.4.0+8.g95e12d347,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.4,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pbr==5.11.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.1.0,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.28.1,requests-oauthlib==1.3.1,rsa==4.7.2,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.16.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.45,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,tabulate==0.8.10,tblib==1.7.0,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.14+29.g25a15500,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='1397309907'
py38-gpu run-test: commands[0] | pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in ./.tox/py38-gpu/lib/python3.8/site-packages (22.3.1)
py38-gpu run-test: commands[1] | pip install .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tqdm>=4.27 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (4.64.1)
Requirement already satisfied: betterproto<2.0.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.2.5)
Requirement already satisfied: transformers<4.19 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (4.18.0)
Requirement already satisfied: pyarrow>=1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (10.0.1)
Requirement already satisfied: numpy>=1.17.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.23.5)
Requirement already satisfied: tensorflow-metadata in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers4rec==0.1.14+29.g25a15500) (1.12.0)
Requirement already satisfied: stringcase in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (1.2.0)
Requirement already satisfied: grpclib in ./.tox/py38-gpu/lib/python3.8/site-packages (from betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (0.4.3)
Requirement already satisfied: sacremoses in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.0.53)
Requirement already satisfied: filelock in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (3.8.2)
Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.12.1)
Requirement already satisfied: regex!=2019.12.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2022.10.31)
Requirement already satisfied: requests in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2.28.1)
Requirement already satisfied: pyyaml>=5.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (6.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (0.11.1)
Requirement already satisfied: packaging>=20.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (22.0)
Requirement already satisfied: protobuf<4,>=3.13 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (3.20.3)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (1.57.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in ./.tox/py38-gpu/lib/python3.8/site-packages (from tensorflow-metadata->transformers4rec==0.1.14+29.g25a15500) (1.3.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (4.4.0)
Requirement already satisfied: h2<5,>=3.1.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (4.1.0)
Requirement already satisfied: multidict in ./.tox/py38-gpu/lib/python3.8/site-packages (from grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (6.0.3)
Requirement already satisfied: idna<4,>=2.5 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from requests->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.26.13)
Requirement already satisfied: six in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.16.0)
Requirement already satisfied: click in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (8.1.3)
Requirement already satisfied: joblib in ./.tox/py38-gpu/lib/python3.8/site-packages (from sacremoses->transformers<4.19->transformers4rec==0.1.14+29.g25a15500) (1.2.0)
Requirement already satisfied: hyperframe<7,>=6.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->transformers4rec==0.1.14+29.g25a15500) (4.0.0)
Building wheels for collected packages: transformers4rec
  Building wheel for transformers4rec (pyproject.toml): started
  Building wheel for transformers4rec (pyproject.toml): finished with status 'done'
  Created wheel for transformers4rec: filename=transformers4rec-0.1.14+29.g25a15500-py3-none-any.whl size=477552 sha256=a34013fb17a23ccc9a5553f1bb9ec5aea2683af5ecb7e8a5cad766cfae194040
  Stored in directory: /tmp/pip-ephem-wheel-cache-ibqfeybo/wheels/cb/5d/b4/e081835ae498194a418e957657f998bdff0fa2bd103855a861
Successfully built transformers4rec
Installing collected packages: transformers4rec
  Attempting uninstall: transformers4rec
    Found existing installation: transformers4rec 0.1.14+29.g25a15500
    Uninstalling transformers4rec-0.1.14+29.g25a15500:
      Successfully uninstalled transformers4rec-0.1.14+29.g25a15500
Successfully installed transformers4rec-0.1.14+29.g25a15500
___________________________________ summary ____________________________________
  py38-gpu: commands succeeded
  congratulations :)
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[workspace] $ /bin/bash /tmp/jenkins12529385364285494603.sh

nzarif added 2 commits November 19, 2022 01:45

part1 refactoring done and all torch unit tests pass

ca51777

fixed formatting with black

867e7c0

nzarif requested review from gabrielspmoreira and sararb November 19, 2022 01:57

nzarif self-assigned this Nov 19, 2022

nzarif added enhancement New feature or request area/api labels Nov 19, 2022

remove hf_format from paper script

68b8c73

nzarif added 6 commits November 21, 2022 19:31

part1 refactoring done and all torch unit tests pass

4d6647f

fixed formatting with black

4399e7b

remove hf_format from paper script

8cb4b25

removed hf_format from notebooks

bf2deab

Merge branch 'refactor_part1' of github.com:NVIDIA-Merlin/Transformer…

9caca0b

…s4Rec into refactor_part1

rebased with latest main

6eab668

make flake8 ignore line 230 of prediction_task.py

f7547ba

sararb reviewed Nov 22, 2022

View reviewed changes

removed hf_format from all classes

ebf4bec

Merge branch 'main' into refactor_part1

25a1550

sararb approved these changes Nov 22, 2022

View reviewed changes

sararb mentioned this pull request Nov 22, 2022

[BUG] getting wrong dimension from Triton as a response when serving a TF4Rec model with TorchScript #538

Closed

sararb merged commit 8cd1847 into main Nov 22, 2022

rnyak mentioned this pull request Nov 23, 2022

[RMP] Provide PyTorch serving support for T4R models in Torchscript NVIDIA-Merlin/Merlin#255

Closed

27 tasks

oliverholworthy mentioned this pull request Nov 28, 2022

Remove use of hf_format and training params for PyTorch inference test NVIDIA-Merlin/systems#236

Merged

sararb added the breaking Breaking change label Dec 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactoring part1- flags modification#543

Refactoring part1- flags modification#543
sararb merged 12 commits intomainfrom
refactor_part1

nzarif commented Nov 19, 2022 •

edited by sararb

Loading

Uh oh!

nvidia-merlin-bot commented Nov 19, 2022

Uh oh!

nvidia-merlin-bot commented Nov 19, 2022

Uh oh!

review-notebook-app bot commented Nov 21, 2022

Uh oh!

nvidia-merlin-bot commented Nov 21, 2022

Uh oh!

nvidia-merlin-bot commented Nov 21, 2022

Uh oh!

github-actions bot commented Nov 21, 2022

Uh oh!

sararb left a comment

Uh oh!

sararb Nov 22, 2022

Uh oh!

nvidia-merlin-bot commented Nov 22, 2022

Uh oh!

nvidia-merlin-bot commented Nov 22, 2022

Uh oh!

nvidia-merlin-bot commented Dec 15, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -545,12 +554,19 @@ def __init__(
		self.optimizer = optimizer
		self.hf_format = hf_format

Conversation

nzarif commented Nov 19, 2022 • edited by sararb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Goals ⚽

Implementation Details 🚧

Testing Details 🔍

Uh oh!

nvidia-merlin-bot commented Nov 19, 2022

Uh oh!

nvidia-merlin-bot commented Nov 19, 2022

Uh oh!

review-notebook-app bot commented Nov 21, 2022

Uh oh!

nvidia-merlin-bot commented Nov 21, 2022

Uh oh!

nvidia-merlin-bot commented Nov 21, 2022

Uh oh!

github-actions bot commented Nov 21, 2022

Documentation preview

Uh oh!

sararb left a comment

Choose a reason for hiding this comment

Uh oh!

sararb Nov 22, 2022

Choose a reason for hiding this comment

Uh oh!

nvidia-merlin-bot commented Nov 22, 2022

Uh oh!

nvidia-merlin-bot commented Nov 22, 2022

Uh oh!

nvidia-merlin-bot commented Dec 15, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nzarif commented Nov 19, 2022 •

edited by sararb

Loading