This has been brought up by @malfet

We should have CI that runs without torchdata in our nightlies, so that we can make sure that torchdata is not a required dependency.

What I would propose is to add another CI job that just for one single OS that just runs tests for the prototype folder.

cc @pmeier @seemethere @bjuncek