This branch currently is behind tinygrad/master. The line count difference bot is disabled.

this needs some test

also it looks quite complicated, in what condition the shard can be created on each device instead of a copy?

in what condition the shard can be created on each device instead of a copy?

Tried to simplify it. Shard can be created directly on each device only for empty tensors on a single device where the size divides evenly across all devices.

i think the correct fix is a rewrite rule that says copying empty is equivalent to creating the empty on the new device (assuming it's not used by other stuff), then you don't need a new code path in shard

i think the correct fix is a rewrite rule that says copying empty is equivalent to creating the empty on the new device (assuming it's not used by other stuff), then you don't need a new code path in shard

Makes sense. Applied that.