https://issues.apache.org/jira/browse/ARROW-8244

cc @rjzamora does this look the correct logic?

Thank you for working on this @jorisvandenbossche!

Yes, these changes are exactly what I had in mind. I don't think there is any danger in setting the file_path here, because the returned metadata will never be included in the file where the data is stored (the only case where the value of None is technically correct). Leaving the field empty in write_table makes sense, because the user is already specifying the full path and may want to manually construct a partitioned dataset. However, for write_to_dataset, I would argue that the only correct behavior is to set the file_path by default, and allow the user to modify it if desired/necessary (which is what you have in this PR).

@rjzamora Thanks for the feedback! I agree that just setting the file path is probably the only sensible behaviour, so we can simply change that.

I added a test for the non-partitioned case as well.