I suggest .unit.to_string(format='latex_inline'). Particualrly useful if the unit is Angstroem or \mu that can actually be written with the real symbol in Latex.

This seems to change the behavior of the tests, e.g. "s" becomes "$\mathrm{s}$". Should I just change the expected test output then?

Yes, please change the expected output of the tests.
LaTeX can encode symbols and formulas that plain ASCII cannot (e.g. the
A with a circle on the top that means Angstroem, or proper use of
exponents, try e.g. a unit like u.km / u.s**2). When we use a LaTeX
writer, we should make full use of astropy's ability to write the units
in LaTeX form with all the bells and whistles that normal ASCII output
does not have.

I agree with @hamogu to use the latex_inline format and change the tests accordingly.

One more thing I just realized: What happens is no unit is set?
In that case I would prefer not to see an empty line. What do you think?

Agree, if there are no columns with a defined unit then there should not be an empty line.

Maybe you can tidy up the logic a bit with something like:

units = self.latex.get('units', {})
units.update((x.name, x.unit.to_string(format='latex_inline')) for x in self.cols if x.unit)
if units:
    lines.append(self.splitter.join([units.get(x.name, ' ') for x in self.cols)])

Oops, just realized I have the precedence wrong here, so you need to flip things.