Meta tracking issue to make alerting configuration nicer than a painful series of steps through the Grafana UI; I am pretty sure another exists for this already as we are working on this - I just couldn't find it.