Anomaly detection jobs have a model_memory_limit setting.  This is
supposed to restrict the amount of memory the job can use, however,
in the past the limit only applied to anomaly detection and not to
categorization.

This change applies memory limiting to categorization as follows:

- When a job is in hard_limit status no new categories will be
  created.  The input document that could not be categorized is
  discarded as it cannot take part in anomaly detection without a
  category.  The failed_category_count statistic is incremented
  each time this happens.
- When a job is in soft_limit status, we stop recording examples
  for the category.

Backport of elastic#1167