steps to reproduce:

1. set on namespace label
2. create ClusterExternalSecret
3. wait before secret synced at selected namespace
4. remove label from namespace
   after that k8s secret and external secret still exists in namespace and ClusterExternalSecret in status i see:

Status:
  Conditions:
    Status:   True
    Type:     Ready
    Message:  one or more namespaces failed
    Status:   True
    Type:     NotReady
  Failed Namespaces:
    Namespace:  test
    Reason:     external secret in non matching namespace could not be deleted

Is it a bug or i need manual remove externalsecret from namespace after remove label?