@andreslucena No need on my end. Last time the technical implementation seemed fine.

Only thing to add is that as I have written in some of the previous reviews, I would not feed the classifier constantly with the moderated data as "spam". For example, in the instances we manage, spammers are using actual other people's comments and injecting their spam in those. When the classifier gets fed this data as "spam", it will eventually start to classify genuine content as SPAM with higher probability.

I have written more details in the broader context of solving the SPAM issue at #10038. For content classification engine, this implementation is fine (but would require more pre-training data to actually work well).

I have written more details in the broader context of solving the SPAM issue at #10038. For content classification engine, this implementation is fine (but would require more pre-training data to actually work well).

Wow, I actually missed those comments. Great job with researching this topic!

So,for mitigating the problems that Antti found, and to merge this without future issues, we'd need to:

1. Remove the current datasets, as we don't have the good numbers of contents to this actually be useful
2. To remove the feature of training with the new moderations, as the spammers are using "good" content, so we will start having lots of false positives

Can you do that @alecslupu? Thanks

Then, after this PR is (finally) merge, we can focus on the other aspects of the "Steps forward" explanation from #10038 (comment)

To remove the feature of training with the new moderations, as the spammers are using "good" content, so we will start having lots of false positives

I am not sure what you mean by this.

Remove the current datasets, as we don't have the good numbers of contents to this actually be useful

Done in f3b8f48

To remove the feature of training with the new moderations, as the spammers are using "good" content, so we will start having lots of false positives

I am not sure what you mean by this.

I forgot how this actually works to be honest xD

This is a manual task that need to be done by the admin. According to what Antti is mentioning in its comment, spammers are reusing actual contents to "confuse" these kinds of filters. We can leave it in for now, but once we have the new datasets for moderations we can review this approach (i.e. remove this task if it isn't necessary and could be counterproductive)

I forgot how this actually works to be honest xD

This is a manual task that need to be done by the admin.

@andreslucena To me it doesn't seem manual:

The classifier is always fed the moderated content automatically when it is hidden from the website.

@andreslucena To me it doesn't seem manual:

You're right!

@alecslupu can you remove what Antti is mentioning, please?

@ahukkanen the part you initially mentioning:

Is actually the part that submits the data to be analyzed. It does not do the training of spam / ham.

The following is the one that automatically adds the data hidden by admin to spam base.

I will remove only this bit.

Is actually the part that submits the data to be analyzed. It does not do the training of spam / ham.

Partially correct but at the very bottom of the snippet there is logic that trains the user profile classifier with the blocked user's data.

Is actually the part that submits the data to be analyzed. It does not do the training of spam / ham.

Partially correct but at the very bottom of the snippet there is logic that trains the user profile classifier with the blocked user's data.

tackled here: #13550