17

A couple of weeks ago, @dbc asked, In "Low Quality Posts" is it required to cross-check external links with profile pages to identify Improper Self Promotion?, to which I responded with my process for evaluating spam. As part of that, I noted:

Note: This may require clicking through to e.g. source code for a library. This can be hard to spot if they're just using e.g. an import statement or package reference.

That last bit about import statements or package references seemed a bit fussy at the time, and I considered removing it. Since then, however, I've seen a couple of cases of exactly that.

The most overt example was an answer that was clearly promoting their own library. In another (now deleted) post, I warned the contributor about the self-promotion policy. In response, they edited their answer to remove the affiliation, and simply referenced their library via an import statement:

import numpy as np
import fastfinance as ff
…

This is obviously a bit sloppy as that import is going to fail without first downloading the library via e.g. pip. And, regardless, the contributor hadn't packaged or registered their library on the PyPI. But, if they had, and I wasn't aware of the previous edit, would I have recognized something like the following as self-promotion?

First, run the following at your CLI:

pip install fastfinance

And then, add this to your code:

import numy as np
import fastfinance as ff
…

I doubt it.

This makes me wonder, in the spirit of @dbc's previous question, in "Low Quality Answers", should we cross-check package references to identify Improper Self Promotion?

I.e., should we see something like:

pip install somelibrary

or:

npm i somelibrary

as just as suspicious as:

First, download somelibrary…

This isn't something I generally do, unless there are other obvious concerns, but now I wonder if this should be standard practice for package references in the Low Quality Answers queue.

10
  • Note: I have previously flagged the example post for moderator attention, so it may be removed in the future. I think I've offered enough context so that this post remains relevant without it. Commented Jan 14, 2022 at 22:41
  • 2
    I'm not a Python dev so I don't know if any of these libs are common/standard, but I would consider any answer which uses libs without explicitly mentioning them at least "meh" or of questionable quality. So a downvote and a comment could be the result and when the OP updates the post and mentions the lib without disclosing the affiliation, then it could result in a spam flag. But I wouldn't consider it the job of a reviewer to check every used library in a block of if it is spam, when those libs aren't mentioned anywhere else. Commented Jan 15, 2022 at 0:02
  • 1
    libraries are needed to circumvent all yourself, so if a library does as intended, i don't see the harm Commented Jan 15, 2022 at 1:29
  • 3
    @nbk: If that’s the case, the self-promotion rules should be revisited—which is a separate question. As it stands, however, relying on tools or libraries that you authored either without disclosure, or as a pattern that is primarily intended to popularize your work, is considered inappropriate. Such answers are regularly flagged for (and removed by) moderators when they’re linked to. So long as that stands, relying on package references offers a less-obvious back door to that policy, which I suspect many of us (including me) overlook during review. Commented Jan 15, 2022 at 1:49
  • 5
    Specifically Python has a ton of functionality in the standard library (libraries that you can import without having to install anything), and a rich ecosystem of well-known, high-profile third-party libraries for all kinds of programming. Experienced python users can usually spot unusually esoteric libraries, and I would expect that these are almost never needed in an answer on Stack Overflow. But this doesn't matter too much because we don't expect reviewers to be subject matter experts, so they shouldn't have to recognise whether a given library is mainstream or not. Commented Jan 15, 2022 at 11:27
  • @JeremyCaney if he head posted a link to his site and didn't disclosure his affiliation it could be self promoting, but nothing of that case is in the example in visited and so this is a widely gray area hand has to be examined in every single case. i understand the necessaty for rules but those can only be guidelines. alink only answer is low quality and you should click on it. to see where it leqads Commented Jan 15, 2022 at 12:15
  • 1
    How would you even know if there is an affiliation if user names don't match cross platform? Commented Jan 15, 2022 at 14:40
  • @charlietfl: Obviously, a determined and clever individual can find a way to mask every aspect of their affiliation. In practice, though, plenty don’t. Sometimes the names don’t match, but the avatars do, or they’re linking to a resource that was posted today. Other times, there’s no apparent association outside of a suspicious pattern of referencing an unheard of library across multiple posts. Commented Jan 15, 2022 at 18:12
  • 3
    @JeremyCaney My higher level point is it's often not possible to make a connection and the level of sleuthing required should never be held against someone (ie in audit). Best practice would certainly be to make an attempt of course Commented Jan 15, 2022 at 18:15
  • @charlietfl: I certainly agree with that. Fortunately, anything at this level necessitates a custom flag with further explanation, and thus shouldn’t show up as audits. Commented Jan 15, 2022 at 19:35

4 Answers 4

Reset to default
15

This whole effort (checking external resources up to the source code level) is worth for all content on SO, not only the low quality answer queue, but as a corollary, should not be required for the low quality answer queue.

If that amount of work would be really required (checking a library for possible self-promotion surely takes a minute or two or longer if done thoroughly), I would understand if people won't review anymore. Leave it to the regular visitors instead to check for possible undisclosed self-promotion.

For example, if somebody told me that for a certain Python problem, one needs a certain Python package I would not get suspicious per se. Why should I?

6
  • 4
    Yeah, that would kill most review queues. You are supposed to do fast spot checks to determine if the answer is relevant to the question and on the face decent quality. Commented Jan 15, 2022 at 12:17
  • 1
    I’m not sure I disagree with your conclusion, but I take issue with the reasoning in your first paragraph. We needn’t let perfect be the enemy of the good. Every check we do in the queues should apply to every post on the site. But we prioritize particular posts for review. E.g., posts from new contributors get particular scrutiny, or posts on old questions. Often, people serially promoting their library will find old questions to submit an answer to, and end up in Late Answers, at least. (Not LQA, admittedly, but I consider FA and LA natural extensions of LQA). Commented Jan 15, 2022 at 18:30
  • As for why you should be suspicious, you would be if they linked to the library, as that’s part of the review criteria. Why should an unrecognized package reference be any different than a link? Commented Jan 15, 2022 at 18:34
  • 1
    @JeremyCaney if you have time and the inclination to do so, do it. Just don't expect anyone else to do it. And more importantly, there should be no rule or enforcement punishing reviewers from not doing it. Commented Jan 16, 2022 at 16:30
  • @Braiam: For sure. I definitely see this as a "should" not a "must", along the lines of how you should edit any formatting or grammar mistakes you see when reviewing otherwise-acceptable posts. Fortunately, as noted elsewhere, these situations always necessitate custom flags, and thus shouldn't show up as review audits. Commented Jan 16, 2022 at 23:47
  • 1
    @JeremyCaney I see it as a level below: may. Someone may cross reference libraries while reviewing, but it's not expected and not encouraged. Commented Jan 17, 2022 at 13:53
8

To do this correctly, you'd need to have a tool to:

  • Listen to new Python questions, and check if they mention any packages
  • Look up packages on PyPI
  • Look up packages on Safety, Snyk, libraries.io or other repositories that have package quality information (i.e., is it malicious, cryptomining, involves spending money so it acts as a form of advertising?)
  • Have a human evaluate all this information and decide if the package is problematic. (And if the problem is unannounced self-promotion, this is a lot of work for a social nicety—I can promote someone else's crap/commercial code and I can promote my own crap/commercial code. The difference is that people feel like the latter is a social faux pas. Or, in the case of things that cost money, should we ban everything that costs money because SO is going to be the thing that brings the communist utopia?)

Cross-referencing a user on PyPi, unless the names match up identically, is a nontrivial task. I've been writing code to solve this problem and it is hard.

Less than that and you're just sweeping sand into the ocean.

In the meanwhile, people will have to watch out for themselves.

0
2

After trying this out myself over the last week, I want to stake out a softer, more nuanced position.

Based on my experience, this adds a trivial amount of overhead to the average review, yet is a relatively easy heuristic to add to our review rubric. I see it as something we should strive for in order to keep the site in top shape, but not be penalized for missing, just like editing posts during review.

Further details below.

Effort Involved

It's important to recognize that this concern applies to a very small number of posts.

Nearly all package references are linked to—which we should be checking regardless. Of those that are exclusively referenced via an install commands, the vast majority relate directly to the library in question (e.g., "How do I install [x]?"). Beyond that, almost all are well-known packages that you quickly become familiar with, even if it's not a language you know (e.g., NumPy, pandas, Matplotlib). Once that's acknowledged, we're really only talking about one post in every every few hundred reviews.

Given that, the effort required for this isn't actually as cumbersome as @Trilarion suggests in their answer. While it may, in fact, add "a minute or two or longer" to these specific posts, it's such a rare situation that it doesn't affect the average post. Further, it is typically just a matter of Googling the package, clicking on the first or second link to e.g. GitHub, and checking the handle, avatar, and stars. Most cases will be clearly established libraries with no overt affiliation, at which point we click "Looks OK" (assuming no other issues).

That said, I fully agree with @MatthewMartin's answer that, ideally, this is something that should be automated as part of the spam scoring system—acknowledging that is not trivial.

Accountability

This isn't something that should be strictly required. Nobody should fail a review audit or receive a review suspension for failing to identity self-promotion hidden behind a package reference.

That's already built into the system, since these will necessitate a custom flag to properly recognize. I.e., posts deleted under this criteria shouldn't come up as audits.

Recommendation

I see checking these package references as a best practice, very similar to editing posts to fix typos, grammar, and formatting issues. Editing actually adds far more overhead, and is something we should be doing if we're taking reviewing seriously, but it's not something you'll (directly) fail an audit for.

The bigger issue is simply being aware of this, and recognizing package manager references as being just as suspicious as links. Once you're over that hurdle, this becomes pretty much like any of those other edge cases (such as suspiciously well-written content from a new contributor potentially being plagiarized).

0

I'm the fastfinance author.

I just want to rectify some stuff you said as you pretend my library to be suspicious.

  • I'm not a spammer; I'm a real person...
  • When you sent the self-promoting warning, I was a new Stack Overflow user and I wasn't aware of all those crazy rules, but I answered I was ok to follow them. I removed the concerned posts and I don't post link to my library anymore. I can't do more than that...
  • If you take a look at fastfinance discussions on GitHub, you will see that I'm still looking for contributor to help me package the library and make it available on PyPI. If a library is not available on PyPI, it doesn't mean it's a bad library. It just means you are not ok to use a non-ready to go solution like new lazy programmers today...
  • There isn't any affiliation as there is just me making the library on my free time and I don't make money off of that. This is just my contribution to open source world...
  • You used my project as an example of a suspicious thing, but you didn't inform me you were doing that. I didn't have a chance to explain myself, and that is not fair...
4
  • 3
    I have no reason to believe your library is suspicious in terms of e.g., reliability or security. Without disclosure that you are the author, your post was suspicious as spam. It doesn’t matter if you are making money off the library, or if it is free; you still must disclose your affiliation. (The same goes for linking to e.g., a blog post you wrote.) I do understand that it takes time to familiarize yourself with the rules around here. Indeed, this question wasn’t intended to critique your library, but to ensure I understood those rules myself, as a reviewer. Commented Nov 3, 2022 at 6:14
  • 1
    Your posts initially had disclosure but you lacked a proper example which made it a link only answer which get deleted since they aren't useful. You later updated them to add examples but removed disclosure which is required if you are affiliated (in this case as the creator) in any way to the package / library etc. you promote (intentional or not) in the post. You can post links to your library but you must 1) not exclusively answer questions where you can post these links, 2) disclose that you are the author / creator of the package, 3) Provide a proper example instead of just a link. Commented Nov 3, 2022 at 6:29
  • 1
    I'm ok with all you said guys, Jeremy sent me the rules and I'm ok with them but can you say good things about my library now ? Thank you Commented Nov 3, 2022 at 6:56
  • 2
    You library and you are both likely good (I didn't check but nothing suggested here gives me reason to check.) The issue is not answering questions with examples of use of your library without some form of attribution. It really could be as simple as "My library fastfinance ..." instead of "The library fastfinance" or "import fastfinance". Commented Nov 3, 2022 at 14:25

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.