Skip to content

Conversation

@riedgar-ms
Copy link
Member

@riedgar-ms riedgar-ms commented Mar 16, 2022

Bring civilised citations to the 'Fairness in Machine Learning' page in the documentation. Also do some reformatting of the ReST file, to reduce the length of a selection of lines.

Related to #936

Signed-off-by: Richard Edgar <riedgar@microsoft.com>
Signed-off-by: Richard Edgar <riedgar@microsoft.com>
Signed-off-by: Richard Edgar <riedgar@microsoft.com>
Signed-off-by: Richard Edgar <riedgar@microsoft.com>
Signed-off-by: Richard Edgar <riedgar@microsoft.com>
@riedgar-ms riedgar-ms requested a review from a team March 16, 2022 13:47
@riedgar-ms
Copy link
Member Author

This should probably go in before #1042 (because when I add citations to that, I really don't want to do so by hand)

Signed-off-by: Richard Edgar <riedgar@microsoft.com>
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3287560.3287598},
doi = {10.1145/3287560.3287598},
abstract = {A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science---such as abstraction and modular design---are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones.},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the abstract needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No; I simply didn't go to the effort of removing it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But... shouldn't we? Do we really want to keep abstracts of papers here? It's not like we're going to render those...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could trim it; I was just doing the simplest thing based on the citation source.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're not rendering it let's just cut it(?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to have as few modifications as possible on the copy pasted chunk. Also, this doesn't take that much space, why bother.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are providing URL, I would personally lean towards skipping most of the extra meta-information here like isbn, abstract, numpages, keywords. But I don't care too much about extra meta-information that doesn't get displayed.

Two things that I think we should care about are:

  • consistency in the naming of the bibtex ids (format , where keyword is most of the time the first word in the paper title)
  • consistency in the appearance of the references at the end of files (this is something that is actually not too hard to fix later, because we only need to edit .bib file...)
  • linking to peer-reviewed versions of papers rather than arXiv whenever peer-reviewed versions are available

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping metainformation is the business of the bibstyle file. Isn't that pretty much the whole point of BibTeX?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps #1048 is a good place to continue this conversation. @riedgar-ms I agree except for the abstract (which is neither relevant nor is there even a remote chance of it ever being rendered, plus it's massive). I wrote that in #1048, too, so feel free to object there 😆

Copy link
Member

@MiroDudik MiroDudik Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only issue that I see re. consistency is that these bibtex entries are generating multiple URLs when rendered--multiple URLs are also confusing in their own right (what are we actually referring to?). I'm in favor of dropping "eprint" and "doi" and retaining URL. (I think that's simpler that trying to tweak the stylefile?)

Signed-off-by: Richard Edgar <riedgar@microsoft.com>
@riedgar-ms riedgar-ms requested a review from romanlutz March 16, 2022 14:52
Copy link
Member

@MiroDudik MiroDudik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot check how these render... can you take a snippet and post in a comment or one of the replies?

But, based on looking at bibtex entries below, I'd be in favor of something more condensed (e.g., that refers to the venue as FAT* and the publisher as ACM). The easiest way to construct that is to basically take the condensed DBLP entry:

But take the URL from the standard DBLP entry:

Sorry for being a pest, but I think it would be good to refer to academic papers consistently, and this one is not too hard.

address = {New York, NY, USA},
url = {https://doi.org/10.1145/3287560.3287598},
doi = {10.1145/3287560.3287598},
abstract = {A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science---such as abstraction and modular design---are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones.},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are providing URL, I would personally lean towards skipping most of the extra meta-information here like isbn, abstract, numpages, keywords. But I don't care too much about extra meta-information that doesn't get displayed.

Two things that I think we should care about are:

  • consistency in the naming of the bibtex ids (format , where keyword is most of the time the first word in the paper title)
  • consistency in the appearance of the references at the end of files (this is something that is actually not too hard to fix later, because we only need to edit .bib file...)
  • linking to peer-reviewed versions of papers rather than arXiv whenever peer-reviewed versions are available

Signed-off-by: Richard Edgar <riedgar@microsoft.com>
@riedgar-ms
Copy link
Member Author

The current rendering on my machine:
image

Is ACM not considered a canoncial source of bibliographic information?

Copy link
Member

@MiroDudik MiroDudik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good... (modulo the merge). But if you end up ensuring that there's only one URL associated with each reference (by removing doi and eprint), I definitely won't mind :-)... but it's a low order concern for me.

@hildeweerts hildeweerts merged commit 38364ae into fairlearn:main Apr 5, 2022
@hildeweerts
Copy link
Contributor

@MiroDudik feel free to make any small adjustments in a new PR!

@riedgar-ms riedgar-ms deleted the riedgar-ms/bibtex-fairness-ml-doc branch October 23, 2023 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants