trento.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"
    />

    <title>Fairness in Machine Learning &mdash; Part 1</title>

    <link rel="stylesheet" href="dist/reset.css" />
    <link rel="stylesheet" href="dist/reveal.css" />
    <link rel="stylesheet" href="custom_themes/sussex.css" id="theme" />

    <!-- Theme used for syntax highlighting of code -->
    <link rel="stylesheet" href="plugin/highlight/monokai.css" />

    <style>
      .citeme {
        float: right;
        color: #000000;
        font-size: 12pt;
        font-style: italic;
        margin: 0;
      }
    </style>
  </head>

  <body>
    <div id="hidden" style="display: none">
      <div id="static-content">
        <footer>
          <p>Tango Project 2023</p>
        </footer>
      </div>
    </div>
    <div class="reveal">
      <div class="slides">
        <section>
          <h1>Fairness in Machine Learning</h1>

          <p>by Oliver Thomas</p>
          <p>ot44@sussex.ac.uk - Predictive Analytics Lab (PAL)</p>
          <p>University of Sussex, UK & BCAM, Spain</p>

          <div
            style="display: flex; justify-content: center; align-items: center"
          >
            <img
              src="images/sussex_logo.svg"
              style="
                max-width: 10%;
                max-height: 100%;
                border: none;
                vertical-align: middle;
                margin-right: 5px;
              "
            />

            <img
              src="images/logos/bcam.png"
              style="
                max-width: 10%;
                max-height: 100%;
                border: none;
                vertical-align: middle;
                margin-left: 5px;
              "
            />
          </div>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## Contents

            1. Intro and notation
            2. What is fairness in machine learning and why should I care?
                1. Algorithmic fairness definitions
                2. Approaches to enforce algorithmic fairness
                3. Causality and Counterfactual Fairness
            3. Practical Applications
                1. Transparency in algorithmic fairness
                2. RLHF

          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## Non-Contents
            <p>&nbsp;</p>
            <p>&nbsp;</p>
            <p>&nbsp;</p>
            1. This lecture will not "solve" the problem of fairness

          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## Take home messages

            - Fairness is _Domain Specific_.
            - If you don't think about bias, it will come back to haunt you.
            - Accuracy only tells part of the story.
            - Fairness is an active and exciting research topic.
          </textarea>
        </section>

        <section>
          <h2>1. Introduction & Notation</h2>
          <h3>Machine Learning</h3>
          <blockquote>
            ...using statistical techniques to give computer systems the ability
            to "learn" (e.g., progressively improve performance on a specific
            task) from data, without being explicitly programmed.
          </blockquote>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 1. Introduction & Notation
            ### Classification

            - given some input $X$, predict a class label $Y \in \\{0, 1, ..., C-1\\}$
            - $X$ is usually a **vector**
                - often with high number of dimensions, e.g. more than 1 million for a picture
            - simplest case: **binary classification**, $Y \in \\{0, 1\\}$
                - for example: is there a hot dog in this picture ($Y=1$) or not ($Y=0$)?
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 1. Introduction & Notation
            ### Classification

            - we are looking to train a function $f$ that maps $X$ to $Y$
            - the output is the prediction: $\hat{Y} = f(X)$
            - we want $\hat{Y}$ to be as close as possible to the label $Y$
            - $f$ can be implemented as
                - a neural network
                - an SVM
                - a decision-tree model
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 1. Introduction & Notation
            ### Training data

            - training data: a set of pairs $(x, y)$
                - input data $x$ with corresponding label $y$
            - we are looking for model that works well on _unseen data_
            - so we try to gather training data that is _similar_ to deployment setting
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 1. Introduction & Notation
            ### Training data

            - BUT this is not always possible
            - if we make predictions on data that is *very different* from the training data,
              the model will perform badly
            - problem if the training data does not describe reality well
          </textarea>
        </section>

        <section>
          <h2>1. Introduction & Notation</h2>
          <h3>Machine learning systems</h2>

          <p align="left">
            Machine learning systems are being implemented in all walks of life
          </p>
          <table>
            <tbody>
              <tr>
                <td style="border: none" width="30%">
                  <img
                    src="images/scs.jpg"
                    width="65%"
                    title="Social Credit System"
                    style="margin: -20px 0px"
                  />
                  <p style="font-size: 10px">Picture credit: Kevin Hong</p>
                </td>
                <td style="border: none" width="30%">
                  <img
                    src="images/algowatch.jpg"
                    width="65%"
                    title="Automating Society"
                    style="margin: -20px 0px"
                  />
                  <p style="font-size: 10px">Picture credit: AlgorithmWatch</p>
                </td>
                <td style="border: none" width="30%">
                  <img
                    src="images/CDEI.png"
                    width="65%"
                    title="Automating Society"
                    style="margin: -20px 0px"
                  />
                  <p style="font-size: 10px">
                    Picture credit: Centre for Data Ethics and Innovation, UK
                  </p>
                </td>
              </tr>
              <tr>
                <td style="border: none" width="30%">
                  Social credit system, China
                </td>
                <td style="border: none" width="30%">
                  Personal budget calculation, UK
                </td>
                <td style="border: none" width="30%">
                  Financial services, Crime and justice,
                </td>
              </tr>
              <tr>
                <td style="border: none" width="30%"></td>
                <td style="border: none" width="30%">Loan decision, Finland</td>
                <td style="border: none" width="30%">Recruitment,</td>
              </tr>
              <tr>
                <td style="border: none" width="30%"></td>
                <td style="border: none" width="30%">etc.</td>
                <td style="border: none" width="30%">Local government</td>
              </tr>
            </tbody>
            <aside class="notes">
              Companies are already using or planning to use machine learning
              for these tasks.
            </aside>
          </table>
        </section>

        <section>
          <p>&nbsp;</p>
          <p>&nbsp;</p>
          <p>&nbsp;</p>
          But there are problems:
        </section>

        <section data-background-color="#fff">
          <img
            src="images/pp_mb.png"
            width="72%"
            title="Pro-Publica - Machine Bias"
          />
        </section>

        <section data-background-color="#fff">
          <img
            width="72%"
            src="images/amazon.png"
            title="Amazon CV Screening"
          />
        </section>

        <section data-background-color="#fff">
          <img
            src="images/bots-at-the-gate.png"
            width="40%"
            title="Bots At The Gate"
          />
        </section>

        <section data-background-color="#fff">
          <img src="images/norman.png" width="58%" title="Norman" />
        </section>

        <section>
          <img
            src="images/nyclu.png"
            width="60%"
            title="New York Civil Liberties Union"
          />
          <p style="font-size: 10px">
            Picture credit: New York Civil Liberties Union
          </p>
        </section>

        <section>
          <h2>2. What is Fair ML</h2>
          <h3>Algorithmic bias</h2>

          <section>
            <ul>
              <li>
                machine learning systems are making decisions that affect humans
              </li>
              <li>these decisions should be <b>fair</b></li>
              <li>
                by default machine learning algorithms tend to be biased in some
                way
              </li>
              <ul>
                <li>why?</li>
              </ul>
            </ul>
          </section>

          <section>
            <p>Because the world is complicated!</p>
            <p>The data is unlikely to be perfect.</p>
            <p>
              The developer's (often arbitrary) decisions have a downstream
              impact
            </p>
            <p>Business goals don't always align with fair behaviour</p>
          </section>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Algorithmic bias

            (Broadly) The problem can be divided into <ins>two categories</ins>.

            Both types of bias can appear together.

            1. bias stemming from biased training data
            2. bias stemming from the algorithms themselves
          </textarea>
        </section>

        <section>
          <p>&nbsp;</p>
          <p>&nbsp;</p>
          <h1>Biased training data</h1>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Bias in, bias out

            - Databases: GIGO
            - ML: BIBO
                - the ML algorithm just learns what is in the training data
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Examples of bias

            **Task**: generate description for images

            ![](./images/women_also_snowboard/title.png)
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Sampling bias &#10132; tyranny of the majority

            - In the <span class="highlight">imSitu</span> situation recognition <span class="highlight">dataset</span>, the activity cooking is over 33\% more likely to involve females than males in a training set, and a trained algorithm further <span class="highlight">amplifies</span> the disparity to 68\%
            <span class="citeme">Zhao et al.: Men also like shopping, EMNLP 2017</span>

            <img src="images/women_also_snowboard/example3.png" title="Men also like shopping"/>
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Sampling bias &#10132; tyranny of the majority

            - <span class="highlight">The reason is</span>: the algorithm predicts the gender from the activity and not from looking at the person
            <span class="citeme">Anne Hendricks et al.: Women also snowboard, ECCV 2018</span>

            <img src="images/women_also_snowboard/example2.png" title="Women also snowboard"/>
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Desired result

            - what we would want is that the algorithm only looks at the person when predicting the gender
            - if the gender is not recognizable from the picture, the algorithm should be "unsure"
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Biased data

            - in the previous example, certain training examples were underrepresented
                - sampling bias
            - other case: data is simply wrong
                - e.g. data was gathered by humans who just lied
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Enforcing a fair outcome

            - no matter in what way the data is biased: we want to **enforce a fair outcome**
                - idea: just tell the algorithm that it should treat all groups in the same way
            - question: **how do we define a fair outcome?**
                - really hard question
                - start with the simplest definition
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Example

            - task: predict whether someone should be hired ($Y=1$) or not ($Y=0$)
            - two races: <span style="color: blue">blue race</span> and <span style="color: green">green race</span>
                - blue: $S=0$, green: $S=1$
            - in the training set: **20% of blue** applicants were hired, **80% of green** applicants were hired
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Statistical parity

            $$
            P(\hat{Y}=1 | S=0) = P(\hat{Y}=1 | S=1)
            $$

            - $\hat{Y} \in \\{0,1\\}, S \in \\{0,1\\}$
            - $S$: sensitive attribute (for example gender, race)
            - $\hat{Y}$: prediction

            **meaning**: the probability to get a positive prediction is the same for each group
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Statistical parity

            - concrete: when evaluating the algorithm on the test set, both groups ($S=0$ and $S=1$)
              should have the same number of positive predictions ($\hat{Y}=1$)
            - in the hiring example from before: same percentage from both races will be hired (e.g. 50%)
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Side effects of statistical parity

            - enforcing statistical parity necessarily produces **lower accuracy**
            - consider this: we want to enforce 50% acceptance for the blue race but the training data only has 20% accepted
                - some individuals of the blue race have to be "misclassified"
                  ($\hat{Y} = 1$ instead of $\hat{Y} = 0$)
                  to make the numbers work
            - not surprising because we are computing accuracy against the *biased data*
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Trade-off: Fairness &ndash; Accuracy

            - usually we don't want to have too bad accuracy with respect to the data
            - goal: find algorithm that produces fair result at highest possible accuracy
            - otherwise it's easy: a *random classifier* is very fair (but useless)
                - random classifier: just predict $\hat{Y} = 1$ 50% of the time regardless of input
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Summary of Statistical Parity

            - very simple criterion
            - introduces Fairness &ndash; Accuracy trade-off
            - more sophisticated criteria are possible (based on causal graphs)
          </textarea>
        </section>

        <section>
          <p>&nbsp;</p>
          <h1>Bias introduced by the ML algorithm</h1>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Why would an ML algorithm introduce bias?

            Consider again the hiring example:
            - two features: SAT score and race (blue and green) of individuals
            - task: predict if they should be hired ($Y=1$) or not ($Y=0$)
            - composition of the dataset: 50% blue, 50% green. 20% of blue have $Y=1$, 80% of green have $Y=1$.
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Bias from algorithm

            The dataset is heavily skewed but let's ignore that for now and just try to make accurate predictions for this dataset.
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Bias from algorithm

            (Reminder: 20% of blue have $Y=1$, 80% of green have $Y=1$.)

            A simple way to make relatively accurate predictions:
            - ignore the SAT score
            - for green individuals always predict $Y=1$
            - for blue individuals always predict $Y=0$

            Result: up to 80% accuracy (80% in blue subgroup and 80% in green subgroup)
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Bias from algorithm

            - the dataset was already skewed but the algorithm's prediction are even more "unfair"
            - this is because it's easier to just base the decision on race than to figure out the effect of the SAT
            - this is an extreme case but similar things can actually happen
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2. What is Fair ML
            ### Bias from algorithm

            **What we do not want:** the algorithm "being lazy" in a subgroup

            **What we want:** the algorithm should make equally good predictions for all subgroups

            Criterion that enforces this: *Equality of Opportunity*
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Equality of Opportunity

            $$
            P(\hat{Y}=1 | S=0, Y=1) = P(\hat{Y}=1 | S=1, Y=1)
            $$

            - with $Y, \hat{Y} \in \\{0,1\\}$ and $S \in \\{0,1\\}$

            **meaning**: the probability of predicting 1, given that the label is 1, should be the same for all groups

            **in practice**: the TPR (true positive rate) should be the same in all groups
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Equalised Odds

            $$
            P(\hat{Y}=y | S=0, Y=y) = P(\hat{Y}=y | S=1, Y=y)
            $$

            for all possible values for $y$

            - stricter version of *Equality of Opportunity*
            - TPR and TNR (true negative rate) must be the same in all groups
                - TPR = TP / (TP + FN), TNR = TN / (TN + FP)
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Properties

            - **EoO** (Equality of Opportunity) and **EO** (Equalised Odds) both assume that the training data is correct
            - a perfect classifier (that always predicts the correct label) fulfills **EoO** and **EO**
            - however: a random classifier does as well
                - a random classifier achieves 50% TPR (and TNR) in all groups
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Accuracy&ndash;Fairness trade-off

            - achieving EoO or EO at low accuracy is easy (see random classifier)
            - here, even more than for Statistical Parity, it is important to find an algorithm with good fairness and good accuracy
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Fairness definitions

            - Discrimination in the law:
            - <span class="highlight">Direct discrimination</span> with respect to intent
            - <span class="highlight">Indirect discrimination</span> with respect to consequences
            <span class="citeme">Article 21, EU Charter of Fundamental Rights</span>
            - From the legal context to algorithmic fairness, e.g.:
            - Removing direct discrimination by <span class="highlight">not using group information at prediction</span>
            - Removing indirect discrimination by enforcing <span class="highlight">equality on the outcomes</span> between groups
          </textarea>
        </section>

        <section>
          <section data-markdown>
            <textarea data-template>
              ## 2.1 Definitions
              ### All statistics-based Fairness Criteria

              - Statistical Parity
              - Equalised Odds
              - Equality of Opportunity
              - Calibration by Group
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 2.1 Definitions
              ### Statistical Parity
              #### (**independence** based notion of fairness, $S \perp \hat{Y}$)

              $$
              P(\hat{Y}=1 | S=0) = P(\hat{Y}=1 | S=1)
              $$
              $$
              Y \in \\{0,1\\} \\\\
              S \in \\{0,1\\}
              $$
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 2.1 Definitions
              ### Equalised Odds
              #### (**separation** based notion of fairness, $S \perp \hat{Y} | Y$)

              $$
              P(\hat{Y}=1 | S=0, Y=y) = P(\hat{Y}=1 | S=1, Y=y)
              $$
              $$
              Y \in \\{0,1\\} \\\\
              S \in \\{0,1\\}
              $$
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 2.1 Definitions
              ### Equality of Opportunity
              #### (**separation** based notion of fairness, $S \perp \hat{Y} | Y$)

              $$
              P(\hat{Y}=1 | S=0, Y=1) = P(\hat{Y}=1 | S=1, Y=1)
              $$
              $$
              Y \in \\{0,1\\} \\\\
              S \in \\{0,1\\}
              $$
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 2.1 Definitions
              ### Calibration by Group
              #### (**sufficiency** based notion of fairness, $S \perp Y | \hat{Y}$)

              $$
              P(Y=1 | \hat{Y}=\hat{y}, S=s) = \hat{y}
              $$
              $$
              \hat{Y} \in \[0...1\] \\\\
              S \in \\{0,1\\}
              $$
            </textarea>
          </section>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Which fairness criteria to choose?</h3>
          <p>&nbsp;</p>
          <p><strong>Let's have all the fairness!</strong></p>

          <p>
            If we are fair with regards to all notions of fair, then we're
            fair... right?
          </p>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Which fairness criteria to choose?</h3>
          <p>Independence based fairness (i.e. Statistical Parity)</p>

          $$ \hat{Y} \perp S $$

          <p>Separation based fairness (i.e. Equalised Odds/Opportunity)</p>

          $$ \hat{Y} \perp S | Y $$
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Which fairness criteria to choose?</h3>

          <p>For both to hold (in binary classification), then either</p>
          <ul>
            <li>$S \perp Y$, our data is fair, or</li>
            <li>$\hat{Y} \perp Y$, we have a random predictor.</li>
          </ul>

          <p>
            Similarly, Sufficiency cannot hold with either notion of fairness.
          </p>
        </section>

        <section>
          <h1>Illustrative Example</h1>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Which fairness criteria to choose?</h3>
          <p>Consider a university, and we are in charge of administration!</p>
          <p>We can only accept 50% of all applicants.</p>
          <p>10,000 applicants are female and 10,000 of applicants are male.</p>
          <p>
            We have been tasked with being fair with regard to <b>gender</b>.
          </p>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>University Admission</h3>
          <p>
            We have an acceptance criteria that is highly predictive of success.
          </p>
          <p>
            80% of those who meet the acceptance criteria will successfully
            graduate.
          </p>
          <p>
            Only 10% of those who don't meet the acceptance criteria will
            successfully graduate.
          </p>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>University Admission</h3>
          <p>
            As we're a good university we have a lot of applications from people
            who don't meet the acceptance criteria.
          </p>
          <p>60% of female applicants meet the acceptance criteria.</p>
          <p>40% of male applicants meet the acceptance criteria.</p>
          <p>Remember, we can only accept 50% of all applicants</p>
          <h3>What should we do?</h3>
        </section>

        <section>
          <section>
            <h3>Truth Tables</h3>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td></td>
                  <td></td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td></td>
                  <td></td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td></td>
                  <td></td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td></td>
                  <td></td>
                </tr>
              </tbody>
            </table>
          </section>

          <section>
            <h3>Truth Tables</h3>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$10000 \times 0.6 \times 0.8$</td>
                  <td>$10000 \times 0.4 \times 0.1$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$10000 \times 0.6 \times 0.2$</td>
                  <td>$10000 \times 0.4 \times 0.9$</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$10000 \times 0.4 \times 0.8$</td>
                  <td>$10000 \times 0.6 \times 0.1$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$10000 \times 0.4 \times 0.2$</td>
                  <td>$10000 \times 0.6 \times 0.9$</td>
                </tr>
              </tbody>
            </table>
          </section>

          <section>
            <h3>Truth Tables</h3>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$4800$</td>
                  <td>$400$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$1200$</td>
                  <td>$3600$</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$3200$</td>
                  <td>$600$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$800$</td>
                  <td>$5400$</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section>
            <h2>University Admission</h2>
            <p>Our current system satisfies calibration-by-group!</p>
            $$Y \perp S | \hat{Y}$$
          </section>
        </section>

        <section>
          <p>
            How would we solve this problem being fair using Statistical Parity
            as our measure?
          </p>
          <section>???</section>
          <section>
            <p>Select 50% of applicants of both female and male applicants</p>
            <p>
              10% of qualified female applicants are being rejected whilst an
              additional 10% of unqualified males are being accepted.
            </p>
          </section>
          <section>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$5000 \times 0.8$</td>
                  <td>$(1000 \times 0.8) + (4000 \times 0.1)$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$5000 \times 0.2$</td>
                  <td>$(1000 \times 0.2) + (4000 \times 0.9)$</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$(4000 \times 0.8) + (1000 \times 0.1)$</td>
                  <td>$5000 \times 0.1$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$(4000 \times 0.2) + (1000 \times 0.9)$</td>
                  <td>$5000 \times 0.9$</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$4000$</td>
                  <td>$1200$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$1000$</td>
                  <td>$3800$</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$3300$</td>
                  <td>$500$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$1700$</td>
                  <td>$4500$</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td style="color: red">-800</td>
                  <td style="color: green">800</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td style="color: red">-200</td>
                  <td style="color: green">200</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td style="color: green">100</td>
                  <td style="color: red">-100</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td style="color: green">900</td>
                  <td style="color: red">-900</td>
                </tr>
              </tbody>
            </table>
          </section>
        </section>

        <section>
          <p>
            How would we solve this problem being fair using Equal Opportunity
            as our measure?
          </p>
          <section>???</section>
          <section>$$TPR = \frac{TP}{TP + FN}$$</section>
          <section>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$TP$</td>
                  <td>$FN$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$FP$</td>
                  <td>$TN$</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section>
            Select 55.5% of female applicants and 44.5% of male applicants,
            giving a TPR of 85.4% for both groups.
          </section>
          <section>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$4440$</td>
                  <td>$760$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$1110$</td>
                  <td>$3690$</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td>$3245$</td>
                  <td>$555$</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td>$1205$</td>
                  <td>$4995$</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section>
            <h4>Female Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td style="color: red">-360</td>
                  <td style="color: green">360</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td style="color: red">-90</td>
                  <td style="color: green">90</td>
                </tr>
              </tbody>
            </table>
            <h4>Male Applicants</h4>
            <table>
              <thead>
                <tr>
                  <th></th>
                  <th>Accepted</th>
                  <th>Not</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td>Actually Graduate</td>
                  <td style="color: green">45</td>
                  <td style="color: red">-45</td>
                </tr>
                <tr>
                  <td>Don't Graduate</td>
                  <td style="color: green">405</td>
                  <td style="color: red">-405</td>
                </tr>
              </tbody>
            </table>
          </section>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### All statistics-based Fairness Criteria

            - Statistical Parity
            - Equalised Odds
            - Equality of Opportunity
            - Calibration by Group
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.1 Definitions
            ### Individual Fairness

            "Similar individual should be treated similarly"

            <span class="citeme">Dwork et al. - Fairness Through Awareness</span>

            <br/>
            What is similar?

            What is similarly?
          </textarea>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Fairness based on similarity</h3>
          <ul>
            <li>
              First define a distance metric on your datapoints (i.e. how
              similar are the datapoints)
              <ul>
                <li>
                  Can be just Euclidean distance but is usually something else
                  (because of different scales)
                </li>
                <li>This is the hardest step and requires domain knowledge</li>
              </ul>
            </li>
          </ul>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Pre-processing based on similarity</h3>
          <ul>
            <li>
              An individual is then considered to be unfairly treated if it is
              treated differently than its "neighbours".
            </li>
            <li>
              For any data point we can check how many of the $k$ nearest
              neighbours have the same class label as that data point
            </li>
            <li>
              If the percentage is under a certain threshold then there was
              discrimination against the individual corresponding to that data
              point.
            </li>
            <li>
              Then: flip the class labels of those data points where the class
              label is considered unfair
            </li>
          </ul>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Considering similarity during training</h3>
          <p>Alternative idea:</p>
          <ul>
            <li>
              a classifier is fair if and only if the predictive distributions
              for any two data points are at least as similar as the two points
              themselves
              <ul>
                <li>
                  (according to a given similarity measure for distributions and
                  a given similarity measure for data points)
                </li>
              </ul>
            </li>
          </ul>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Considering similarity during training</h3>
          <ul>
            <li>Needed similarity measure for distributions</li>
            <li>
              Then add fairness condition as additional loss term to
              optimization loss
            </li>
          </ul>
        </section>

        <section>
          <h2>2.1 Definitions</h2>
          <h3>Which fairness criteria should we use?</h3>
          <p>
            We've seen several statistical definitions and Individual Fairness
          </p>
          <p>
            There's no right answer, all of the previous examples are "fair".
            It's important to consult
            <span class="highlight">domain experts</span> to find which is the
            best fit for each problem.
          </p>
          <p>There is no one-size fits all.</p>
        </section>

        <section>
          <h2>2.x Aside</h2>
          <h3>Delayed Impact of Fair Learning</h3>
          <p>In the real world there are implications.</p>
          <p>
            An individual doesn't just cease to exist after we've made our loan
            or bail decision.
          </p>
          <p>The decision we make has <b>consequences</b>.</p>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.x Aside
            ### Delayed Impact of Fair Learning

            Consider a bank's lending decision.

            The outcome is not simply reject or accept the applicant for a loan. In fact, there are multiple effects of this decision.

            If the applicant receives a loan, then goes on to successfully pay it back, then not only will the bank make a profit, but then the applicant's credit score will increase.

            This will make future loan decisions more favorable to the applicant.
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.x Aside
            ### Delayed Impact of Fair Learning

            We could think of this in terms of loss functions.

            The banks objective was to maximize profit. After deciding to give the loan, the loan was repaid (with interest). The bank made money.

            To maximize the objective (minimize the negative of the loss) the bank wants to give out as many loans that are likely to be repaid as possible.
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.x Aside
            ### Delayed Impact of Fair Learning

            In this scenario we have a slightly different concern. We don't want loans to be given to one subgroup (such as blue race), more often than members of another subgroup (green race).

            Our concern is that if the number of people who receive loans from one group outweigh the other, then as a whole, the credit rating will become a euphemism for race.
          </textarea>
        </section>

        <section>
          <section>
            <h2>2.x Aside</h2>
            <img src="images/outcome_curve.png" width="65%" />
          </section>
          <section>
            <h2>Possible areas</h2>
            <table>
              <tr>
                <th>Area</th>
                <th>Description</th>
              </tr>
              <tr>
                <td>Active Harm</td>
                <td>
                  Expected change in credit score of an individual is negative
                </td>
              </tr>
              <tr>
                <td>Stagnation</td>
                <td>Expected change in credit score of an individual is 0</td>
              </tr>
              <tr>
                <td>Improvement</td>
                <td>
                  Expected change in credit score of an individual is positive
                </td>
              </tr>
            </table>
          </section>
          <section>
            <h2>Possible areas</h2>
            <table>
              <tr>
                <th>Area</th>
                <th>Description</th>
              </tr>
              <tr>
                <td>Relative Harm</td>
                <td>
                  Expected change in credit score of an individual is less than
                  if the selection policy had been to maximize profit
                </td>
              </tr>
              <tr>
                <td>Relative Improvement</td>
                <td>
                  Expected change in credit score of an individual is better
                  than if the selection policy had been to maximize profit
                </td>
              </tr>
            </table>
          </section>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.2 Approaches
            ### How to enforce fairness?
    
            <span class="highlight">Three</span> different ways to enforce fairness:
            <img src="images/fairmethods.png" width=75% title="Fair Methods"/>
        
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            <br/><br/><br/><br/>
            More on those shortly...
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            # Simple Approaches
          </textarea>
        </section>

        <section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Removing a sensitive feature</h3>
            <p>
              "If we just remove the sensitive feature, then our model can't be
              unfair"
            </p>
            <p></p>
            <p>This doesn't work, why?</p>
          </section>

          <section>
            <h2>2.2 Approaches</h2>
            <h3>Removing a sensitive feature</h3>

            <p>
              Because ML methods are excellent at finding
              <span class="highlight">patterns in data</span>
            </p>

            <p><span class="citeme">Pedreschi et al.</span></p>
          </section>
        </section>

        <section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Reweighing</h3>
            <p>
              Kamiran & Calders determined that one source of unfairness was
              imbalanced training data.
            </p>
            <p>Simply count the current distribution of demographics</p>
            <img src="images/reweighing_before.png" />
          </section>

          <section>
            <h2>2.2 Approaches</h2>
            <h3>Reweighing</h3>
            <p>
              Then either up/down-sample or assign instance weights to members
              of each group in the training set so that the results are
              "normalised".
            </p>
            <img src="images/reweighing_after.png" />
          </section>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.2 Approaches
            ### Fair Representations

            Popular approach is to add produce a "fair" representation. Consider that we have 2 roles, a data vendor, who is charge of collecting the data and preparing it. Our other role is a data user, someone who will be making predictions based on our data.

            The data vendor is concerned that the data user may be using their data to make unfair decisions. So the data vendor decides to learn a new, fair representation.
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 2.2 Approaches
            ### Fair Representations
            The state-of-the-art methods for fair representation are based on an *adversarial* network, using a "Gradient-Reversal Layer" from Ganin et al.

            <span class="citeme">Ganin et al. 2016</span>

            ![](./images/adversarial.svg)
          </textarea>
        </section>

        <section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Problems with doing this?</h3>
            <br />
            <h4>Any Ideas?</h4>
          </section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Problems with doing this?</h3>
            <h4>What does this representation mean?</h4>
            <p>
              The learned representation is uninterpretable by default. Recently
              Quadrianto et al constrained the representation to be in the same
              same as the input so that we could look at what changed
            </p>
          </section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Problems with doing this?</h3>
            <h4>What if the vendor data user decides to be fair as well?</h4>
            <p>
              Referred to as "fair pipelines". Work has only just begun
              exploring these. Current research shows that these don't work (at
              the moment!)
            </p>
          </section>
        </section>

        <section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Variational Fair Autoencoder</h3>
            <br />
            <p>
              idea: Let's "disentangle" the sensitive attribute using the
              variational autoencoder framework!
            </p>
          </section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Variational Fair Autoencoder</h3>
            <img src="images/vfae_graphical_model.png" />
            <p><span class="citeme">Louizos et al. 2017</span></p>
          </section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Variational Fair Autoencoder</h3>
            <p>
              Where $z_1$ and $z_2$ are encouraged to confirm to a prior
              distribution
            </p>
          </section>
          <section>
            <h2>2.2 Approaches</h2>
            <h3>Problems with doing this?</h3>
            <p>Similar criticisms to adversarial model, but more principled - we're able to be more explicit about our assumptions</p>
          </section>
        </section>

        <section>
          <h2>2.2 Approaches</h2>
          <h3>How to enforce fairness?</h3>
          <h4>During Training</h4>
          Instead of building a fair representation, we just make the fairness
          constraints part of the objective during training of the model. An
          early example of this is by Zafar et al.
        </section>

        <section>
          <h2>2.2 Approaches</h2>
          <h3>How to enforce fairness?</h3>
          <h4>During Training</h4>
          <p>Given we have a loss function, $\mathcal{L}(\theta)$.</p>
          <p>In an unconstrained classifier, we would expect to see</p>
          $$ \min{\mathcal{L}(\theta)} $$
        </section>

        <section>
          <h2>2.2 Approaches</h2>
          <h3>How to enforce fairness?</h3>
          <h4>During Training</h4>
          <p>
            To reduce Disparate Impact, Zafar adds a constraint to the loss
            function.
          </p>

          $$ \begin{aligned} \text{min } & \mathcal{L}(\theta) \\ \text{subject
          to } & P(\hat{y} \neq y|s = 0) − P(\hat{y} \neq y|s = 1) \leq \epsilon
          \\ \text{subject to } & P(\hat{y} \neq y|s = 0) − P(\hat{y} \neq y|s =
          1) \geq -\epsilon \end{aligned} $$
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Motivation</h3>
          <ul>
            <li>
              Fairness methods so far have only looked at
              <em>correlations</em> between the sensitive attribute and the
              prediction
            </li>

            <li>But isn't a causal path more important?</li>
          </ul>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Example:</h3>
          <p>Admissions at Berkeley college.</p>
          <p>
            Men were found to be accepted with much higher probability. Was it
            discrimination?
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Was it discrimination?</h3>
          <p>
            Not necessarily! The reason was that women were applying to more
            competitive departments.
          </p>
          <p>
            Departments like medicine and law are very competitive (hard to get
            in).
          </p>
          <p>
            Departments like computer science are much less competetive (because
            it's boring ;).
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <ul>
            <li>$R$: admission decision</li>
            <li>$A$: gender</li>
            <li>$X$: department choice</li>
          </ul>

          <p>
            <img src="images/berkeley_admission.png" style="width: 30%" />
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <p>
            If we can understand what <b>causes</b> unfair behavior, then we can
            take steps to mitigate it.
          </p>
          <p class="fragment">
            Basic idea: a sensitive attribute may only affect the prediction via
            legitimate causal paths.
          </p>
          <p class="fragment">But how do we model causation?</p>
        </section>

        <section>
          <h2>2.3 Causal Graphs</h2>
          <p><b>Solution:</b> build causal graphs of your problem</p>
          <p>
            <b>Problem:</b> causality cannot be inferred from observational data
          </p>
          <ul>
            <li>Observational data can only show correlations</li>
            <li>
              For causal information we have to do <i>experiments</i>. (But that
              is often not ethical.)
            </li>
            <li>Currently: just guess the causal structure</li>
          </ul>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Path-specific causal fairness</h3>
          <p>
            $A$ is a sensitive attribute, and its direct effect on $Y$ and
            effect through $M$ is considered unfair. But $L$ is considered
            admissible.
          </p>

          <p>
            <img src="images/path-specific-fairness.png" style="width: 70%" />
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Path-specific causal fairness</h3>
          <p>With a structural causal model (SCM), like this:</p>
          <p><img src="images/scm.png" style="width: 50%" /></p>
          <p>
            You can figure out exactly how each feature should be incorporated
            in order to make a fair prediction.
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Path-specific causal fairness</h3>
          <ul>
            <li>
              <b>Advantage:</b> we can figure out <em>exactly</em> how to avoid
              unfairness
            </li>
            <li>
              <b>Problem:</b> requires an SCM, which we usually don't have and
              which is hard to get
            </li>
            <li>
              <b>Problem:</b> we have to decide for each feature individually
              whether it's admissible
              <ul>
                <li>
                  This will not work for computer vision tasks where the raw
                  features are pixels
                </li>
              </ul>
            </li>
          </ul>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Counterfactual Fairness</h3>
          <ul>
            <li>
              slightly different idea but also based on SCMs (structural causal
              model)
            </li>
            <li>
              we're basically asking: in the counterfactual world where your
              gender was different, would you have been accepted?
              <ul>
                <li>
                  <em>counterfactual</em>: something that has not happened or is
                  not the case
                </li>
              </ul>
            </li>
          </ul>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Counterfactuals</h3>
          <p>Example of a counterfactual statement:</p>
          <blockquote>
            If Oswald had not shot Kennedy, no-one else would have.
          </blockquote>
          <p>
            This is counterfactual, because Oswald did in fact shoot Kennedy.
          </p>
          <p>It's a claim about a counterfactual world.</p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Counterfactual Fairness</h3>
          <p>$U$: set of all unobserved background variables</p>
          <p>$P(\hat{y}_{s=i}(U) = 1|x, s=i)=P(\hat{y}_{s=j}(U) = 1|x, s=i)$</p>
          <p>$i, j \in \{0, 1\}$</p>
          <p>
            $\hat{y}_{s=k}$: prediction in the counterfactual world where $s=k$
          </p>
          <p>
            <b>practical consequence:</b> $\hat{y}$ is counterfactually fair if
            it does not causally depend (as defined by the causal model) on $s$
            or any descendants of $s$.
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>The problem of getting the SCM</h3>
          <ul>
            <li>
              Causal models cannot be learned from observational data only
              <ul>
                <li>You can try, but there is no <em>unique</em> solution</li>
              </ul>
            </li>

            <li>Different causal models can lead to very different results</li>
          </ul>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Example with Causal Graphs</h3>
          <p><b>Example:</b> Law school success</p>
          <p>
            Task: given GPA score and LSAT (law school entry exam), predict
            grades after one year in law school: FYA (first year average)
          </p>
          <p>
            Additionally two sensitive attributes: <i>race</i> and <i>gender</i>
          </p>
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Example with Causal Graphs</h3>
          <p>Two possible graphs</p>
          <img src="./images/lsat_causal.png" />
        </section>

        <section>
          <h2>2.3 Causality</h2>
          <h3>Causality &mdash; Summary</h3>
          <ul>
            <li>
              Causal models are very promising and arguably the "right" way to
              solve the problem.
            </li>
            <li>However these models are difficult to obtain.</li>
          </ul>
        </section>

        <section>
          <h1>Practical Applications</h1>
        </section>

        <section>
          <h1>Transparency and Fairness</h1>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 3.1 Transparency
            ### Pre-processing

            - Many <span class="highlight">adversarial-based</span> fair representation learning approaches, e.g. using a "Gradient-Reversal Layer"

            <img src="images/models/adversarialfair.png" width=38% title="Adversarially Fair"/>

            <span class="citeme">Ganin et al.,Domain-adversarial training of neural networks, JMLR 2016</span>
            <span class="citeme">Edwards and Storkey, Censoring representations with an adversary, ICLR 2016</span>
            <span class="citeme">Beutel et al., Data decisions and theoretical implications when adversarially learning fair representations, Jul 2017</span>
            <span class="citeme">Madras et al., Learning adversarially fair and transferable representations, ICML 2018</span>
          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## 3.1 Transparency
            ### Problems?
            
            Can we visualise/understand the latent representation $z$?
          </textarea>
        </section>

        <section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair in the data domain
              - A fair representation $T_\omega(X)$ which has the sensitive attribute removed, but remains in the space of $X$
              - Main disentangling assumption:
              $$\phi(X) = \phi(T_{\omega}(X)) + \underbrace{\phi (\lnot T_{\omega}(X))}_{\text{Spurious}}$$
              - <span class="highlight">Spurious</span>: dependent on the sensitive attribute.
              - <span class="highlight">Non-Spurious</span>: independent of the sensitive attribute.

              <span class="citeme">NQ, Sharmanska, Thomas, Discovering fair representations in the data domain, CVPR 2019</span>
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair in the data domain

              <img src="images/models/Architecture.png" width=70% title="Architecture"/>

              - <span class="highlight">No GANs?</span> To perform translation from the input to the non-spurious
              domain via GANs, we need to have real data (e.g. images, tabular data) of both domains
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair in the data domain

              The optimisation problem:

              $\min_{T_\omega} \sum_{n=1}^N\mathcal{L}(T_{\omega}(\mathbf{x}^n),y^n)$

              $+ \lambda_1 \sum_{n=1}^{N}\|\mathbf{x}^n-T_{\omega}(\mathbf{x}^n)\|_2^2$

              $+ \lambda_2 \\|(- \mathrm{HSIC}(\phi (\lnot T_{\omega}(\mathbf{x})),S|Y=+1) + \mathrm{HSIC}(\phi ( T_{\omega}(\mathrm{x})),S|Y=+1)\\|)$
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair in the data domain

              Experiments on the <span class="highlight">Adult Income dataset</span>
              - This dataset is frequently used in fairness research.
              - It has $45,222$ data points, each with $14$ features such as relationship status, gender, occupation.
              - We use gender as a binary sensitive attribute.
              - The binary task is to predict whether a person is a high-earner (>\$50K p.a.) or not.
            </textarea>
          </section>

          <section>
            <h2>3.1 Transparency</h2>
            <h3>Results on the adult income dataset</h3>
            <div style="display: flex; justify-content: space-between;">
              <div style="width: 49%;">
                  <p>- Analysis on the relationship feature</p>
                  <p>
                      - Feature values of the
                      <span class="highlight">minority group are transformed to match the majority group</span>
                  </p>
                  <p>- Here, the wife value is translated to husband</p>
              </div>
              <div style="width: 49%;">
                  <img src="images/output.png" style="width: 70%;" title="outputs" />
              </div>
          </div>
          </section>

          <section>
            <h2>3.1 Transparency</h2>
            <h3>Results on the adult income dataset</h3>
            <p>Interpretable can be fair!</p>

            <small>
              <table>
                <thead>
                  <tr>
                    <td></td>
                    <td colspan="2">original $X$</td>
                    <td colspan="2">
                      <span class="highlight">fair & interpretable $X$</span>
                    </td>
                    <td colspan="2">latent embedding $Z$</td>
                  </tr>
                </thead>
                <tbody>
                  <tr>
                    <td></td>
                    <td width="10%">Accuracy $\uparrow$</td>
                    <td width="10%">Eq. Opp $\downarrow$</td>
                    <td width="10%">Accuracy $\uparrow$</td>
                    <td width="10%">Eq. Opp $\downarrow$</td>
                    <td width="10%">Accuracy $\uparrow$</td>
                    <td width="10%">Eq. Opp $\downarrow$</td>
                  </tr>
                  <tr>
                    <td width="10%">LR</td>
                    <td width="10%">$85.1\pm0.2$</td>
                    <td width="10%">$\mathbf{9.2\pm2.3}$</td>
                    <td width="10%">$84.2\pm0.3$</td>
                    <td width="10%">$\mathbf{5.6\pm2.5}$</td>
                    <td width="10%">$81.8\pm2.1$</td>
                    <td width="10%">$\mathbf{5.9\pm4.6}$</td>
                  </tr>
                  <tr>
                    <td width="10%">SVM</td>
                    <td width="10%">$85.1\pm0.2$</td>
                    <td width="10%">$\mathbf{8.2\pm2.3}$</td>
                    <td width="10%">$84.2\pm0.3$</td>
                    <td width="10%">$\mathbf{4.9\pm2.8}$</td>
                    <td width="10%">$81.9\pm2.0$</td>
                    <td width="10%">$\mathbf{6.7\pm4.7}$</td>
                  </tr>
                  <tr>
                    <td width="20%">Fair Reduction LR</td>
                    <td width="10%">$85.1\pm0.2$</td>
                    <td width="10%">$\mathbf{14.9\pm1.3}$</td>
                    <td width="10%">$84.1\pm0.3$</td>
                    <td width="10%">$\mathbf{6.5\pm3.2}$</td>
                    <td width="10%">$81.8\pm2.1$</td>
                    <td width="10%">$\mathbf{5.6\pm4.8}$</td>
                  </tr>
                  <tr>
                    <td width="10%">Fair Reduction SVM</td>
                    <td width="10%">$85.1\pm0.2$</td>
                    <td width="10%">$\mathbf{8.2\pm2.3}$</td>
                    <td width="10%">$84.2\pm0.3$</td>
                    <td width="10%">$\mathbf{4.9\pm2.8}$</td>
                    <td width="10%">$81.9\pm2.0$</td>
                    <td width="10%">$\mathbf{6.7\pm4.7}$</td>
                  </tr>
                  <tr>
                    <td width="10%">Kamiran & Calders LR</td>
                    <td width="10%">$84.4\pm0.2$</td>
                    <td width="10%">$\mathbf{14.9\pm1.3}$</td>
                    <td width="10%">$84.1\pm0.3$</td>
                    <td width="10%">$\mathbf{1.7\pm1.3}$</td>
                    <td width="10%">$81.8\pm2.1$</td>
                    <td width="10%">$\mathbf{4.9\pm3.3}$</td>
                  </tr>
                  <tr>
                    <td width="10%">Kamiran & Calders SVM</td>
                    <td width="10%">$85.1\pm0.2$</td>
                    <td width="10%">$\mathbf{8.2\pm2.3}$</td>
                    <td width="10%">$84.2\pm0.3$</td>
                    <td width="10%">$\mathbf{4.9\pm2.8}$</td>
                    <td width="10%">$81.9\pm2.0$</td>
                    <td width="10%">$\mathbf{6.7\pm4.7}$</td>
                  </tr>
                  <tr>
                    <td width="10%">Zafar et al.</td>
                    <td width="10%">$85.0\pm0.3$</td>
                    <td width="10%">$\mathbf{1.8\pm0.9}$</td>
                    <td width="10%">---</td>
                    <td width="10%">---</td>
                    <td width="10%">---</td>
                    <td width="10%">---</td>
                  </tr>
                </tbody>
              </table>
            </small>
          </section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair in the data domain

              Experiments on <span class="highlight"> CelebA dataset</span>
              - $202,599$ celebrity images with $40$ attributes.
              - We use gender as a binary sensitive attribute.
              - We use the subjective attribute memorability as the classification task.
            </textarea>
          </section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Results on the CelebA dataset

              Examples of the spurious residual and non-spurious translation
              <div style="max-width: 80%; margin: 0 auto; display: flex; flex-direction: column; border: 1px solid #ddd; padding: 10px;">

                <!-- Non-spurious Section -->
                <div style="display: flex; border-bottom: 2px solid #000; padding-bottom: 10px;">
                    <div style="flex-basis: 10%; border: none; font-weight: bold;">Non-spurious</div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/006126.jpg" style="width: 100%;" title="Im1" /></div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/015365.jpg" style="width: 100%;" title="Im1" /></div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/015505.jpg" style="width: 100%;" title="Im1" /></div>
                </div>
            
                <!-- Spurious Section -->
                <div style="display: flex; padding-top: 10px;">
                    <div style="flex-basis: 10%; border: none; font-weight: bold;">Spurious</div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/006126res.jpg" style="width: 100%;" title="RIm1" /></div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/015365res.jpg" style="width: 100%;" title="RIm1" /></div>
                    <div style="flex-basis: 20%; padding: 0 10px;"><img src="images/celeba_res/015505res.jpg" style="width: 100%;" title="RIm1" /></div>
                </div>
              </div>
            </textarea>
          </section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Results on the CelebA dataset

              In the semantic attribute domain ... same conclusion (<span class="highlight">changes in the eyes and lips regions</span>)
              <div style="max-width: 80%; margin: 0 auto; display: flex; flex-direction: column; border: 1px solid #ddd; padding: 10px;">
                <div style="flex-basis: 30%; padding: 0 10px;"><img src="images/features.png" style="width: 100%;" title="features" />
              </div>
            </textarea>
          </section>
        </section>

        <section>
          <h2>3.1 Transparency</h2>
          <h3>Problems?</h3>
          <p>Spurious and non-spurious visualisations are not exciting!</p>
          <p>Transferability to new tasks</p>
          <p>Disentangling assumption is quite restrictive</p>
          <!-- <img src="images/cmnist%20train.png" width="30%" /> -->
          <!-- <img src="images/cmnist%20test.png" width="30%" /> -->
        </section>
        <section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 1
              We use deep generative modelling:

              $\mathcal{L} = \mathbb{E}{q_{\theta_{enc}}(z|x)}[\log p_{\theta_{dec}}(x|z, s) - \log p_{\theta_{dec}}(s|z)]- \beta D_{KL}(q_{\theta_{enc}}(z |x) \| p(z))$
              - Representing data points in a latent space
              - Disentangling the latent space (previous work was on the reconstruction space) into two components:
                  - <span class="highlight">Spurious</span>: dependent on the sensitive attribute.
                  - <span class="highlight">Non-Spurious</span>: independent of the sensitive attribute.
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 1
              <img src="images/models/CVAE.png" width="60%">
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 1
              <img src="images/train_original_x_31500.png" width="30%">
              <img src="images/train_reconstruction_y_31500.png" width="30%">
              <img src="images/train_reconstruction_s_31500.png" width="30%">
            </textarea>
          </section>
        </section>

        <section>
          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 2
              We use deep generative modelling:

              $\min_\theta \mathbb{E}_x (- \log p_\theta (x|z)) + \lambda I(z_d, s)$

              - Representing data points in a latent space
              - Disentangling the latent space (previous work was on the reconstruction space) into two components:
                  - <span class="highlight">Spurious</span>: dependent on the sensitive attribute.
                  - <span class="highlight">Non-Spurious</span>: independent of the sensitive attribute.
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 2
              <img src="images/models/CFlow.png" width="30%">
            </textarea>
          </section>

          <section data-markdown>
            <textarea data-template>
              ## 3.1 Transparency
              ### Fair, and transferable... Idea 2
              <img src="images/train_original_x_15000.png" width="30%">
              <img src="images/train_reconstruction_y_15000.png" width="30%">
              <img src="images/train_reconstruction_s_15000.png" width="30%">
            </textarea>
          </section>
        </section>

          <section>
            <section data-markdown>
              <textarea data-template>
                ## 3.x Aside
                ### Contrastive examples

                - The <span class="highlight">ideal dataset</span> contains an imaginary data point for each person, i.e. the one inside the black box, whereby we <span class="highlight">intervene</span> and set the gender attribute to the opposite that is in real life
                <span class="citeme">Thomas, NQ, Imagined Examples for Fairness: Sampling Bias and Proxy Labels, Sep 2019</span>
                <span class="citeme">Sharmanska, Hendricks, Darrell, NQ, Contrastive examples for addressing the tyranny of the majority, May 2019</span>

                <img src="images/balanced_Page_1.png" width=50% title="Balanced1"/>
                <img src="images/balanced_Page_2.png" width=30% title="Balanced2"/>
              </textarea>
            </section>
            <section data-markdown>
              <textarea data-template>
                ## 3.x Aside
                ### Contrastive on Adult Income dataset

                - Change in the <span class="highlight">relationship</span> and <span class="highlight">marital status</span>, i.e. from Husband and Married-civ-spause in real samples to Unmarried and/or Wife in contrastive samples

                <img src="images/adult_real.png" width=30% title="Real"/>
                <img src="images/adult_contrastive.png" width=30% title="Contrastive"/>
                <img src="images/adult_diff.png" width=30.5% title="Difference"/>

              </textarea>
            </section>

            <section data-markdown>
              <textarea data-template>
                ## 3.x Aside
                ### Contrastive on Adult Income dataset
                - Connections to <span class="highlight">counterfactuals</span> based on the Nabi \& Shpitser's (2018) causal graph
                - To remove bias towards females, the direct effect of gender on income, as well as, the <span class="highlight">effect of gender on income through marital status have to be suppressed</span>
                - Experimental results
                <small>
                    <table>
                        <tbody><tr>
                            <td></td>
                            <td width="10%">Accuracy $\uparrow$</td>
                            <td width="10%">TPR Diff. $\downarrow$</td>
                            <td width="10%">FPR Diff. $\downarrow$</td>
                        </tr>
                        <tr>
                            <td width="10%">LR (Real)</td>
                            <td width="10%">$85.16\pm0.14$ </td>
                            <td width="10%">$7.98\pm1.52$ </td>
                            <td width="10%">$7.23\pm0.41$</td>
                        </tr>
                        <tr>
                            <td width="10%">Kamiran & Calders LR (Real)</td>
                            <td width="10%">$84.37\pm0.28$ </td>
                            <td width="10%">$14.3\pm1.16$ </td>
                            <td width="10%">$1.17\pm0.29$ </td>
                        </tr>
                        <tr>
                            <td width="20%">LR (Real and NN contrastive)</td>
                            <td width="10%">$85.01\pm0.25$ </td>
                            <td width="10%">$14.80\pm1.90$ </td>
                            <td width="10%">$8.20\pm0.51$ </td>
                        </tr>
                        <tr>
                            <td width="20%">LR (Real and GAN contrastive)</td>
                            <td width="10%">$82.48\pm0.44$ </td>
                            <td width="10%">$4.95\pm3.67$ </td>
                            <td width="10%">$3.94\pm1.33$ </td>
                        </tr>
                            </tbody>
                    </table>
                </small>
              </textarea>
            </section>

            <section data-markdown>
              <textarea data-template>
                ## 3.x Aside
                ### Interpretability with contrastive examples

                - All previous work with adversarial learning try to <span class="highlight">remove</span> sensitive attributes from data
                - Instead, we use adversarial learning to <span class="highlight">generate</span> data points with pre-specified sensitive attributes (contrastive examples)
                - Contrastive examples can be easily interpreted because they do have the semantic meaning of the input
              </textarea>
            </section>
          </section>

          <section>
            <section>
              <h2>3.x Aside</h2>
              <h3>Fairness without constraints</h3>
              <ul>
                <li>
                  Most fairness methods for "during training" are formulated as
                  constraints
                </li>
                <li>
                  This means either
                  <ul>
                    <li>
                      Using very special models where constrained-based optimization
                      is possible
                    </li>
                    <li>Using Lagrange multipliers</li>
                  </ul>
                </li>
              </ul>
            </section>

            <section>
              <h2>3.x Aside</h2>
              <h3>Lagrange multipliers</h3>
              <p>Constrained optimization problem:</p>
              <p>
                $$\min\limits_\theta\, f(\theta) \quad\text{s.th.}~ g(\theta) = 0$$
              </p>
              <p>This gets reformulated as</p>
              <p>$f(\theta) + \lambda g(\theta)$</p>
              <p>
                where $\lambda$ has to be determined, but is usually just set to
                some value
              </p>
            </section>

          <section>
            <h2>3.x Aside</h2>
            <h3>Shortcomings of this</h3>
            <ul>
              <li>
                The value of $\lambda$ has no real meaning and is usually just set
                to arbitrary values
              </li>
              <li>It's hard to tell whether the constraint is fulfilled</li>
            </ul>
          </section>

          <section>
            <h2>3.x Aside</h2>
            <h3>Alternate approach</h3>
            <ul>
              <li>Introduce <em>pseudo labels</em> $\bar{y}$.</li>
              <li>Make sure $\bar{y}$ is fair ($\bar{y}\perp s$).</li>
              <li>Use $\bar{y}$ as prediction target.</li>
            </ul>

            <p>
              $$P(y=1|x,s,\theta)=\sum\limits_{\bar{y}\in\{0,1\}}
              P(y=1|\bar{y},s)P(\bar{y}|x,\theta)$$
            </p>
          </section>

          <section>
            <h2>3.x Aside</h2>
            <h3>Alternate approach</h3>
            <p>For demographic parity:</p>
            <p>$P(\bar{y}|s=0)=P(\bar{y}|s=1)$</p>
            <p>
              This allows us to compute $P(y=1|\bar{y},s)$. Which is all we need.
            </p>
            <p>
              Details:
              <em>Tuning Fairness by Balancing Target Labels</em> (Kehrenberg et
              al., 2020)
            </p>
          </section>

          <section>
            <h2>3.x Aside</h2>
            <h3>Multinomial sensitive attribute</h3>
            <p>
              The usual metrics for Demographic Parity are only defined for binary
              attributes: it's either a <em>ratio</em> or a <em>difference</em> of
              two probabilities.
            </p>

            <p><b>Alternative</b>:</p>
            <p>Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient (HGR)</p>
            <ul>
              <li>defined on the domain [0,1]</li>
              <li>$HGR(Y,S) = 0$ iff $Y\perp S$</li>
              <li>
                $HGR(Y,S) = 1$ iff there is a deterministic function to map
                between them
              </li>
            </ul>
          </section>
        </section>

        <section>
          <h2>3.2 RLHF</h2>
          <h3>Reinforcement Learning with Human Feedback</h3>
          <p>So far we've looked at statistical fairness definition (Sec 2.1)</p>
          <p>We then went onto approaches that use Generative Models (Secs 2.2 & 3.1)</p>
          <p>All discussion so far about fairness through the lens of classification</p>
          <p>But the most famous use of "AI" doesn't see the world in terms of classification</p>
          <h4>How does ChatGPT produce outputs that aren't completely unhinged?</h4>
        </section>

        <section>
          <h2>3.2 RLHF</h2>
          <h3>Reinforcement Learning with Human Feedback</h3>
          <img src="images/rlhf.svg" style="border: none;"/>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## Contents

            1. Intro and notation
            2. What is fairness in machine learning and why should I care?
                1. Algorithmic fairness definitions
                2. Approaches to enforce algorithmic fairness
                3. Causality and Counterfactual Fairness
            3. Practical Applications
                1. Transparency in algorithmic fairness
                2. RLHF

          </textarea>
        </section>

        <section data-markdown>
          <textarea data-template>
            ## Take home messages

            - Fairness is _Domain Specific_.
            - If you don't think about bias, it will come back to haunt you.
            - Accuracy only tells part of the story.
            - Fairness is an active and exciting research topic.
          </textarea>
        </section>

        
      </div>
    </div>

    <script type="module" src="settings.js"></script>
  </body>
</html>