-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsussex.html
1135 lines (936 loc) · 50.8 KB
/
sussex.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<title>Fairness in Machine Learning</title>
<link rel="stylesheet" href="dist/reset.css">
<link rel="stylesheet" href="dist/reveal.css">
<link rel="stylesheet" href="custom_themes/sussex_dark.css" id="theme">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="plugin/highlight/monokai.css">
</head>
<style>
.citeme {
float: right;
color: #fff;
font-size: 12pt;
font-style: italic;
margin: 0;
}
</style>
<body>
<div id="hidden" style="display:none;"><div id="static-content">
<footer>
<p><font color=#fff></font></p>
</footer>
</div> </div>
<div class="reveal"><div class="slides">
<section data-background-color="#000">
<h1><span class="highlight">Fairness in <br>Machine Learning</span></h1>
<p>by Novi Quadrianto</p>
<p>with thanks to Oliver Thomas and Thomas Kehrenberg</p>
<img src="images/sussex_logo.png" style="width: 14%; border: none;"/>
</section>
<section data-background-color="#000" data-markdown><textarea data-template>
## Contents
- What and why of fairness in machine learning
- Algorithmic fairness definitions
- Approaches to enforce algorithmic fairness
- Bayesian and deep approaches
- Interpretability in algorithmic fairness
</textarea></section>
<section data-background-color="#000" data-markdown><textarea>
## Take Home Messages
- Bias (a systematic deviation from a true state) is everywhere
- Uncontrolled bias can cause unfairness in machine learning
- If you don't think about bias, it will come back to haunt you
</textarea></section>
<section data-background-color="#000">
<h2>Machine learning systems</h2>
<p align="left">Machine learning systems are being implemented in all walks of life</p>
<table>
<tbody>
<tr>
<td style="border: none;" width="30%"> <img src="images/scs.jpg" width=65% title="Social Credit System" style="margin:-20px 0px"/> <p style="font-size:10px">Picture credit: Kevin Hong</p></td>
<td style="border: none;" width="30%"> <img src="images/algowatch.jpg" width=65% title="Automating Society" style="margin:-20px 0px"/><p style="font-size:10px">Picture credit: AlgorithmWatch</p></td>
<td style="border: none;" width="30%"> <img src="images/CDEI.png" width=65% title="Automating Society" style="margin:-20px 0px"/><p style="font-size:10px">Picture credit: Centre for Data Ethics and Innovation, UK</p></td>
</tr>
<tr>
<td style="border: none;" width="30%">Social credit system, China</td>
<td style="border: none;" width="30%">Personal budget calculation, UK</td>
<td style="border: none;" width="30%">Financial services, Crime and justice, </td>
</tr>
<tr>
<td style="border: none;" width="30%"></td>
<td style="border: none;" width="30%">Loan decision, Finland</td>
<td style="border: none;" width="30%">Recruitment,</td>
</tr>
<tr>
<td style="border: none;" width="30%"></td>
<td style="border: none;" width="30%">etc.</td>
<td style="border: none;" width="30%">Local government</td>
</tr>
</tbody>
</table>
</section>
<section data-background-color="#000" data-markdown><textarea data-template>
## Classification
- Given some input $X$, predict a class label $Y \in \\{1, ..., C\\}$
- $X$ is usually a **vector**
- often with high number of dimensions
- Simplest case: **binary classification**, $Y \in \\{-1, 1\\}$
- for example: to give a loan ($Y=1$) or not ($Y=-1$) to this person?
</textarea></section>
<section data-background-color="#000" data-markdown><textarea data-template>
## Classification
- We are looking to train a function $f$ that maps $\mathcal{X}$ to $\mathcal{Y}$
- The output is the prediction: $\hat{Y} = f(X)$
- we want $\hat{Y}$ to be as close as possible to the label $Y$
- $f$ can be implemented as
- a deep neural network
- an SVM
- a Gaussian process model
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Training data
- Training data: a set of pairs $(X,Y)$
- input data $X$ with corresponding label $Y$
- We are looking for model that works well on the training data
- If we make predictions on data that is *very different* from the training data,
the model will perform badly
- Problem if the training data encode <span class="highlight">societal biases</span>
</textarea></section>
<section data-background-color="#fff">
<img src="images/pp_mb.png" width=72% title="Pro-Publica - Machine Bias"/>
</section>
<section data-background-color="#fff">
<img width=72% src="images/amazon.png" title="Amazon CV Screening"/>
</section>
<section data-background-color="#fff">
<img width=90% src="images/sveaekonomi.png" title="Svea Ekonomi"/>
</section>
<section data-background-color="#fff">
<img src="images/norman.png" width=58% title="Norman"/>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Fairness in machine learning
- A fair machine learning system takes biased datasets and outputs non-discriminatory decisions to people with differing <span class="highlight">protected attributes</span> such as race, gender, and age
- For example, ensuring classifier to be equally accurate on male and female populations
<img width=90% src="images/fairML.png" title="Fair ML"/>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Sources of unfairness
- The problem can be divided into <span class="highlight">two categories</span> (both types of bias can appear together):
- Bias stemming from biased training data
- Bias stemming from the algorithms themselves
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Bias from training data
- <span class="highlight">Sampling bias</span>: the data sample on which the algorithm is trained for is not representative of the overall population
- <span class="highlight">Selective labels</span>: only observe the outcome of one side of the decision
- <span class="highlight">Proxy labels</span>: e.g. for predictive policing, we do not have data on who commits crimes, and only have data on who is arrested
<span class="citeme">Chouldechova \& Roth: The frontiers of fairness in machine learning, Oct 2018</span>
<span class="citeme">Tolan: Fair and unbiased algorithmic decision making: current state and future challenges, Dec 2018</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Bias from algorithm
- <span class="highlight">Tyranny of the majority</span>: it is simpler to fit to the majority groups than to the minority groups because of generalization
- <span class="highlight">Feedback effects</span>: model at time $t+1$ has to consider training data plus decisions of the model at time $t$
<span class="citeme">Chouldechova \& Roth: The frontiers of fairness in machine learning, Oct 2018</span>
<span class="citeme">Tolan: Fair and unbiased algorithmic decision making: current state and future challenges, Dec 2018</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Sampling bias ➔ the tyranny
- In the <span class="highlight">imSitu</span> situation recognition <span class="highlight">dataset</span>, the activity cooking is over 33\% more likely to involve females than males in a training set, and a trained algorithm further <span class="highlight">amplifies</span> the disparity to 68\%
<span class="citeme">Zhao et al.: Men also like shopping, EMNLP 2017</span>
<img src="images/women_also_snowboard/example3.png" title="Men also like shopping"/>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Sampling bias ➔ the tyranny
- <span class="highlight">The reason is</span>: the algorithm predicts the gender from the activity and not from looking at the person
<span class="citeme">Anne Hendricks et al.: Women also snowboard, ECCV 2018</span>
<img src="images/women_also_snowboard/example2.png" title="Women also snowboard"/>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Sampling bias ➔ the tyranny
- In the <span class="highlight">UCI Adult Income dataset</span>, 30\% of the male individuals earn more than 50K per year (high income), however of the female individuals only 11\% have a high income
- If an algorithm is trained on this data, the skewness ratio is <span class="highlight">amplified</span> from 3:1 to 5:1
- Simply removing sensitive attribute <emph>gender</emph> from the training data is not sufficient
<span class="citeme">Pedreshi, Ruggieri, Turini: Discrimination-aware data mining, KDD, 2008.</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Calders \& Verwer: Three naïve Bayes approaches for discrimination-free classification, Data Mining and Knowledge Discovery 2010.</span>
<span class="citeme">Kamishima et al.: Fairness-Aware classifier with prejudice remover regularizer, ECML 2012 </span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Enforcing fairness
- No matter in what way the data is biased: we want to enforce fairness
- Idea: just tell the algorithm that it should treat all groups in the same way
- Question: <span class="highlight">how do we define fairness?</span>
- Really hard question
- IN SHORT, it is an application-specific
</textarea></section>
<section>
<p> </p>
<p> </p>
<h1>Algorithmic fairness definitions</h1>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Fairness definitions
- Discrimination in the law:
- <span class="highlight">Direct discrimination</span> with respect to intent
- <span class="highlight">Indirect discrimination</span> with respect to consequences
<span class="citeme">Article 21, EU Charter of Fundamental Rights</span>
- From the legal context to algorithmic fairness, e.g.:
- Removing direct discrimination by <span class="highlight">not using group information at test time</span>
- Removing indirect discrimination by enforcing <span class="highlight">equality on the outcomes</span> between groups
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Running example
- Task: predict whether someone should be admitted ($Y=1$) or not ($Y=-1$) to the MSc programme at Sussex!
- Two protected groups: <span style="color: blue">blue</span> group and <span style="color: green">green</span> group
- blue: $S=0$, and green: $S=1$
- In the training set: <span style="color: blue">20% of blue</span> applicants were admitted, <span style="color: green">50% of green</span> applicants were admitted
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Equality on the outcomes
<span class="highlight">Positive prediction outcomes</span>:
$
\text{Pr}(\hat{Y}=1 | S=0) = \text{Pr}(\hat{Y}=1 | S=1)
$
- $\hat{Y} \in \\{1,-1\\}$ and it is our predicted label
- When evaluating the algorithm on the test set, both groups ($S=0$ and $S=1$)
should have the same number of positive predictions ($\hat{Y}=1$)
- In the MSc programme admission example: same percentage from both blue and green groups will be admitted (e.g. 30%)
- This criterion is called <span class="highlight">statistical or demographic parity</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Side effects of statistical parity
- Enforcing statistical parity necessarily produces <span class="highlight">lower accuracy</span>
- Consider this: we want to enforce 30% acceptance for the <span style="color: blue">blue</span> group but the training data only has 20% accepted
- some individuals of the <span style="color: blue">blue</span> group have to be "misclassified" ($\hat{Y} = 1$ instead of $\hat{Y} = -1$)
to make the numbers work
- Not surprising because we are computing accuracy against the biased data
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Fairness – Accuracy trade-off
- Usually we don't want to have too bad accuracy with respect to the (bias) data
- Goal: find algorithm that produces fair result at highest possible accuracy
- Otherwise it's easy: a *random classifier* is very fair (but useless)
- Random classifier: just predict $\hat{Y} = 1$ 50% of the time regardless of input
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Back to the running example
Consider again the MSc programme admission example:
- Two features: test/essay score and group (blue and green) of individuals
- Task: predict if they should be admitted ($Y=1$) or not ($Y=-1$)
- Composition of the training dataset: 50% <span style="color: blue">blue</span>, 50% <span style="color: green">green</span>. 20% of <span style="color: blue">blue</span> have $Y=1$, 50% of <span style="color: green">green</span> have $Y=1$
- The dataset is heavily skewed but let's ignore that for now and just try to make accurate predictions for this dataset
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Accurate predictions
(Reminder: 20% of <span style="color: blue">blue</span> have $Y=1$, 50% of <span style="color: green">green</span> have $Y=1$)
- A simple way to make relatively accurate predictions:
- for <span style="color: green">green</span> individuals base the prediction on test/essay score
- for <span style="color: blue">blue</span> individuals ignore test/essay score and always predict $Y=-1$
- Result: up to 90% accuracy (80% in <span style="color: blue">blue</span> group and 100% in <span style="color: green">green</span> group)
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Accurate but not fair
- The dataset was already skewed but the algorithm's prediction are even more "unfair"
- This is because it's easier to just base the decision on group information than to figure out the effect of the test score
- This is a toy case but it can happen with real data too
- <span class="highlight">What we do not want</span>: the algorithm "being lazy" in a subgroup
- <span class="highlight">What we want</span>: the algorithm should make equally good predictions for all subgroups
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Equality on the outcomes
<span class="highlight">TPR outcomes</span>:
$
\text{Pr}(\hat{Y}=1 | S=0, Y=1) = \text{Pr}(\hat{Y}=1 | S=1, Y=1)
$
- with $Y, \hat{Y} \in \\{1,-1\\}$ and $S \in \\{0,1\\}$
- The probability of predicting positive class, given that the true label is positive, should be the same for all groups
- In the summer school admission example: both blue and green groups will have the same true positive rate TPR (e.g. 60%)
- This criterion is called <span class="highlight">equality of opportunity</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Equalised odds
<span class="highlight">TPR and TNR outcomes</span>:
$
\text{Pr}(\hat{Y}=y | S=0, Y=y) = \text{Pr}(\hat{Y}=y | S=1, Y=y)
$
for all possible values for $y$
- Stricter version of equality of opportunity
- TPR and TNR (true negative rate) must be the same in all groups
- TPR = TP / (TP + FN), TNR = TN / (TN + FP)
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Accuracy–Fairness trade-off
- EOpp (Equality of Opportunity) and EOdd (Equalised Odds) both assume that the training data is correct
- A perfect classifier (that always predicts the correct label) fulfills EOpp and EOdd
- However: a random classifier does as well
- a random classifier achieves 50% TPR (and TNR) in all groups
- Achieving EOpp or EOdd at low accuracy is easy
</textarea></section>
<section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Mini-summary
Several notions of statistical fairness:
- Statistical parity
- Equalised odds
- Equality of opportunity
- Predictive parity
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Statistical fairness notions
- Statistical parity (
$
\hat{Y} \perp S
$):
$
\text{Pr}(\hat{Y}=1 | S=0) = \text{Pr}(\hat{Y}=1 | S=1)
$
- Equalised odds ($
\hat{Y} \perp S | Y
$):
$
\text{Pr}(\hat{Y}=y | S=0, Y=y) = \text{Pr}(\hat{Y}=y | S=1, Y=y)
$
- Predictive parity ($
Y \perp S | \hat{Y}
$):
$
\text{Pr}(Y=y | S=0, \hat{Y}=y) = \text{Pr}(Y=y | S=1, \hat{Y}=y)
$
- <span class="highlight">Let's have all the fairness metrics</span>! If we are fair with regards to all notions of fair, then we're fair... right?
</textarea></section>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Mutual exclusivity by Bayes' rule
<small>
$$
\underbrace{\text{Pr}(Y=1|\hat{Y}=1)}_{\text{Positive Predicted Value (PPV)}} = \frac{\text{Pr}(\hat{Y}=1|Y=1)\overbrace{\text{Pr}(Y=1)}^{\text{Base Rate (BR)}}}{\underbrace{\text{Pr}(\hat{Y}=1|Y=1)}_{\text{True Positive Rate (TPR)}}\text{Pr}(Y=1) + \underbrace{\text{Pr}(\hat{Y}=1|Y=-1)}_{\text{False Positive Rate (FPR)}}(1-\text{Pr}(Y=1))}
$$
</small>
- Suppose we have FPR<sub>S=1</sub> = FPR<sub>S=0</sub> and TPR<sub>S=1</sub> = TPR<sub>S=0</sub> (<span class="highlight">equalised odds</span>), can we have PPV<sub>S=1</sub> = PPV<sub>S=0</sub> (<span class="highlight">predictive parity</span>)?
- YES! But only if we have a <span class="highlight">perfect dataset</span> (i.e. BR<sub>S=1</sub> = BR<sub>S=0</sub>) or a <span class="highlight">perfect predictor</span> (i.e. FPR=0 and TPR=1 for S=1 and S=0)
<span class="citeme">Kehrenberg, Chen, NQ: Tuning fairness by marginalizing latent target labels, Oct 2018</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Roth, Impossibility results in fairness as Bayesian inference, Feb 2019</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Mutual exclusivity by Bayes' rule
<small>
$$
\overbrace{\text{Pr}(\hat{Y}=1)}^{\text{Acceptance/Positive Rate (AR or PR)}} = \underbrace{\text{Pr}(\hat{Y}=1|Y=1)}_{\text{TPR}}\underbrace{\text{Pr}(Y=1)}_{\text{Base Rate (BR)}} + \underbrace{\text{Pr}(\hat{Y}=1|Y=-1)}_{\text{FPR}}(1-\text{Pr}(Y=1))
$$
</small>
- Suppose we have FPR<sub>S=1</sub> = FPR<sub>S=0</sub> and TPR<sub>S=1</sub> = TPR<sub>S=0</sub> (<span class="highlight">equalised odds</span>), can we have AR<sub>S=1</sub> = AR<sub>S=0</sub> (<span class="highlight">statistical parity</span>)?
- YES! But only if we have a <span class="highlight">perfect dataset</span> (i.e. BR<sub>S=1</sub> = BR<sub>S=0</sub>)
<span class="citeme">Kehrenberg, Chen, NQ: Tuning fairness by marginalizing latent target labels, Oct 2018</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Roth, Impossibility results in fairness as Bayesian inference, Feb 2019</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Which fairness notion to use?
- In our admission example, we have 20K applicants (50% <span style="color: blue">blue</span>, 50% <span style="color: green">green</span>), and we can only accept 50% of all applicants
- Our entrance test is highly predictive of success
- 80% of those who pass the test will successfully graduate
- And, only 10% of those who don't pass the test will graduate
- We have a lot of applications from people who don't pass the test
- 60% of <span style="color: blue">blue</span> applicants pass the test
- 40% of <span style="color: green">green</span> applicants pass the test
</textarea></section>
<section>
<h3>Confusion Tables</h3>
<h4><span style="color: blue">blue</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Don't Graduate</td>
<td></td>
<td></td>
</tr></tbody>
</table>
<br>
<h4><span style="color: green">green</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Don't Graduate</td>
<td></td>
<td></td>
</tr></tbody>
</table>
</section>
<section data-background-color="#000">
<p>Solving this problem with <span class="highlight">statistical parity</span> fairness metric?</p>
<section>???</section>
<section>Select 50% of applicants of both <span style="color: blue">blue</span> and <span style="color: green">green</span> applicants</section>
<section>
<small>
<h4><span style="color: blue">blue</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>4000 (80%)</td>
<td>1200</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>1000 (20%)</td>
<td>3800</td>
</tr>
<tr><td></td><td>5000</td><td></td></tr></tbody>
</table>
<br>
<h4><span style="color: green">green</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>3300</td>
<td>500 (10%)</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>1700</td>
<td>4500 (90%)</td>
</tr>
<tr><td></td><td>5000</td><td></td></tr></tbody>
</table>
</small>
<p>
10% of qualified <span style="color: blue">blue</span> applicants are being rejected whilst an additional 10% of unqualified <span style="color: green">green</span> are being accepted</p>
</section>
</section>
<section data-background-color="#000">
<p>Solving this problem with <span class="highlight">equality of opportunity</span> fairness metric?</p>
<section>???</section>
<section>Select 55.5% of <span style="color: blue">blue</span> applicants and 44.5% of <span style="color: green">green</span> applicants, giving a TPR of 85.4% for both groups.</section>
<section>
<small>
<h4><span style="color: blue">blue</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>4440</td>
<td>760</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>1110</td>
<td>3690</td>
</tr>
<tr><td></td><td>5550</td><td></td></tr></tbody>
</table>
<br>
<h4><span style="color: green">green</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>3245</td>
<td>555</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>1205</td>
<td>4995</td>
</tr>
<tr><td></td><td>4450</td><td></td></tr></tbody>
</table>
</small>
<p>4.5% of qualified <span style="color: blue">blue</span> applicants are being rejected whilst an additional 4.5% of unqualified <span style="color: green">green</span> are being accepted</p>
</section>
</section>
<section data-background-color="#000">
<p>Solving this problem with <span class="highlight">predictive parity</span> fairness metric?</p>
<section>???</section>
<section>Select only the applicants who pass the test</section>
<section>
<small>
<h4><span style="color: blue">blue</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>4800</td>
<td>400</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>1200</td>
<td>3600</td>
</tr>
<tr><td></td><td>6000</td><td></td></tr></tbody>
</table>
<br>
<h4><span style="color: green">green</span> Applicants</h4>
<table>
<thead><tr>
<th></th>
<th>Accepted</th>
<th>Not</th>
</tr></thead>
<tbody><tr>
<td>Actually Graduate</td>
<td>3200</td>
<td>600</td>
</tr>
<tr>
<td>Don't Graduate</td>
<td>800</td>
<td>5400</td>
</tr>
<tr><td></td><td>4000</td><td></td></tr></tbody>
</table>
</small>
<p>Could lead to systemic reinforcement of bias</p>
</section>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Which fairness notion to use?
- There's no right answer, all the above are "fair"
- It's important to consult <span class="highlight">domain experts</span> to find which is the best fit for each problem
- There is no one-size fits all
- There is also a notion of <span class="highlight">individual fairness</span> (similar individuals to be treated similarly), and
metrics <span class="highlight">in-between</span> statistical and individual fairness
</textarea></section>
<section>
<p> </p>
<p> </p>
<h1>Algorithmic fairness methods</h1>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## How to enforce fairness?
<span class="highlight">Three</span> different ways to enforce fairness:
<img src="images/fairmethods.png" width=75% title="Fair Methods"/>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Pre-processing
- Simplest pre-processing approach is to <span class="highlight">reweight</span> training data points, those with higher weight are used more often and vice versa with lower weight.
- For example, the weight for a data point with $S=0$ and $Y=1$ is:
$$
W(S=0,Y=1) = \frac{\text{Pr}(Y=1)\text{Pr}(S=0)}{\text{Pr}(Y=1,S=0)} = \frac{\\#(S=0)\\#(Y=1)}{\\#(S=0 \land Y=1)}
$$
- To make the reweighted dataset to be a <span class="highlight">perfect dataset</span> ($Y \perp S$)
<span class="citeme">Kamiran and Calders, Data preprocessing techniques for classification without discrimination, KAIS 2012</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Pre-processing
- From reweighing to <span class="highlight">resampling</span>
- Sampling data points with replacement according to weights
<img src="images/resampling.png" width=38% title="Resampling"/>
<img src="images/illustration.png" width=32% title="Illustration"/>
<span class="citeme">Kamiran
and Calders, Data preprocessing techniques for classification without discrimination, KAIS 2012</span>
<span class="citeme">Sharmanska, Hendricks, Darrell, NQ, Contrastive examples for addressing the tyranny of the majority, May 2019</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Pre-processing
- Another popular approach is to produce a "fair" representation
- Consider that we have 2 roles, a <span class="highlight">data vendor</span>, who is charge of collecting the data and preparing it
- Our other role is a <span class="highlight">data user</span>, someone who will be making predictions based on our data
- The data vendor is concerned that the data user may be using their data to make unfair decisions. So the <span class="highlight">data vendor decides to learn a new, fair representation</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Pre-processing
- Many <span class="highlight">adversarial-based</span> fair representation learning approaches, e.g. using a "Gradient-Reversal Layer"
<img src="images/models/adversarialfair.png" width=38% title="Adversarially Fair"/>
<span class="citeme">Ganin et al.,Domain-adversarial training of neural networks, JMLR 2016</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Edwards and Storkey, Censoring representations with an adversary, ICLR 2016</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Beutel et al., Data decisions and theoretical implications when adversarially learning fair representations, Jul 2017</span>
<span class="citeme">Madras et al., Learning adversarially fair and transferable representations, ICML 2018</span>
</textarea></section>
<section data-background-color="#000">
<section>
<h2>Problems with doing this?</h2>
<h4>Any Ideas?</h4>
</section>
<section data-markdown><textarea data-template>
## Problems with doing this?
Does this representation really hide S?
- A work by Elazar and Goldberg show that adversarially trained latent embeddings still retain sensitive attribute information when a <span class="highlight">post-hoc classifier</span> is trained on them
<span class="citeme">Elazar and Goldberg, Adversarial removal of demographic attributes from text data, EMNLP 2018</span>
</textarea></section>
<section data-markdown><textarea data-template>
## Problems with doing this?
What if the vendor data user decides to be fair as well?
- Referred to as "<span class="highlight">fair pipelines</span>". Work has only just begun exploring these. Current research shows that these don't work (at the moment!)
<span class="citeme">Bower et al., Fair pipelines, Jul 2017</span>
<div style="height:20px;font-size:1px;"> </div>
<span class="citeme">Dwork and Ilvento, Fairness under composition, Jun 2018</span>
</textarea></section>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## In-processing
- Specify fairness metrics as constraints on learning
- Optimise for accuracy under those constraints
- No free lunch: additional constraints lower accuracy
</textarea></section>
<section>
<section data-markdown data-background-color="#000"><textarea data-template>
## In-processing: Constraint
- Given we have a loss function, $\mathcal{L}(\theta)$
- In an unconstrained classifier, we would expect to see $\underset{\theta}{\min}{\mathcal{L}(\theta)}$
- To enforce <span class="highlight">statistical parity</span> metric, a constraint is added $$
\begin{aligned}
&\underset{\theta}{\min} \mathcal{L}(\theta) \\
&\text{subject to } P(\hat{Y} =1 |S = 0) = P(\hat{Y} =1 |S = 1) \\
\end{aligned}$$
- Problem: The formulation is <span class="highlight">not convex</span>!
<span class="citeme">Zafar et al., Fairness constraints: Mechanisms for fair classification, AISTATS 2017</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## In-processing: Constraint
- Need to find a better way to specify the constraints
- Instead of $P(\hat{Y} =1 |S = 0) = P(\hat{Y} =1 |S = 1)$
- Limit the differences in the average strength of acceptance
and rejection across members of different sensitive groups: $Cov(S-\hat{S}, f) = \mathbf{E}[(S-\hat{S})f(X)] - \mathbf{E}[(S-\hat{S})]\mathbf{E}[f(X)]$ $\approx \frac{1}{N} \sum\limits_{i=1}^{N} (S_i - \hat{S}) \cdot f_i$
- $Cov(S-\hat{S}, f)=0$ → equal positive rates across groups → <span class="highlight">statistical parity</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## In-processing: Constraint
- Or, return <span class="highlight">a set of models</span> that trade-off fairness and accuracy
- Could promote a <span class="highlight">transparent</span> selection of operating points
<img src="images/pareto.png" width=33% title="Pareto Efficient Solutions"/>
<span class="citeme">NQ and Sharmanska, Recycling privileged learning and distribution matching for fairness, NIPS 2017</span>
</textarea></section>
</section>
<section data-background-color="#000">
<section data-markdown><textarea data-template>
## In-processing: Likelihood
- **Example**: Logistic Regression
* model: $f(X) = \theta_1 X_1 + \ldots + \theta_D X_D$,
* probability score: $ \text{Pr}(Y=1|X, \theta)=\sigma(f(X))$
* with dependency on sensitive grouping $S$: $ P(Y=1|X, \theta, S)$
</textarea></section>
<section data-markdown><textarea data-template>
## In-processing: Likelihood
- Manipulate the likelihood $\text{Pr}(Y=1|X, \theta, S)$ to enforce fairness
- Introduce (latent) <span class="highlight">target labels</span> $\bar{Y}$: $\text{Pr}(Y=1|X, \theta, S) = \sum\limits_{\bar{Y}} \text{Pr}(Y=1|\bar{Y},S)\text{Pr}(\bar{Y}|X, \theta, S)$ (Intuition: change the learning goal)
- Choose four free parameters such that $\theta$ is targeting a fair output (consult the slides on <span class="highlight">mutual exclusivity</span> of fairness metrics)!
$
\text{Pr}(Y=0|\bar{Y}=0,S=0) \;\ \text{Pr}(Y=0|\bar{Y}=0,S=1)
$
$
\text{Pr}(Y=1|\bar{Y}=1,S=0) \;\ \text{Pr}(Y=1|\bar{Y}=1,S=1)
$
<span class="citeme">Kehrenberg, Chen, NQ, Tuning Fairness by Marginalizing Latent Target Labels, Oct 2018</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## In-processing: Likelihood
<!-- - conceptually works best with *Demographic Parity*
* but was also extended to Equal Odds/Opportunity -->
- Controlling fairness with the <span class="highlight">quantities that matter</span> (e.g. TPR, and PR/AR)
 <!-- .element: style="height: 7em; border: none;"-->
 <!-- .element: style="height: 7em; border: none;"-->
</textarea></section>
<!-- <section data-markdown><textarea data-template>
## In-process: Likelihood
 <!-- .element: style="width: 70%; border: none;" -->
<!-- </textarea></section> -->
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Post-processing
- Train two separate models: one for all datapoints with $S=0$ and another one for $S=1$
- The thresholds of the model are then tweaked until they produce the same <i>positive rate</i>, i.e. $P(\hat{Y}=1|S=0)=P(\hat{Y}=1|S=1)$
- Disadvantage: $S$ has to be known for making predictions in order to choose the correct model
<span class="citeme">Calders \& Verwer: Three naïve Bayes approaches for discrimination-free classification, Data Mining and Knowledge Discovery 2010.</span>
</textarea></section>
<section>
<p> </p>
<p> </p>
<h1>Interpretability in fairness</h1>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Learning fair representations
- Inspired by the work on <span class="highlight">attribution map</span> in computer vision for transparency of deep learning models:
<center>
<img src="images/gradcam.jpg" width=50% title="GradCam" style="margin:0px -10px"/> <p style="font-size:10px">Picture credit: Selvaraju, et al., 2017.</p>
</center>
- Ensure that the learned representation can be "<span class="highlight">overlayed</span>" back to the input data
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Fair representations in the data domain
- A fair representation $Z:=T_\omega(x)$ which has the protected attribute removed, but remains in the space of $x$
- Main disentangling assumption: $\phi(x) = \phi(T_{\omega}(x)) + \underbrace{\phi (\lnot T_{\omega}(x))}_{\text{Spurious}}$
- <span class="highlight">Spurious</span>: dependent on the protected attribute
- <span class="highlight">Non-Spurious</span>: independent of the protected attribute
<span class="citeme">NQ, Sharmanska, Thomas, Discovering fair representations in the data domain, CVPR 2019</span>
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Architecture and optimization
<img src="images/models/Architecture.png" width=70% title="Architecture"/>
<p align="left">$\underset{T_\omega}{\text{min.}}\quad \underbrace{\mathrm{Dep.}(\phi ( T_{\omega}(\mathrm{x})),S|y=1)}_{\text{non-spurious loss}} \quad\underbrace{- \mathrm{Dep.}(\phi (\lnot T_{\omega}(\mathbf{x})),S|y=1)}_{\text{spurious loss}} $
$+ \text{prediction loss} \quad+ \text{reconstruction loss}$</p>
</textarea></section>
<section data-background-color="#000">
<h2>Transparency in fairness</h2>
<p align="left">Experiments on <span class="highlight"> CelebA dataset</span>: $202,599$ celebrity face images with $40$ attributes, <span class="highlight">gender</span> as the binary protected attribute, the attribute <span class="highlight">smiling</span> as the classification task</p>
<small>
<table>
<thead><tr>
<td></td>
<td>Acc.</td>
<td><span class="highlight">TPR Diff.</span></td>
<td>TPR male</td>
<td>TPR female</td>
</tr>
</thead>
<tbody>
<tr>
<td>original <span class="highlight">image</span> representation $\mathbf{x}\in\mathbb{R}^{2048}$</td>
<td width="10%">89.70</td>
<td width="10%">7.54</td>
<td width="15%">92.03</td>
<td width="15%">84.50</td>
</tr>
<!-- <tr>
<td>original <span class="highlight">attribute</span> representation $\mathbf{x}\in\mathbb{R}^{40}$</td>
<td width="10%">79.1</td>
<td width="10%">39.9</td>
<td width="10%">90.8</td>
<td width="10%">50.9</td>
</tr>-->
<tr>
<td>fair <span class="highlight">image</span> representation in the data domain $T_{\omega}(\mathbf{x})\in\mathbb{R}^{2048}$</td>
<td width="10%">91.31</td>
<td width="10%">4.76</td>
<td width="15%">91.85</td>
<td width="15%">87.09</td>
</tr>
<!-- <tr>
<td>fair <span class="highlight">attribute</span> representation in the data domain $T_{\omega}(\mathbf{x})\in\mathbb{R}^{40}$</td>
<td width="10%">75.9</td>
<td width="10%">12.4</td>
<td width="10%">87.2</td>
<td width="10%">74.8</td>
</tr>-->
</tbody>
</table>
</small>
<table align="center" style="border-collapse: collapse; border: none;">
<tbody ><tr style="border: none;"><td width="40%" colspan="4" style="border: none;"></td><td width="1%" style="border: none;"></td><td width="40%" style="border: none;"></td></tr>
<tr style="border: none;">
<td width="10%" style="border: none;">Non-spurious</td>
<td width="20%" style="border: none;"><img src="images/celeba_res/006126.jpg" width=100% title="Im1"/></td>
<td width="10%" style="border: none;">Spurious</td>
<td width="20%" style="border: none;"><img src="images/celeba_res/006126res.jpg" width=100% title="RIm1"/></td>
</tr></tbody>
</table>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Challenges?
- Spurious/non-spurious (w.r.t. gender) visualisations are too coarse!
<table align="center" style="border-collapse: collapse; border: none;">
<tbody ><tr style="border: none;"><td width="40%" colspan="4" style="border: none;"></td><td width="1%" style="border: none;"></td><td width="40%" style="border: none;"></td></tr>
<tr style="border: none;">
<td width="30%" style="border: none;">Spurious (what we have)</td>
<td width="20%" style="border: none;"><img src="images/celeba_res/006126res.jpg" width=100% style="margin:-20px 0px"/></td>
<td width="30%" style="border: none;">Spurious (what we want)</td>
<td width="20%" style="border: none;"><img src="images/spurious.png" width=60% style="margin:-20px 0px"/></td>
</tr></tbody>
</table>
- Residual unfairness (transferability) problem
<center>
<img src="images/nyclu.png" width="35%" title="New York Civil Liberties Union" style="margin:-20px 0px"/><p style="font-size:10px">Picture credit: New York Civil Liberties Union</p>
</center>
</textarea></section>
<section data-background-color="#000">
<h2>Fair and transferable </h2>
<table>
<tbody>
<tr>
<td style="border: none;" width="50%"> <img src="images/diagram.png" width=100% style="margin:-20px 0px"/></td>
<td style="border: none;"> <p>Disentangling the latent space (c.f. on the reconstruction space) into two components:<ul>
<li><span class="highlight">Spurious</span>: dependent on the protected attribute</li>
<li><span class="highlight">Non-Spurious</span>: independent of the protected attribute</li>
</ul></p></td>
</tr>
</tbody>
</table>
<span class="citeme">Kehrenberg, Bartlett, Thomas, NQ, NoSiNN: Removing spurious correlations with null-sampling, under review</span>
</section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Invertible Neural Network
- We leverage flow-based models to <span class="highlight">preserve all information relevant to $y$</span> that is independent of $s$ (during pre-training phase)
- Flow-based models permit exact likelihood estimation through warping a base density with a series of <span class="highlight">invertible transformation</span>
<!--- We have:
$\underset{\theta}{\text{min.}}\quad \mathbb{E}_x [- \log p_\theta (x|z)] + \lambda \text{Dep.}(z^{\text{invariant}}, s)$
with $\log p(x) = \log \mathcal{N}(z_0; 0, \mathbb{I}) + \sum_\{i=1\}^\{L\} \log | \det (\frac{\mathrm{d} f_i}{\mathrm{d}z_\{i-1\}})|$-->
- We conjecture that the latent representations of flow-based models are more robust to <span class="highlight">out-of-distribution</span> data
<!--
- The invertible network $f_\theta$ maps the inputs $x$ to a representation $z$: $f_\theta(x) = [z^{\text{spurious}}, z^{\text{invariant}}] := z$
- We have:
$\underset{\theta}{\text{min.}}\quad \mathbb{E}_x [- \log p_\theta (x|z)] + \lambda \text{Dep.}(z^{\text{invariant}}, s)$ with $\log p(x) = \log \mathcal{N}(z_0; 0, \mathbb{I}) + \sum_\{i=1\}^\{L\} \log | \det (\frac{\mathrm{d} f_i}{\mathrm{d}z_\{i-1\}})|$
\mathbb{E}_x [- \log p_\theta (x|z)] + \lambda \text{Dep.}(z_d, s)$, with $\log p(x) = \log \mathcal{N}(z; 0, \mathbb{I}) + \sum_{i=1}^{L} \log | \det ( \frac{\mathop{}\!\mathrm{d} f_i}{ \mathrm{d}z_{i-1}}) |-->
</textarea></section>
<section data-markdown data-background-color="#000"><textarea data-template>
## Transferability results
- Experiments on <span class="highlight">Adult Income dataset</span>
- We evaluate the performance of our method for mixing factors ($\eta$) of value $\\{0, 0.1, ..., 1\\}$ (at $\eta = 0.5$ the dataset is perfectly balanced)
<center>
<img src="images/transfer_2.png" width=40% title="Transferability" style="margin:0px 0px"/>
</center>
</textarea></section>
<section data-background-color="#000">
<h2>Transparency in fairness</h2>
<table>
<tbody>