-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathindex.html
284 lines (257 loc) · 15.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description"
content="Deformable Neural Radiance Fields creates free-viewpoint portraits (nerfies) from casually captured videos.">
<meta name="keywords" content="Nerfies, D-NeRF, NeRF">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>LREC-COLING 2024 Tutorial: Hallucination in Large Language Models</title>
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="./static/css/bulma.min.css">
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="./static/css/index.css">
<link rel="icon" href="./static/images/favicon.svg">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./static/js/fontawesome.all.min.js"></script>
<script src="./static/js/bulma-carousel.min.js"></script>
<script src="./static/js/bulma-slider.min.js"></script>
<script src="./static/js/index.js"></script>
</head>
<body>
<nav class="navbar" role="navigation" aria-label="main navigation">
<div class="navbar-brand">
<a role="button" class="navbar-burger" aria-label="menu" aria-expanded="false">
<span aria-hidden="true"></span>
<span aria-hidden="true"></span>
<span aria-hidden="true"></span>
</a>
</div>
</nav>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-2 publication-title">
<span style="font-size: 80%">LREC-COLING 2024 Tutorial:</span><br />
Hallucination in Large Language Models
</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<table>
<tr>
<!-- <th scope="row">TR-7</th> -->
<td width="25%" height="25%" style="text-align: center; padding: 3px"><img width="150" height="150" src="static/imgs/vipula.jpg"></td>
<td width="25%" height="25%" style="text-align: center; padding: 3px"><img width="150" height="150" src="static/imgs/aman.jpg"></td>
<td width="25%" height="25%" style="text-align: center; padding: 3px"><img width="150" height="150" src="static/imgs/amit.jpg"></td>
<td width="25%" height="25%" style="text-align: center; padding: 3px"><img width="150" height="150" src="static/imgs/amitava.jpg"></td>
</tr>
<tr>
<!-- <th scope="row">TR-7</th> -->
<td width="25%" style="text-align: center"><a href="https://vr25.github.io/" style="border-radius: 50%">Vipula Rawte</a><sup>1</sup>,</td>
<td width="25%" style="text-align: center"><a href="https://amanchadha.com/" style="border-radius: 50%">Aman Chadha</a><sup>2</sup>,</td>
<td width="25%" style="text-align: center"><a href="https://amit.aiisc.ai/" style="border-radius: 50%">Amit Sheth</a><sup>1</sup>,</td>
<td width="25%" style="text-align: center"><a href="https://www.amitavadas.com/" style="border-radius: 50%">Amitava Das</a><sup>1</sup></td>
</tr>
</table>
<!-- <a href="https://vr25.github.io/">Vipula Rawte</a><sup>1</sup>,</span>
<span class="author-block">
<a href="https://amanchadha.com/">Aman Chadha</a><sup>2</sup>,</span>
<span class="author-block">
<a href="https://amit.aiisc.ai/">Amit Sheth</a><sup>1</sup>,
</span>
<span class="author-block">
<a href="https://www.amitavadas.com/">Amitava Das</a><sup>1</sup>, -->
</span>
</div>
<div class="is-size-6 publication-authors">
<span class="author-block"><sup>1</sup>AIISC,</span>
<span class="author-block"><sup>2</sup>Amazon</span>
</div>
<br />
<div class="is-size-5 publication-authors">
<b>Saturday May 25, 2024 Morning (Virtual)</b>
</div>
<div class="is-size-5 publication-authors">
<!--
Zoom link available on <a href="https://underline.io/events/395/sessions?eventSessionId=15330&searchGroup=lecture" target="_blank">Underline</a>
Visit <a target="_blank" href="https://us06web.zoom.us/rec/play/6fqU9YDLoFtWqpk8w8I7oFrszHKW6JkbPVGgHsdPBxa69ecgCxbmfP33asLU3DJ74q5BXqDGR2ycOTFk.93teqylfi_uiViNK?canPlayFromShare=true&from=share_recording_detail&continueMode=true&componentName=rec-play&originRequestUrl=https%3A%2F%2Fus06web.zoom.us%2Frec%2Fshare%2FNrYheXPtE5zOlbogmdBg653RIu7RBO1uAsYH2CZt_hacD1jOHksRahGlERHc_Ybs.KGX1cRVtJBQtJf0o">this link</a>
for the Zoom recording of the tutorial
</div>
<div class="is-size-6 publication-authors">
For those who have not registered to ACL: we will release video recordings after the tutorial
</div>
<br />
<div class="is-size-5 publication-authors">
QnA: <a href="https://tinyurl.com/retrieval-lm-tutorial" target="_blank"><b>tinyurl.com/retrieval-lm-tutorial</b></a>
</div>-->
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Abstract. -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">About this tutorial</h2>
<div class="content has-text-justified">
<!--<p>
In the fast-paced domain of Large Language Models (LLMs), the issue of hallucination is a prominent challenge. Despite continuous endeavors to address this concern, it remains a highly active area of research within the LLM landscape. Grasping the intricacies of this problem can be daunting, especially for those new to the field. This tutorial aims to bridge this knowledge gap by introducing the emerging realm of hallucination in LLMs. It will comprehensively explore the key aspects of hallucination, including benchmarking, detection, and mitigation techniques. Furthermore, we will delve into the specific constraints and shortcomings of current approaches, providing valuable insights to guide future research efforts for participants.
</p>
<p>
In this tutorial, we aim to provide a comprehensive and coherent overview of recent
advances in retrieval-based LMs. We will start
by first providing preliminaries covering the foundations of LM (e.g., masked LMs, autoregressive LMs) and retrieval systems (e.g., nearest-neighbor search methods widely used in neural retrieval systems; Karpukhin et al. 2020). We will then focus
on recent progress in architectures, learning approaches, and applications of retrieval-based LMs.
</p>-->
<p>
In the fast-paced domain of Large Language Models (LLMs), the issue of hallucination is a prominent challenge. Despite continuous endeavors to address this concern, it remains a highly active area of research within the LLM landscape. Grasping the intricacies of this problem can be daunting, especially for those new to the field. This tutorial aims to bridge this knowledge gap by introducing the emerging realm of hallucination in LLMs. It will comprehensively explore the key aspects of hallucination, including benchmarking, detection, and mitigation techniques. Furthermore, we will delve into the specific constraints and shortcomings of current approaches, providing valuable insights to guide future research efforts for participants.
</p>
</div>
</div>
</div>
<!--/ Abstract. -->
<!-- Paper video. -->
<!--/ Paper video. -->
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Schedule</h2>
<p>
Our tutorial will be held on May 25. Slides are available <a href="./slides/25 Hallucinations in Large Language Models.pdf" target='_blank'>here</a>. <!-- (all the times are based on EDT = Toronto local time).
<em>Slides may be subject to updates.</em> -->
</p>
<div class="content has-text-justified">
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-0pky{border-color:inherit;text-align:left;vertical-align:top}
.tg .tg-0lax{text-align:left;vertical-align:top}
</style>
<table class="tg">
<thead>
<tr>
<th class="tg-0pky">Time</th>
<th class="tg-0lax">Section</th>
<th class="tg-0lax">Presenter</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tg-0lax">09:00—09:45</td>
<td class="tg-0lax">Section 1: Introduction </td> <!-- <a href="./slides/1-intro.pdf" target='_blank'>[Slides]</a> -->
<td class="tg-0lax">Vipula</td>
</tr>
<tr>
<td class="tg-0lax">09:45—10:30</td>
<td class="tg-0lax">Section 2: Hallucination Detection </td> <!-- <a href="./slides/1-intro.pdf" target='_blank'>[Slides]</a> -->
<td class="tg-0lax">Aman</td>
</tr>
<tr>
<td class="tg-0lax">10:30—11:00</td>
<td class="tg-0lax">Coffee break</td>
<td class="tg-0lax"></td>
</tr>
<tr>
<td class="tg-0lax">11:00—11:45</td>
<td class="tg-0lax">Section 3: Hallucination Mitigation </td> <!-- <a href="./slides/1-intro.pdf" target='_blank'>[Slides]</a> -->
<td class="tg-0lax">Vipula</td>
</tr>
<tr>
<td class="tg-0lax">11:45—12:30</td>
<td class="tg-0lax">Section 4: Open Challenges </td> <!-- <a href="./slides/1-intro.pdf" target='_blank'>[Slides]</a> -->
<td class="tg-0lax">Amitava</td>
</tr>
<tr>
<td class="tg-0lax">12:30—13:00</td>
<td class="tg-0lax">Q & A Session</td>
<td class="tg-0lax"></td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<!-- Concurrent Work. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Reading List</h2>
<br />
<ul>
<li><a href="https://arxiv.org/pdf/2309.05922"><b>A Survey of Hallucination in Large Foundation Models</b></a> (Rawte et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2309.01219"><b>Siren’s Song in the AI Ocean:
A Survey on Hallucination in Large Language Models</b></a> (Shi et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2309.06794"><b>Cognitive Mirage: A Review of Hallucinations in Large Language Models</b></a> (Ye et al., 2023)</li>
<li><a href="https://aclanthology.org/2022.acl-long.236.pdf"><b>Hallucinated but Factual! Inspecting the Factuality of Hallucinations in
Abstractive Summarization</b></a> (Cao et al., 2023)</li>
<li><a href="https://aclanthology.org/2023.emnlp-main.155.pdf"><b>The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations</b></a> (Rawte et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2309.11495"><b>CHAIN-OF-VERIFICATION REDUCES HALLUCINATION
IN LARGE LANGUAGE MODELS</b></a> (Dhuliawala et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2308.11764v2"><b>Halo: Estimation and reduction of hallucinations in open-source weak large language models</b></a> (Elaraby et al., 2023)</li>
<li><a href="https://aclanthology.org/2023.eacl-main.234.pdf"><b>When Do Pre-Training Biases Propagate to Downstream Tasks?
A Case Study in Text Summarization</b></a> (Ladhak et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2305.11747"><b>HaluEval: A Large-Scale Hallucination Evaluation Benchmark
for Large Language Models</b></a> (Li et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2307.03987"><b>A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation</b></a> (Varshney et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2305.13534"><b>How Language Model Hallucinations Can Snowball</b></a> (Zhang et al., 2023)</li>
<li><a href="hhttps://arxiv.org/pdf/2303.08896"><b>SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models</b></a> (Manakaul et al., 2023)</li>
<li><a href="https://arxiv.org/pdf/2305.15852"><b>SELF-CONTRADICTORY HALLUCINATIONS OF LLMS: EVALUATION, DETECTION AND MITIGATION</b></a> (Mündler et al., 2023)</li>
</ul>
<br />
</div>
</div>
</section>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@article{ hallucination-llm-tutorial,
author = { Rawte, Vipula and Chadha, Aman and Sheth, Amit and Das, Amitava },
title = { LREC-COLING 2024 Tutorial: Hallucination in Large Language Models },
journal = { LREC-COLING 2024 },
year = { 2024 },
}</code></pre>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="content has-text-centered">
<a class="icon-link" href="https://github.com/vr25/lrec-coling-hallucination-tutorial" class="external-link" disabled>
<i class="fab fa-github"></i>
</a>
</div>
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This website is licensed under a <a rel="license"
href="http://creativecommons.org/licenses/by-sa/4.0/">Creative
Commons Attribution-ShareAlike 4.0 International License</a>.
</p>
<p>
This means you are free to borrow the <a
href="https://github.com/nerfies/nerfies.github.io">source code</a> of this website,
we just ask that you link back to this page in the footer.
Please remember to remove the analytics code included in the header of the website which
you do not want on your website.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>