-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCITATION.cff
129 lines (128 loc) · 4.35 KB
/
CITATION.cff
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: >-
Towards AI-Assisted Protocol Analysis in Design Research:
Automating Question Labelling with GPT-4 According to
Eris' (2004) Taxonomy
message: >-
If you use this code repository, please cite the
associated paper using the metadata provided in this
repository
type: software
authors:
- given-names: Ahmed Shahriar
family-names: Sakib
email: [email protected]
affiliation: 'University of Waterloo, Canada'
orcid: 'https://orcid.org/0000-0001-8611-9616'
- given-names: Ada
family-names: Hurst
orcid: 'https://orcid.org/0000-0002-2481-8566'
affiliation: 'University of Waterloo, Canada'
email: [email protected]
- given-names: Frank
family-names: Safayeni
email: [email protected]
affiliation: 'University of Waterloo, Canada'
identifiers:
- type: doi
value: 10.1007/978-3-031-71918-9_3
description: >-
Link to the published conference paper at Springer
Nature.
- type: url
value: 'https://rdcu.be/dYUou'
description: >-
Springer Nature SharedIt link for free access to the
full-text PDF of the conference paper.
- type: other
value: 978-3-031-71917-2
description: Print ISBN
- type: other
value: 978-3-031-71918-9
description: Online ISBN
repository-code: >-
https://github.com/ahmedshahriar/AI-Assisted-Protocol-Analysis-in-Design-Research
url: >-
https://link.springer.com/chapter/10.1007/978-3-031-71918-9_3
abstract: >-
This study explores the potential of large language models
(LLM)-based tools, specifically GPT-4 -- a
state-of-the-art language processing model - to assist in
the analysis of verbal protocols of design. We focus on
Eris' taxonomy, a well-established framework that
classifies questions asked by participants in a
design-focused task according to three broad categories:
low-level, deep reasoning, and generative design
questions. Using a large dataset of pre-classified
questions from design review meetings, a series of
experiments test GPT-4's capability in the categorization
task and evaluate how different factors influence its
precision. Results indicate that GPT-4 matches performance
by human coders -- a promising result for design
researchers who can benefit from this tool with little
prior natural language processing expertise. Overall,
findings offer insights into the strengths and limitations
of LLMs in this context and suggest directions for future
research into the use of LLM-based tools in qualitative
analyses of design activity.
keywords:
- design research
- protocol analysis
- Artificial Intelligence
- GPT-4
- Design cognition
- LLM
- Eris' taxonomy
- qualitative research
- Question classification
- NLP
- Automated question labeling
- text analytics
- Machine Learning
- DCC’24
- design computing
license: Apache-2.0
references:
- type: conference-paper
authors:
- given-names: Ahmed Shahriar
family-names: Sakib
email: [email protected]
affiliation: 'University of Waterloo, Canada'
orcid: 'https://orcid.org/0000-0001-8611-9616'
- given-names: Ada
family-names: Hurst
email: [email protected]
affiliation: 'University of Waterloo, Canada'
orcid: 'https://orcid.org/0000-0002-2481-8566'
- given-names: Frank
family-names: Safayeni
email: [email protected]
affiliation: 'University of Waterloo, Canada'
title: >-
Towards AI-Assisted Protocol Analysis in Design Research: Automating
Question Labelling with GPT-4 According to Eris' (2004) Taxonomy
collection-title: Design Computing and Cognition '24
year: 2024
month: 9
editors:
- given-names: John
family-names: Gero
email: [email protected]
affiliation: 'University of North Carolina at Charlotte, USA'
orcid: 'https://orcid.org/0000-0001-9026-535X'
publisher:
name: Springer Nature Switzerland
address: 'Cham, Switzerland'
conference:
name: Design Computing and Cognition'24
location: Concordia University
city: Montreal
country: CA
date-start: '2024-07-07'
date-end: '2024-07-10'
start: 38
end: 55
doi: 10.1007/978-3-031-71918-9_3