Skip to content

Commit faac0ba

Browse files
authored
common : ensure llama_batch size does not exceed max size (ggml-org#9668)
A crash was observed when the number of tokens added to a batch exceeds llama_batch size. An assertion in llama_batch_add was added to protect against llama_batch size overflow.
1 parent f99d3f8 commit faac0ba

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

common/common.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1437,6 +1437,8 @@ void llama_batch_add(
14371437
llama_pos pos,
14381438
const std::vector<llama_seq_id> & seq_ids,
14391439
bool logits) {
1440+
GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");
1441+
14401442
batch.token [batch.n_tokens] = id;
14411443
batch.pos [batch.n_tokens] = pos;
14421444
batch.n_seq_id[batch.n_tokens] = seq_ids.size();

0 commit comments

Comments
 (0)