Skip to content

Commit

Permalink
llama : correct llm_build_moe_ffn() arguments in build_arctic()
Browse files Browse the repository at this point in the history
  • Loading branch information
sszymczy committed May 26, 2024
1 parent bb9c361 commit f3b5e7d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -11142,7 +11142,7 @@ struct llm_build_context {
model.layers[il].ffn_down_exps,
n_expert, n_expert_used,
LLM_FFN_SILU, true,
true, hparams.expert_weights_scale,
false, 0.0,
cb, il);
cb(cur, "ffn_moe_out", il);

Expand Down

0 comments on commit f3b5e7d

Please sign in to comment.