Can (bert) classifier models be used?

[SerialKicked]

Hey, I've been using LLamaSharp as a secondary system from my middleware library (doing embeddings, and stuff like that). And recently, I've been trying to use it for BERT classifier models (like this quantized version) to no avail.

I think upstream know how to deal with classifiers nowadays, but not 100% sure. Anyway, I tried a bunch of things, like:

• Using Embeddings (with different pooling types) but nope, I do get a float[768] (embeddings) instead of the float[28] the model should return. That was optimistic i know.

• I tried something like that

    // ModelContext, ModelSettings, ModelWeights... are all variables with a properly loading model.
    
    var tokens = ModelContext!.NativeHandle.Tokenize("My text to analyze", add_bos: true, special: false, Encoding.UTF8);
    var batch = new LLamaBatch();
    int lastRow = batch.AddRange(tokens, start: 0, sequence: LLamaSeqId.Zero, logitsLast: true);
    ModelContext.Decode(batch);
    var logits = ModelContext.NativeHandle.GetLogitsIth(lastRow).ToArray();

But nope, it's 30k empty floats.

I'm probably doing a lot wrong, but also maybe it doesn't handle bert-type classifiers? I'm not sure, but any help or guidance would sure be appreciated, because I really, really, don't want to mix and match with ML.NET or worse, use python, just for that.

Cheers.