Skip to content

Feature request: allow users to freeze model weights when training a classifier #272

@next-game-solutions

Description

@next-game-solutions

At the moment, the freeze=False option in the _initialize method of the StaticModelForClassification class is hardcoded, allowing the embedding model's weights to be updated during training. It would be beneficial for certain use cases to freeze the weights instead and only train the classifier's MLP head.

For example, embeddings of the documents to be classified could be pre-computed using the original embedding model elsewhere in a production system (e.g. for other purposes too, not only for classification). The classification part of the system would then only use the trained MLP head head based on such pre-computed embeddings, making that part extremely light-weight and efficient.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions