TransformerEncoder layer unable to be saved

[mateuszlugowski]

Describe the bug

Trying to save a model containing TransformerEncoder layer results in an error. This is due to some weights of it having the same exact name, and h5 format is raising a ValueError because of that.

To Reproduce

from keras_nlp.layers import TransformerEncoder

inp = keras.layers.Input((30, 128))
tl = TransformerEncoder(
                intermediate_dim=512,
                num_heads=8,
            )
model = keras.models.Model(inp, tl(inp))
model.save_weights('test.h5')

[pranavvp16]

I also encountered the same error while implementing ELECTRA in keras. The previous version of TransformerEncoder was able to save successfully which I found from this notebook written by @abheesht17 , which implements previous version of keras-nlp. I tried to run the same notebook with the latest version of keras-nlp and saving of TransformerEncoder in .h5 failed.

The only change I made in the new notebook was rename _self_attention_layernorm to _self_attention_layer_norm which is the according to the latest naming in TransformerEncoder. notebook2

[sonali-kumari1]

Hi @mateuszlugowski,

Thanks for reporting the issue.

keras-nlp has been renamed to keras-hub and all the new development and future release will be as keras-hub package.

The error you're getting, is because you are trying to save weights using model.save_weights('test.h5') but you should use model.save_weights('test.weights.h5'). For more information you can refer this documentation.

Attaching gist for your reference.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.