TPU VM trained weights
#1186
-
|
Hallo, |
Beta Was this translation helpful? Give feedback.
Answered by
rwightman
Mar 22, 2022
Replies: 1 comment
-
|
@romanoss they work the same on TPU or GPU as far as inference or fine-tuning is concerned, they are standard float32 weights, so no float16/bfloat16 issues to worry about. I did all training on TPU VM v3 instances, but all of the posted validation numbers were run/verified on a RTX 3090. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
romanoss
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@romanoss they work the same on TPU or GPU as far as inference or fine-tuning is concerned, they are standard float32 weights, so no float16/bfloat16 issues to worry about. I did all training on TPU VM v3 instances, but all of the posted validation numbers were run/verified on a RTX 3090.