472
Discussion[D] GPT-3, The $4,600,000 Language Model(self.MachineLearning)
submitted 5 years, 7 months ago* (edited 18 hours, 40 minutes after) by mippie_moe to /r/MachineLearning (3m)
since 5 years, 7 months ago
1 of 1
Tip Reveddit Real-Time can notify you when your content is removed.
your account history
Tip Check if your account has any removed comments.
view my removed comments you are viewing a single comment's thread.
view all comments
Assuming the Ti 1080 has 10.5 TFLOP/s_series) and the calculation took 3.14e23 FLOP then you can train GPT-3 in a meager 8228511 and a half GPU hours. Genesis Cloud would only charge you 30ct per GPU hour so this would only cost you only 2.4 mio. $ after all. Not exactly a bargain but you do get the first 50$ off.
Disclosure: I am a working student there...