The fast tempo of innovation and the proliferation of recent fashions have raised considerations about know-how lock-in. Lock-in happens when companies change into overly reliant on a particular mannequin with bespoke scaffolding that limits their skill to adapt to improvements. Upon its launch, GPT-4 was the identical price as GPT-3 regardless of being a superior mannequin with a lot increased efficiency. Because the GPT-4 launch in March 2023, OpenAI costs have fallen one other six instances for enter information and 4 instances for output information with GPT-4o, launched Might 13, 2024. After all, an evaluation of this kind assumes that era is bought at price or a set revenue, which might be not true, and important capital injections and adverse margins for capturing market share have probably backed a few of this. Nonetheless, we doubt these levers clarify all the development good points and worth reductions. Even Gemini 1.5 Flash, launched Might 24, 2024, affords efficiency close to GPT-4, costing about 85 instances much less for enter information and 57 instances much less for output information than the unique GPT-4. Though eliminating know-how lock-in might not be attainable, companies can scale back their grip on know-how adoption through the use of business fashions within the brief run.
Avoiding lock-in dangers
In some respects, the bitter lesson is a part of this extra appreciable dialogue about lock-in dangers. We count on scaling to proceed, at the very least for one more couple of interactions. Until you might have a selected use case with apparent business potential, or function inside a high-risk and extremely regulated trade, adopting the know-how earlier than the total scaling potential is set and exhausted could also be hasty.
Finally, coaching a language mannequin or adopting an open-source mannequin is like swapping a leash for a ball and chain. Both method, you’re not strolling away with out leaving some pores and skin within the recreation. Chances are you’ll want to coach or tune a mannequin in a slender area with specialised language and tail data. Nonetheless, coaching language fashions entails substantial time, computational sources, and monetary funding. This will increase the danger for any technique. Coaching a language mannequin can price a whole lot of hundreds to hundreds of thousands of {dollars}, relying on the mannequin’s dimension and the quantity of coaching information. The financial burden is exacerbated by the nonlinear scaling legal guidelines of mannequin coaching, during which good points in efficiency could require exponentially larger compute sources—highlighting the uncertainty and danger concerned in such endeavors. Bloomberg’s technique of together with a margin of error of 30 % of their computing funds underscores the unpredictable nature of coaching.