An excessive amount of duplication
Some degree of competitors and parallel growth is wholesome for innovation, however the present state of affairs seems more and more wasteful. A number of organizations are constructing related capabilities, with every contributing a large carbon footprint. This redundancy turns into significantly questionable when many fashions carry out equally on commonplace benchmarks and real-world duties.
The variations in capabilities between LLMs are sometimes delicate; most excel at related duties corresponding to language technology, summarization, and coding. Though some fashions, like GPT-4 or Claude, might barely outperform others in benchmarks, the hole is usually incremental quite than revolutionary.
Most LLMs are educated on overlapping knowledge units, together with publicly out there web content material (Wikipedia, Widespread Crawl, books, boards, information, and so forth.). This shared basis results in similarities in information and capabilities as fashions take in the identical factual knowledge, linguistic patterns, and biases. Variations come up from fine-tuning proprietary knowledge units or slight architectural changes, however the core basic information stays extremely redundant throughout fashions.