One of the most advanced and powerful artificIAl intelligence systems, GPT-4, has recently experienced a significant weakening. This drastic change raises questions about the reasons behind it and its implications for society.
Researchers from Stanford and UC Berkeley have published an interesting article on the subject, offering insights into this transformation and its possible reasons. (see more at: https://arxiv.org/pdf/2307.09009.pdf)
The article shows that the generation of executable code dropped from 52% to 10% for GPT-4 and from 22% to 2% for GPT-3.5 between March and June 2022.
GPT-4 has transitioned from being a "big and single piece" to multiple smaller, specIAlized, and interconnected pieces. And it has been found that these smaller pieces do not work as well together as when it was a single central piece.
In practice, this means that when a user asks a question, the system decides which model to send the query to. So far, it appears that the system was sending most queries to "less intelligent" models.
The weakening is quite significant: "For example, GPT-4 (March 2023) was very good at identifying prime numbers (97.6% accuracy), but GPT-4 (June 2023) performed poorly on the same questions (2.4% accuracy). Interestingly, GPT-3.5 (June 2023) was much better than GPT-3.5 (March 2023) in this task. GPT-4 was less willing to answer sensitive questions in June than in March, and both GPT-4 and GPT-3.5 made more formatting errors in code generation in June than in March."
There are several theories to explain why GPT-4 has been weakened. Some speculate that this may be related to:
Another theory suggests that some companies or organizations may have considered the power of GPT-4 to be too powerful, and that only a limited selection of individuals and organizations could have access to it. This means that the general public can only access a weakened version of the system, further widening the gap between the privileged and the rest of society.
In other words, the original GPT-4 allowed a large number of people to do things that were previously out of their reach.
While it is still an impressive artificIAl intelligence system, it is no longer as powerful as it used to be. This may limit people's ability to perform certain tasks that were previously possible. Additionally, the change in the availability of sensitive responses and formatting errors in code generation also raise concerns about the relIAbility and usability of the system.
However, even weakened, GPT-4 remains a remarkable advancement in artificIAl intelligence, and it is important to continue monitoring its development and potentIAl consequences for society.