CriticGPT Will Improve the Accuracy of ChatGPT

OpenAI has unveiled an innovative artificial intelligence model built on GPT-4 technology called CriticGPT. It specializes in finding and fixing errors in the code generated by ChatGPT. The main task of CriticGPT is to assist AI trainers in the process of reinforcement learning (Reinforcement Learning from Human Feedback, RLHF). Test results showed that the use of CriticGPT improves the performance of AI trainers by more than 60%.

The company said that the ability of generative AI models to reason logically and model behavior is constantly improving, so ChatGPT becomes more accurate and its errors are harder to detect. This makes it difficult for AI trainers to identify errors and slows down the RLHF process. To solve this problem, experts trained CriticGPT to formulate criticisms that focus on inaccuracies in ChatGPT responses.

According to OpenAI, the CriticGPT model analyzes the code generated by GPT-4, highlights errors, comments on them, and suggests correct solutions. CriticGPT was trained on a dataset containing specially introduced errors (the developers manually added them to the ChatGPT code). Tests have demonstrated that the implementation of CriticGPT significantly improves the quality of work of AI trainers, increasing their productivity by 63%.

Interestingly, CriticGPT's capabilities extend far beyond simple code review. The experts applied the model to a sample of ChatGPT training data that had previously been rated as ideal by human reviewers. CriticGPT identified errors in 24% of cases, which were subsequently confirmed by human reviewers. OpenAI believes this validates the model's ability to adapt to a variety of non-coding tasks. In addition, it emphasizes its ability to identify the most subtle errors that may escape attention even with careful checking.

The disadvantage of CriticGPT OpenAI is that it is trained on a limited amount of data, which makes the model effective only for relatively simple tasks. Training increasingly complex AI systems will require more advanced tools. In this regard, the company plans to scale up models like CriticGPT and integrate them into the RLHF process. This is part of the strategy to develop better tools for assessing the performance of LLM systems because in the near future it will become extremely difficult for a person to make such an assessment without additional support.