GPT-3.5, GPT-4: Learn the difference

ChatGPT is a chatbot application developed by OpenAI. Different versions of GPT (such as GPT-3.5 and GPT-4) are the chatbot’s “brains”, the artificial intelligence that allows ChatGPT recognize, understand, and generate text in a human-like manner.

GPT-3.5

GPT-3.5 is a subclass of the 3^rd iteration of Generative Pre-Trained Transformer. It is a large language model based on transformer architecture that has been trained on vast amounts of text data to understand and respond in natural languages (such as English, Spanish, French, etc.). That’s called natural language processing. The transformer architecture is more advanced than previous recurrent neural architecture. In simple words, it helps the language model better comprehend and apprehend the text.

Trasformers are able to better understand context, perceive the connections among words in a sentence and paragraph, and emphasize the key ideas within a text.

GPT-3.5 has 175 billion learning parameters. At the time this was the most by any other large language model. These parameters are like neural connections, the more the better. The most surprising thing is that at some point, when the number of parameters increases, the model becomes an expert even in areas that no one has specially trained it for: translating from one language to another, solving logical and mathematical problems, for instance.

To make the interaction with GPT-3.5 more natural and safer, a technique called reinforcement learning from human feedback was applied, where human input is utilized to improve machine-learning algorithms.

ChatGPT-4 and how it’s different from ChatGPT-3.5

GPT-4 has 100 trillion parameters!

The new model is many times more advanced. One important difference that immediately catches your eye is that the GPT-4 has learned to recognize images. Here's what it can do:

• describe what's in the picture,
• explain visual jokes,
• come up with a caption for a photo,
• suggest a recipe based on the food in the picture,
• understand graphs, charts and handwritten text.

For example, based on a hand-drawn template, GPT-4 can write the code for the web page you want to create.

GPT-4 also does better than its predecessor at processing textual information: it memorizes large amounts of text for better contextual understanding and gives 40% more accurate answers. GPT-4 can process the equivalent of 300 pages of text (128 000 tokens) in a single prompt, while GPT-3.5 could process only 14 pages (16 000 tokens).

GPT-4 is so smart that it passed the bar exam, placing in the top 10% (GPT-3.5 ended up trailing humans by about 17%). In many tests, the model outperforms even humans. Specifically, in math, physics, and chemistry tests, GPT-4 outperformed 88% of test takers.

*1000 tokens is about 750 words

GPT-4 outperforms GPT-3.5 in many different areas: from songwriting and scriptwriting to technical writing and language translations.

GPT-4 criticism

GPT-4 is not perfect by any means. It seems to us that neural networks are only getting better every day, but a study from Stanford in June 2023 showed that GPT-4 results have deteriorated since March.

Testing GPT-4 and GPT-3.5 in March and June 2023

The model showed significant performance degradation in solving math problems and code generation:

• for example, it had difficulty determining whether the number 17077 was prime,
• and only 10% of the time it was able to write working code in tasks classified as easy by LeetCode.

At the same time, GPT-4 showed improvements in visual reasoning and answering sensitive questions (where the answer could cause harm or break the law).

Critics of this study pointed out possible errors in methodology and noted that the resulting dynamics should be viewed as behavior change rather than deterioration.