As the competition in the generative AI space grows fiercer, OpenAI is upgrading its text-generating models while reducing pricing.
Today, OpenAI announced the release of new versions of GPT-3.5-turbo and GPT-4, the latter being its latest text-generating AI, with a capability called function calling. As OpenAI explains in a blog post, function calling allows developers to describe programming functions to GPT-3.5-turbo and GPT-4 and have the models create code to execute those functions.
For example, function calling can help to create chatbots that answer questions by calling external tools, convert natural language into database queries and extract structured data from text. “These models have been fine-tuned to both detect when a function needs to be called … and to respond with JSON that adheres to the function signature,” OpenAI writes. “Function calling allows developers to more reliably get structured data back from the model.”
Beyond function calling, OpenAI is introducing a flavor of GPT-3.5-turbo with a greatly expanded context window. Context window, measured in tokens, or raw bits of text, refers to the text the model considers before generating any additional text. Models with small context windows tend to “forget” the content of even very recent conversations, leading them to veer off topic — often in problematic ways.
The new GPT-3.5-turbo offers four times the context length (16,000 tokens) of the vanilla GPT-3.5-turbo at twice the price — $0.003 per 1,000 input tokens (i.e. tokens fed into the model) and $0.004 per 1,000 output tokens (tokens the model generates). OpenAI says that it can ingest around 20 pages of text in a single go — short of the hundreds of pages that AI startup Anthropic’s flagship model can process, notably. (OpenAI is testing a version of GPT-4 with a 32,000-token context window, but only in limited release.)
On the plus side, OpenAI says that it’s reducing pricing for GPT-3.5-turbo — the original, not the version with the expanded context window — by 25%. Developers can now use the model for $0.0015 per 1,000 input tokens and $0.002 per 1,000 output tokens, which equates to roughly 700 pages per dollar.
Pricing is also being reduced for text-embedding-ada-002, one of OpenAI’s more popular text embedding models. Text embeddings measure the relatedness of text strings, and are commonly used for search (where results are ranked by relevance to a query string) and recommendations (where items with related text strings are recommended).
Text-embedding-ada-002 now costs $0.0001 per 1,000 tokens, a 75% reduction from the previous price. OpenAI says the reduction was made possible by increased efficiency in its systems — a key area of focus for the startup, no doubt, as it spends hundreds of millions of dollars on R&D and infrastructure.
OpenAI has signaled that incremental updates to existing models — not massive new from-scratch models — are its MO following the release of GPT-4 in early March. At a recent conference hosted by Economic Times, CEO Sam Altman reaffirmed that OpenAI hasn’t begun training the successor to GPT-4, indicating that the company “has a lot of work to do” before it starts that model.