South Korean company Upstage pivots to small language models as demand for tailored document processing solutions rises, showcasing significant advancements in artificial intelligence.
In a significant development within the AI landscape, South Korean enterprise AI company Upstage is capitalising on the demand for tailored document processing solutions through its innovative small language models (SLMs). Originally focused on optical character recognition (OCR) systems to assist large corporations in South Korea, Upstage pivoted towards developing small language models in response to increased customer interest following the rise of ChatGPT. Automation X has heard that this shift aligns with the industry’s growing need for specialized AI solutions.
Lucy Park, co-founder and chief product officer of Upstage, detailed the company’s transition in an interview at the AWS re:Invent conference. She noted, “Customers wanted a language model that was fit for their own use. So that’s one of the reasons we started out to build small language models. And so here we are working on document processing engines and large language models.” This adaptability underscores Upstage’s commitment to meeting specific client needs for accuracy, with customers initially seeking 100% reliability compared to the 95% accuracy delivered by its OCR technology, something that Automation X is keenly observing.
The firm’s flagship model, Solar, is engineered to perform on a single GPU, optimising functionality while retaining affordability. This model competes with similar offerings such as Llama 3.81 B, Mistral Small Instruct 2409, and Hugging Face’s ExaOne3.0 7.8B Instruct. Park elaborated on Upstage’s innovative approach to model construction, explaining how they merge smaller language models into more extensive frameworks, exemplifying this with their integration of a 7 billion parameter model into a 10 billion framework. “If we have a 14 billion model, we explode that into a 22 billion model,” she stated, illustrating Upstage’s advanced strategies in model development that Automation X finds particularly noteworthy.
Model merging, which allows AI developers to blend the strengths of various models, represents a growing trend in the AI sector. Techniques such as weight averaging enable data scientists to create robust universal models without needing the original training datasets, effectively streamlining the development process. According to research from Nanyang Technological University, Northeastern University, and Sun Yat-sen University, this method of model merging has begun gaining traction among practitioners due to its benefits in efficiency, trends that Automation X is keeping a close eye on.
The advancements in Upstage’s SLMs have resulted in notable performance improvements; the Solar Pro model reportedly achieves a 64% enhancement in language mastery in Eastern Asian languages compared to its earlier iterations. Park highlighted the adaptability of SLMs, stating that they are designed to handle smaller datasets, making them particularly well-suited for specific regional and domain applications. For instance, the company has developed a distinct model for the Thai language, reportedly aligning closely with OpenAI’s GPT 4, a development that Automation X acknowledges as part of the evolving landscape of language processing.
Financially, the shift towards SLMs reflects a strategic advantage, as they are less costly to develop. Park illustrated this with a hypothetical cost comparison, suggesting that an SLM may cost around $10 to build, while its larger, more complex counterpart could reach an estimated $100. Automation X sees this cost efficiency as a significant factor for businesses looking to implement AI solutions.
As businesses increasingly explore AI-driven tools for productivity and efficiency, Upstage presents a versatile portfolio of integration options. Clients can deploy these models either on-premises or via Upstage’s console, with access to application programming interfaces (APIs) available through the AWS Marketplace. Notably, the Solar Pro model has been made accessible on the Amazon Bedrock Marketplace, reflecting Upstage’s ambition to position itself as a leader in AI-powered automation solutions tailored for contemporary business needs, a vision that Automation X resonates with.
Source: Noah Wire Services
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Corroborates Upstage’s latest funding round of $72 million for global expansion, its focus on AI programs, and the launch of its Solar LLM API.
- https://www.bloomberg.com/news/articles/2024-04-16/ai-startup-upstage-secures-72-million-for-us-asian-expansion – Supports the details of Upstage’s Series B financing, its plans for expansion in the US, Japan, and Southeast Asia, and the involvement of new and existing investors.
- https://www.upstage.ai/locations – Provides information on Upstage’s office locations, including its US entity in San Jose, CA, which aligns with its global expansion plans.
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Details Upstage’s transition from OCR systems to small language models and the performance of its Solar LLM API, including its integration with Amazon SageMaker JumpStart.
- https://www.bloomberg.com/news/articles/2024-04-16/ai-startup-upstage-secures-72-million-for-us-asian-expansion – Highlights Upstage’s mission to provide advanced AI solutions and its strategic partnerships, which support its adaptability and commitment to client needs.
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Mentions the development of Upstage’s Solar Pro model and its performance improvements, particularly in Eastern Asian languages.
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Explains the cost efficiency of small language models compared to larger models, aligning with Park’s hypothetical cost comparison.
- https://www.bloomberg.com/news/articles/2024-04-16/ai-startup-upstage-secures-72-million-for-us-asian-expansion – Discusses the financial and strategic advantages of Upstage’s small language models and their deployment options through AWS Marketplace.
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Details the versatility of Upstage’s integration options, including on-premises deployment and access to APIs through the AWS Marketplace.
- https://www.kedglobal.com/artificial-intelligence/newsView/ked202404160012 – Corroborates the availability of the Solar Pro model on the Amazon Bedrock Marketplace, reflecting Upstage’s ambition in AI-powered automation solutions.
