Large Language Models (LLMs) are more than just the latest tech trend; they're game-changers, especially in the B2B SaaS space. There's a growing buzz around how they'll transform the industry, and for good reason. The tech world is especially bullish on their potential to innovate in B2B SaaS.

But as we dive deeper, a key question emerges:

Which is the better path - Training your own small LLMs (like Mistral-& B, Llama-7B) or using larger LLMs (GPT-4, Claude 2) with APIs?

It's a big decision with a lot at stake for businesses. In this article, I intend to cut through the noise to explore these two approaches.

How SaaS evolved and its inherent constraints

SaaS has been a game-changer in how businesses operate. It stepped in to automate what used to be human-driven actions, embedding the logic of these actions directly into software. This shift brought obvious benefits. Mistakes were reduced, costs cut down, and the time and resources spent on training significantly lessened.

But this evolution wasn't without its challenges. The core issue was scalability in terms of unique actions.

Engineering teams, regardless of their talent, could only hardcode a limited number of frequently used actions. This limitation wasn't just a technical one; it was also about what made economic sense.

After all, investing resources in coding a wide array of less common actions didn't always add up financially.

This is where the long tail comes in – a vast array of less frequent, but still important, actions that needed attention. As they did not come out of the box, these were either handled manually or required specialized coding by internal teams or external agencies.

This backdrop of scalability and economic challenges in traditional SaaS models sets the stage for a significant evolution – the integration of LLMs, offering solutions where previous models fell short.

The Future of SaaS with LLMs

Currently, most companies are leveraging LLMs only as advanced Natural Language Processing (NLP) tools. This allows them to understand user intent more accurately and then connect it to the appropriate hardcoded logic, thus maintaining the workflow capabilities of traditional SaaS while significantly enhancing user experience.

However, the true potential of LLMs in SaaS goes far beyond just interpreting user intent. At their core, LLMs are sophisticated logical engines capable of building and executing logic in real-time. This capability is a game-changer. It means that LLMs can handle not just the frequently occurring tasks but also the long tail of less common, yet critical, actions that traditional SaaS models struggle with.

With the advent of LLMs, it's now possible to profitably manage the whole long tail. LLMs offer a scalable and efficient way to automate even the most niche tasks, transforming how SaaS can support diverse business needs.

Businesses can now expect their SaaS tools to not only understand and execute routine tasks but also to intelligently navigate and manage complex, unique scenarios – something that was previously unattainable.

In essence, the future of SaaS with LLMs is not just about doing the same things better. It's about expanding the possibilities of what SaaS can do, making it a more integral and dynamic part of business operations.

Comparative Analysis - Small vs. Large Models in SaaS

Having established the transformative potential of LLMs in SaaS, a pivotal question arises: What's the most effective way to create the logic that drives these systems? This isn't just a technical choice – it's a strategic one, with each option presenting a unique set of advantages and challenges.

Small Models - The Compact Solution

Initially, small models seem appealing. They're less demanding in terms of computational power, which translates to faster performance and lower costs. Plus, they offer a level of independence from third-party LLM providers, a crucial consideration when dealing with sensitive internal data.

However, the key Question still lingers: Can small models truly deliver the complex logic needed to drive today's diverse business environment?

It's a pressing question.

For instance, GPT2 (the smaller model) did not show any logical or other emergent abilities. They started with the much larger GPT3. The current consensus is that a large compute gives the model enough capacity to store its model of the world. And the logic flows from that.

Most people are trying to cram the logic of a larger model for specific expertise into a smaller model by fine-tuning it with a dataset of that specific expertise only. It's like trying to turn a child into a marketing guru by teaching them only marketing from an early age – it just doesn't work that way.

In the B2B SaaS domain, where an engineer must grasp the nuances of the industry, data sources, and specific company details to be effective, this depth of understanding becomes crucial. And for this depth to exist, the model size needs to be large.

The model to evaluate the performance of LLM models

When it comes to the effectiveness of LLMs in the B2B SaaS domain, three key parameters stand out: accuracy, latency, and cost.

Each of these factors plays a crucial role in determining the viability and efficiency of the models, whether small or large.

Accuracy:

Small Models: Even the best of the smaller models have their limitations. They are prone to hallucinations – incorrect or irrelevant responses – in about 25% of instances. This rate of inaccuracy can be a significant drawback, especially in complex business scenarios where precision is paramount.

Large Models: While hallucinations are also present in larger models, their frequency is notably lower compared to smaller models. However, it's important to recognize that these inaccuracies are unlikely to be completely eliminated in the future.

Cost:

Small Models: The per-query cost of small models might seem economical at first glance. However, since these models are often fine-tuned for specific company needs, they need to be operational continuously. This requirement can translate into substantial ongoing costs, often starting from $7,000 per month.

Large Models: On the other hand, while individual queries may cost more with large models, their overall monthly expenses can be more manageable. This is because many large models are available via APIs, with costs accruing based on usage rather than constant operation. This pay-per-use model can be more financially viable, especially for companies with variable query volumes.

Latency:

Both small and large models typically process queries within a matter of seconds. This similarity in latency means that the decision between small and large models doesn't hinge on speed but rather on the balance between accuracy, cost, and the specific operational needs of the business.

Our Approach: Utilize large LLM models while correcting the side effects of their hallucinations and reliability issues

We have chosen to build with large LLM models while focusing on removing hallucinates and reliability issues. The management of hallucinates and reliability issues will be needed irrespective of the model size to get to an enterprise-grade solution.

This approach isn't just about tackling current problems. It's a long-term play, ensuring that as these models evolve on logic and other capabilities, they automatically enhance our solutions, making them not just smarter but also more reliable.

In summary, while small models have their advantages, the comprehensive capabilities and evolving reliability of large models make them a more compelling choice for future-proofing SaaS solutions. As these models continue to improve, they promise to make our solutions not just smarter, but more aligned with the complex and ever-changing landscape of business needs.