Large Language Models (LLMs) are more than just the latest tech trend; they're game-changers, especially in the B2B SaaS space. There's a growing buzz around how they'll transform the industry, and for good reason. The tech world is especially bullish on their potential to innovate in B2B SaaS.

But as we dive deeper, a key question emerges:

Which is the better path - Training your own small LLMs (like Mistral-& B, Llama-7B) or using larger LLMs (GPT-4, Claude 2) with APIs?

It's a big decision with a lot at stake for businesses. In this article, I intend to cut through the noise to explore these two approaches.

How SaaS evolved and its inherent constraints

SaaS has been a game-changer in how businesses operate. It stepped in to automate what used to be human-driven actions, embedding the logic of these actions directly into software. This shift brought obvious benefits. Mistakes were reduced, costs cut down, and the time and resources spent on training significantly lessened.

But this evolution wasn't without its challenges. The core issue was scalability in terms of unique actions.

Engineering teams, regardless of their talent, could only hardcode a limited number of frequently used actions. This limitation wasn't just a technical one; it was also about what made economic sense.

After all, investing resources in coding a wide array of less common actions didn't always add up financially.

This is where the long tail comes in – a vast array of less frequent, but still important, actions that needed attention. As they did not come out of the box, these were either handled manually or required specialized coding by internal teams or external agencies.

This backdrop of scalability and economic challenges in traditional SaaS models sets the stage for a significant evolution – the integration of LLMs, offering solutions where previous models fell short.

The Future of SaaS with LLMs

Currently, most companies are leveraging LLMs only as advanced Natural Language Processing (NLP) tools. This allows them to understand user intent more accurately and then connect it to the appropriate hardcoded logic, thus maintaining the workflow capabilities of traditional SaaS while significantly enhancing user experience.

However, the true potential of LLMs in SaaS goes far beyond just interpreting user intent. At their core, LLMs are sophisticated logical engines capable of building and executing logic in real-time. This capability is a game-changer. It means that LLMs can handle not just the frequently occurring tasks but also the long tail of less common, yet critical, actions that traditional SaaS models struggle with.

With the advent of LLMs, it's now possible to profitably manage the whole long tail. LLMs offer a scalable and efficient way to automate even the most niche tasks, transforming how SaaS can support diverse business needs.

Businesses can now expect their SaaS tools to not only understand and execute routine tasks but also to intelligently navigate and manage complex, unique scenarios – something that was previously unattainable.

In essence, the future of SaaS with LLMs is not just about doing the same things better. It's about expanding the possibilities of what SaaS can do, making it a more integral and dynamic part of business operations.

Comparative Analysis - Small vs. Large Models in SaaS

Having established the transformative potential of LLMs in SaaS, a pivotal question arises: What's the most effective way to create the logic that drives these systems? This isn't just a technical choice – it's a strategic one, with each option presenting a unique set of advantages and challenges.

Small Models - The Compact Solution

Initially, small models seem appealing. They're less demanding in terms of computational power, which translates to faster performance and lower costs. Plus, they offer a level of independence from third-party LLM providers, a crucial consideration when dealing with sensitive internal data.

‍