The Introduction to Semantic Kernel
Revolutionizing Software Development with AI Integration
I would be overjoyed if you could support me as a referred member. However, the income I’ve received from my contributions on this platform has not proven sufficient to sustain the frequency of updates I’d hoped for. Facing this reality, I’ve made a difficult decision — to explore new income avenues. However, this isn’t the end; rather, it’s an exciting new beginning. I am thrilled to announce my upcoming Substack newsletter, where I’ll delve into my investing system, harnessing the immense potential of IT technology and embracing a system-thinking approach to investment strategies. Rest assured, I will still be posting when inspiration strikes.
Semantic Kernel (SK) is a lightweight open-source SDK designed by Microsoft, serving as a programming model for infusing AI capabilities into applications. It enables developers to seamlessly combine traditional programming languages like C# and Python with powerful large language models such as GPT-3.5.
The advantages of using Semantic Kernel for businesses are significant. It streamlines AI service integration, optimizes resource management, and simplifies complex interactions. SK provides context management, offers integration flexibility with external systems, and incorporates embedded memory, enhancing the accessibility and cost-effectiveness of AI. Without SK, businesses would face the challenge of managing intricate AI interactions independently, which could consume time and resources.
The emergence of Semantic Kernel represents a paradigm shift in software engineering, akin to transitioning from syntax to semantics in programming. It streamlines the utilization of large language models, enabling natural language interactions with AI through straightforward API calls. SK revolutionizes software’s use of AI, offering convenience and reshaping human-computer interactions in the age of large models.
What is SDK
An SDK, or Software Development Kit, is a comprehensive set of software tools and resources that developers use to create, test, deploy, and maintain applications or software. SDKs typically encompass the following components:
- Libraries — These contain pre-written code modules designed for specific programming tasks. Libraries offer common functions and tools to reduce the amount of code developers must write.
- Documentation and Tutorials — SDKs provide detailed documentation and tutorials to assist developers in getting started and understanding how to use the SDK for application development.
- Sample Code — They include sample applications and code snippets that illustrate how to utilize the SDK to perform specific tasks or address problems. These examples aid developers in comprehending the SDK’s usage.
- Tools — SDKs comprise utilities for various development activities, such as integrated development environment (IDE) plug-ins, simulators, and debuggers, which facilitate application development, debugging, and testing.
SDKs serve a diverse range of software development purposes, including mobile app development, web app development, embedded systems development, and game development.
What is the Kernel?
The kernel serves as a core component of an operating system, responsible for managing hardware resources like the processor, memory, and file systems. Its primary role is to allocate and coordinate these resources to ensure the proper functioning of applications. Furthermore, the kernel operates with varying privilege levels, allowing it to execute critical system operations while upholding system security and integrity.
What is Semantic Kernel?
Semantic Kernel, an open-source Software Development Kit (SDK) from Microsoft, empowers developers to seamlessly integrate powerful Large Language Models (LLMs) into their applications, including services from providers like OpenAI, Azure, and Hugging Face. It achieves this through two key components:
- Connectors — simplify the integration of memory and AI models into applications. Connectors serve as bridges between applications and AI services, streamlining the infusion of AI capabilities into software.
- Plugins — serve as extensions for applications. These Plugins respond to specific triggers and execute predefined operations. Developers can expand their application’s capabilities by seamlessly integrating AI-driven processes using Plugins.
The emergence of large language models like GPT-3.5 has showcased the potential of AI in problem-solving and task completion. However, these models primarily output text data, which requires a bridge like Semantic Kernel to seamlessly integrate them into the programming world.
Large language models like GPT are excellent at explaining tasks but lack the ability to execute them. Training such models, such as GPT-4, from scratch is prohibitively expensive and resource-intensive.
OpenAI utilized around 2.15e25 FLOPS for training GPT-4 over a 90 to 100-day period, employing approximately 25,000 A100 GPUs. The utilization (MFU) during this training was notably low, ranging between 32% and 36%. This low utilization is partly attributed to a high number of failures that necessitated checkpoint restarts.
If we consider the cost, assuming a rate of $1 per hour per A100 GPU in a cloud environment, the training cost for GPT-4 would amount to approximately $63 million. However, using 8192 H100 GPUs for pre-training significantly reduces the time to about 55 days, with a cost of $21.5 million. In this scenario, each H100 GPU is billed at $2 per hour.
Here’s where the Semantic Kernel SDK steps in. It serves as a bridge between these models and real-world actions. Semantic Kernel enhances the language model’s capabilities, enabling it to perform practical tasks like controlling IoT devices. It also facilitates the extraction of specific elements, like predicates, from the model’s responses.
The Semantic Kernel, using tailored prompts, empowers Large Language Models (LLMs) to extract essential information from natural language, including tasks like classification and labeling.
Additionally, unstructured data can be structured by inputting natural language, which the LLM translates into parameters like JSON or XML, making it understandable to computer code. These parameters are directly usable by software, allowing for tasks such as formatting adjustments and error corrections.
Through the Semantic Kernel, LLMs can directly translate user requests into specific code languages, enabling domain-specific actions and instructing machines in natural language without additional training.
The outcomes generated by executing this code are then translated back into plain language by the LLM, facilitating seamless human-computer interactions and bridging the gap between human communication and machine operations.
Semantic Kernel’s operation can be broken down into a series of steps:
- User Input — The process begins with a user asking a question or providing input, which is converted into an “Ask” object representing their query.
- Arrangement — The Semantic Kernel organizes these “Ask” objects, which represent user queries or commands.
- Planning — The system’s planner takes these organized “Ask” objects and divides them into a set of steps, each of which comprises three key components:
- Skill — Each step involves the use of a specific skill or ability, defining the behavior or action required.
- Memory — Contextual memory is employed to retain relevant information and context as the system progresses through the steps.
- Connectors — AI connectors are utilized to access external knowledge sources or APIs.
4. Pipeline Formation — These steps come together to form a pipeline, outlining the sequence of actions to be taken.
5. Execution — The pipeline is executed, which involves the application of skills, utilization of contextual memory, and interaction with external connectors to provide a response or result based on the user’s input.
Using Semantic Kernel (SK) can make a significant difference compared to not using it.
Without Semantic Kernel
Without SK, businesses would need to manually set up parameters like Azure OpenAI’s key, endpoint, model, temperature, max_token, and others directly in their web app code. This process can be complex and time-consuming.
Businesses would typically pass prompts to a single OpenAI model, which may not be the most efficient approach. For instance, if the prompt involves different tasks (e.g., text generation and code generation), it might require distinct models.
Configuring suitable OpenAI super parameters for each part of a prompt may be challenging, leading to suboptimal results.
With Semantic Kernel (Using SK)
SK acts as a middleware, abstracting and simplifying interactions between the application and OpenAI. It provides a bridge between the two, relieving businesses of the technical complexities.
SK analyzes prompts and determines the specific skills required to achieve the goal. Different skills can have their own prompt engineering, associated OpenAI models, and unique superparameters. This ensures that each part of a complex prompt is handled optimally.
SK analyzes prompts and determines the specific skills required to achieve the goal. Different skills can have their own prompt engineering, associated OpenAI models, and unique superparameters. This ensures that each part of a complex prompt is handled optimally.
SK simplifies the interaction with Large Language Models (LLM) like OpenAI. It introduces concepts such as planning, resources, steps, and pipelines to organize and execute user requests efficiently.
Businesses can leverage SK’s customization to choose different skills, memories, and connectors based on various scenarios and needs. This flexibility is crucial in addressing diverse business requirements.
SK helps in better resource utilization, overcoming token limits through embedded memory, and optimizing the number of calls and result quality through planners.
SK eases integration with other systems, enabling businesses to call external APIs or access their own data sources through connectors.
SK’s embedded memory is a valuable feature. It automatically generates and manages memory, allowing businesses to access and manage data more conveniently. This feature also facilitates the combination of memory with LLM AI through planning, resources, steps, and pipelines.
Semantic Kernel signifies a paradigm shift in programming. It resembles a new era in cloud computing. Developers are liberated from concerns about data center operations and maintenance. In the future, software development linked to Large Language Models (LLMs) won’t require model training intricacies. Instead, it will rely on straightforward API calls. AI empowers more convenient semantic comprehension, reshaping human-computer interactions.
References
If you’ve found any of my articles helpful or useful then please consider throwing a coffee my way to help support my work or give me patronage😊, by using
Last but not least, if you are not a Medium Member yet and plan to become one, I kindly ask you to do so using the following link. I will receive a portion of your membership fee at no additional cost to you.