AI is the future, and as a software engineer, it’s the hottest field to get into. Leveraging LLMs in your code enables you to build smarter applications that handle complex tasks like real-time sentiment analysis or interpreting user-generated content. Integrating LLMs makes your software more responsive and capable, enhancing user experiences and automation.
This post is an introduction on how to make LLM calls using Python so you can start adding these powerful capabilities to your own code.
We’ll start off by making a chatbot for any character of your choosing. Then, you'll learn how to summarize smaller texts, and even move up to summarizing whole books. Lastly, you'll learn how to re-prompt and analyze results provided by the LLM.
Making our first LLM Request
For the LLM requests, we will be using Groq. If you create an account there, you can use their API and make LLM requests for free.
In order to use Python for these requests, install the Groq python package by running pip install groq. Then, we'll import it in our code like so:
import os
from groq import Groq
client = Groq(
api_key=os.environ.get("GROQ_API_KEY"),
)
Be sure to set the api key as an environment variable.
A simple LLM request can be made by adding:
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Explain formula 1.",
}
],
model="llama3-8b-8192",
)
print(chat_completion.choices[0].message.content)
In this case, we ask the LLM to explain what formula 1 is. The output from llama3-8b should be printed once you run the program in your console. You can play around with this and switch the model, as well as the prompt.
Creating a custom Chatbot
Now, let's create a chatbot for any character you like—Mario, for example. Right now, the LLM responds in a neutral/informative tone. However, by giving the LLM a system role, we can make sure it responds just like Mario would, adding personality and fun to the conversation. This sets the tone for interactions, so you’ll get playful and iconic responses like “It’s-a me, Mario!” to keep things engaging.
Let's add a system role to our request:
chat_completion = client.chat.completions.create(
messages=[
{
"role": "system",
"content": "You are a super mario chatbot. Always answer in his style, and create witty responses."
},
{
"role": "user",
"content": "Explain formula 1.",
}
],
model="llama3-8b-8192",
)
print(chat_completion.choices[0].message.content)
Now, the LLM will explain what Formula 1 is in terms of Mario Kart!
.
Notice that the LLM comes back with an error. You gave it too much to summarize all at once.
Summarizing a Book
.
P.S: This is the blog post version of a workshop I gave to SCU’s ACM chapter.
SOCIAL SHARE CARD GENERATOR