Skip to Content
DocumentationAlith Using Free LLM

Using Alith with Free LLM (Groq)

This guide will show you how to configure Alith with free LLM models provided by Groq.
Follow the steps below for Rust, Python, and Node.js.

Get Groq API Key

  • Visit Groq  Sign Up for Groq

  • Go to API Keys → Generate a new key

  • Copy your Groq API Key — we’ll use it .env file

  • Add the key as an environment variable:

  • Unix/MacOS

export GROQ_API_KEY=<your-api-key>
  • Windows
GROQ_API_KEY = "<your-api-key>"

Write a code

use alith::{Agent, Chat, LLM};
use std::env;
 
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let api_key = env::var("GROQ_API_KEY")?;
    let model = LLM::from_custom_model(
        "llama3-70b-8192",
        "https://api.groq.com/openai/v1",
        api_key
    )?;
 
    let agent = Agent::new("groq agent", model)
        .preamble("You are a comedian here to entertain the user using humour and jokes.");
 
    let response = agent.prompt("Entertain me!").await?;
    println!("{}", response);
 
    Ok(())
}

Run the code

cargo run --release
Last updated on
CEG Forum Scraper: Technical DescriptionDeveloping