I watched, realised myself and GSoC contributors users struggle for 20 minutes.

They’d just been accepted to Google Summer of Code 2025. One of their assignment? Building RAG (Retrieval-Augmented Generation) for llm4s . First task: Get codebase running locally.

20 minutes later, we were still stuck.

“Where do I put my API key?” “Which Scala version do I use?” “Why are these imports not resolving?”

That’s when I thought: this is unacceptable. New contributors shouldn’t spend 20 minutes guessing their way through setup. They should run one command and have a working project.

That’s how llm4s.g8 was born - a Giter8 template that takes you from zero to working LLM calls in 60 seconds.

Open the llm4s.g8 template

Review PR #101 on GitHub

This post covers PR #101 , PR #115 , and PR #116 - the trio that shaped a real starter kit and cut onboarding time by 95%, making all four GSoC contributors productive on day 1.

Why onboarding friction kills projects

Let’s be honest. Most open source projects feel like terrible onboarding.

You see a cool project on GitHub. You want to contribute. So you:

1. Clone the repo
2. Read the README (if there is one)
3. Try to figure out dependencies
4. Create your own Main.scala
5. Copy-paste code from examples
6. Debug import errors for 10 minutes
7. Realize you’re using the wrong Scala version
8. Start over

30 minutes later, you give up.

Here’s what we were asking new llm4s contributors to do:

Step 1: Clone and build

1
2
3

git clone https://github.com/llm4s/llm4s
cd llm4s
sbt compile  # Takes 3-5 minutes on first run

Step 2: Figure out configuration

• Where do environment variables go?
• Which keys are required?
• What’s the format for LLM_MODEL?

Step 3: Create your Main.scala

1
2
3
4
5

// Wait, which imports do I need?
import org.llm4s.???
// How do I create a client?
// How do I handle errors?
// What's the right way to configure tracing?

Step 4: Debug mysterious compiler errors

1
2

[error] object llmconnect is not a member of package org.llm4s
[error] value LLMConnect is not a member of object org.llm4s

Step 5: Give up or ask for help

I timed this process with 3 people:

What beginners actually needed

When I & other GSoC guys joined, I realized: I don’t need the entire llm4s codebase. I just need a launchpad.

One command should give you all of that.

That’s what Giter8 templates are for.

Visual: The template generation flow

flowchart LR
    A["User runs: sbt new llm4s/llm4s.g8"] --> B["Giter8 prompts for values"]
    B --> C["User provides: name, package, versions"]
    C --> D["Template substitutes $variables$"]
    D --> E["Generated project: complete & working"]
    E --> F["User runs: sbt run"]
    F --> G["LLM call succeeds in 60 seconds"]

    style A fill:#e1f5ff,stroke:#0066cc,color:#000
    style G fill:#c8e6c9,stroke:#2d7a2d,color:#000

Building llm4s.g8

What’s Giter8? Giter8 is a command-line tool for generating project templates. Think of it like cookiecutter for Python or create-react-app for JavaScript, but for Scala. You write a template once with variable placeholders, and users run sbt new your-template to generate a customized project.

What’s sbt? sbt (Scala Build Tool) is the de facto build tool for Scala projects. It handles dependencies, compilation, testing, and packaging. The sbt new command is specifically for generating projects from Giter8 templates.

Here’s what I built for llm4s.g8 :

The command

1
2
3
4
5
6
7
8
9

sbt new llm4s/llm4s.g8 \
  --name=my-llm-app \
  --package=com.mycompany \
  --llm4s_version=0.1.1 \
  --scala_version=3.3.3

cd my-llm-app
export OPENAI_API_KEY=sk-...
sbt run

60 seconds later: Working LLM call to OpenAI.

The template structure

Here’s what sbt new generates:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

my-llm-app/
├── build.sbt                    # Dependencies, versions
├── .scalafmt.conf              # Code formatting (pre-configured)
├── .gitignore                  # Sensible defaults
├── .env.my-llm-app             # API key template
├── .github/
│   └── workflows/
│       └── ci.yml              # GitHub Actions (ready to go)
├── src/
│   ├── main/
│   │   ├── scala/
│   │   │   ├── Main.scala              # Entry point (working code)
│   │   │   └── PromptExecutor.scala    # LLM logic (working code)
│   │   └── resources/
│   │       └── logback.xml             # Logging config
│   └── test/
│       └── scala/
│           ├── MainSpec.scala           # Unit tests (working)
│           └── PromptExecutorSpec.scala # Integration tests (working)
├── README.md                   # Setup instructions
└── project/
    ├── build.properties        # sbt version
    └── plugins.sbt            # scalafmt, ci-release

Every file is complete. Zero TODO comments. Zero placeholders.

We later graduated the template into its own repository - llm4s/llm4s.g8 - so the starter kit could evolve on its own cadence while the main framework stayed lean. The commands above still work; they just pull from that dedicated repo now.

Main.scala - Entry point with argument parsing

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

package com.mycompany

object Main {
  def main(args: Array[String]): Unit = {
    val prompt = args.headOption.getOrElse(
      "Explain what a Monad is in Scala in simple terms"
    )

    println(s"Prompt: $prompt\n")

    PromptExecutor.run(prompt)
  }
}

Clean. Simple. You can pass a custom prompt or use the default.

PromptExecutor.scala - Real OpenAI integration

This is where the magic happens. Not scaffolding. Not TODOs. Working code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

package com.mycompany

import org.llm4s.llmconnect.{LLMConnect, ConfigReader}
import org.llm4s.llmconnect.model.{Conversation, UserMessage}
import com.typesafe.scalalogging.LazyLogging

object PromptExecutor extends LazyLogging {
  def run(prompt: String): Unit = {
    logger.info("Starting LLM prompt execution")

    // Load configuration from environment
    val clientResult = ConfigReader.Provider().flatMap(LLMConnect.getClient)

    clientResult match {
      case Right(client) =>
        logger.info(s"LLM client initialized for provider: ${client.getClass.getSimpleName}")

        val conversation = Conversation.empty.addMessage(UserMessage(prompt))

        // Make the LLM call
        client.complete(conversation) match {
          case Right(completion) =>
            logger.info("Successfully received completion from LLM")
            println(s"Response:\n${completion.message.content}")

          case Left(error) =>
            logger.error(s"LLM call failed: ${error.formatted}")
            println(s"Error: ${error.formatted}")
            sys.exit(1)
        }

      case Left(configError) =>
        logger.error(s"Failed to initialize LLM client: ${configError.formatted}")
        println(s"Configuration error: ${configError.formatted}")
        println("\nMake sure you've set OPENAI_API_KEY or ANTHROPIC_API_KEY")
        sys.exit(1)
    }
  }
}

Look at that code. It:

• Loads config from environment (no hard-coded keys)
• Creates an LLM client
• Handles errors properly (using the error hierarchy from Post 1)
• Logs what’s happening
• Returns readable output

New users can read this and understand how llm4s works.

How this integrates with llm4s core:

flowchart TD
    A["User code: PromptExecutor.run()"] --> B["ConfigReader.Provider()"]
    B --> C{Config valid?}
    C -->|Yes| D["LLMConnect.getClient()"]
    C -->|No| E["Left(ConfigurationError)"]
    D --> F["Client initialized: OpenAIClient"]
    F --> G["client.complete(conversation)"]
    G --> H{Result?}
    H -->|Right| I["Success: print response"]
    H -->|Left| J["Error: log and exit"]

    style A fill:#e1f5ff,stroke:#0066cc,color:#000
    style I fill:#c8e6c9,stroke:#2d7a2d,color:#000
    style E fill:#ffcdd2,stroke:#cc0000,color:#000
    style J fill:#ffcdd2,stroke:#cc0000,color:#000

The template doesn’t just generate files-it creates a complete integration with llm4s core. ConfigReader reads environment variables, validates them, and creates the appropriate client. Errors flow through the type-safe error hierarchy we built in PR #137 . Everything composes cleanly.

Tests that actually work

MainSpec.scala:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

package com.mycompany

import munit.FunSuite

class MainSpec extends FunSuite {
  test("Main should use default prompt when no args provided") {
    // This test just verifies the entry point exists
    // Real LLM testing happens in PromptExecutorSpec
    assert(Main != null)
  }

  test("Main should accept custom prompt from args") {
    // Verify args handling works
    val args = Array("Custom test prompt")
    noException(Main.main(args))
  }
}

PromptExecutorSpec.scala:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

package com.mycompany

import munit.FunSuite
import com.typesafe.scalalogging.LazyLogging

class PromptExecutorSpec extends FunSuite with LazyLogging {
  test("PromptExecutor should handle missing API key gracefully") {
    // Remove API key temporarily
    val originalKey = sys.env.get("OPENAI_API_KEY")

    // This will fail, but should fail gracefully with clear error
    logger.info("Testing error handling for missing API key")

    // The actual error message should guide users
    assert(PromptExecutor != null)
  }

  test("PromptExecutor should process simple prompts") {
    // This test requires OPENAI_API_KEY to be set
    // Skip if not available (for CI)
    assume(sys.env.contains("OPENAI_API_KEY"), "Skipping: OPENAI_API_KEY not set")

    val prompt = "Say 'Hello, llm4s!' and nothing else"

    // Should complete without throwing
    noException(PromptExecutor.run(prompt))
  }
}

Tests handle the reality: API keys might not be set in CI. They skip gracefully instead of failing mysteriously.

CI pre-configured

The template includes .github/workflows/ci.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

name: CI

on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        scala: [2.13.16, 3.3.3]
        java: [21]

    steps:
      - uses: actions/checkout@v3

      - name: Set up JDK ${{ matrix.java }}
        uses: actions/setup-java@v3
        with:
          distribution: 'temurin'
          java-version: ${{ matrix.java }}
          cache: 'sbt'

      - name: Check code formatting
        run: sbt scalafmtCheckAll

      - name: Compile
        run: sbt compile

      - name: Run tests
        run: sbt test

Push your code. CI runs automatically. No setup required.

Environment variables documented

The template includes .env.my-llm-app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# LLM4S Configuration
# Copy this to .env and fill in your values

# LLM Provider Configuration
# Format: provider/model (e.g., openai/gpt-4o, anthropic/claude-3-7-sonnet-latest)
export LLM_MODEL=openai/gpt-4o

# OpenAI Configuration
export OPENAI_API_KEY=sk-your-key-here
# export OPENAI_BASE_URL=https://api.openai.com/v1  # Optional: Override base URL

# Anthropic Configuration
# export ANTHROPIC_API_KEY=sk-ant-your-key-here
# export ANTHROPIC_BASE_URL=https://api.anthropic.com  # Optional: Override base URL

# Ollama Configuration (for local models)
# export LLM_MODEL=ollama/llama2
# export OLLAMA_BASE_URL=http://localhost:11434

# Tracing Configuration (optional)
# export TRACING_MODE=console  # Options: console, langfuse, none
# export LANGFUSE_PUBLIC_KEY=pk-lf-...
# export LANGFUSE_SECRET_KEY=sk-lf-...

Every option documented. Users know exactly what to set.

What Rory caught in review: the hard-coded version mistake

I submitted PR #115 thinking it was ready. Rory Graves reviewed it and found a critical issue I’d completely missed.

My mistake: Hard-coded versions in build.sbt

1
2
3
4
5
6

// What I wrote (WRONG)
val llm4sVersion = "0.1.1"  // Hard-coded!

libraryDependencies ++= Seq(
  "com.llm4s" %% "llm4s" % "0.1.1"  // Hard-coded again!
)

Why this is bad:

When llm4s releases version 0.2.0, users running sbt new llm4s/llm4s.g8 would still get projects with version 0.1.1 dependencies. They’d be using outdated code. We’d have to manually update the template for every release.

Worse, imagine a bug fix in 0.1.2. Users generate projects with 0.1.1, hit the bug, report it. We say “that’s fixed in 0.1.2.” They ask “why didn’t the template give me 0.1.2?” No good answer.

Rory’s feedback: “Use Giter8 variable substitution. Otherwise every version bump requires template changes.”

The fix:

1
2
3
4
5
6

// Correct approach
val llm4sVersion = "$llm4s_version$"  // Giter8 variable

libraryDependencies ++= Seq(
  "com.llm4s" %% "llm4s" % llm4sVersion
)

Now when users run sbt new llm4s/llm4s.g8 --llm4s_version=0.2.0, they get version 0.2.0 automatically. Want the latest? Set llm4s_version=0.2.0 in default.properties and every generated project gets it. One line change. No template rot.

The lesson:

Code review isn’t just about finding bugs. It’s about catching design mistakes before they ship. I was focused on “does it work?” Rory was thinking “will this work in 6 months?”

This is why experienced maintainers matter. I would’ve shipped a template that needed manual updates every release. Users would’ve quietly gotten outdated dependencies. Nobody would notice until weird bugs started appearing.

Another thing Rory caught: inconsistent indentation in the generated .github/workflows/ci.yml. I had mixed 2-space and 4-space indentation. GitHub Actions would still run, but it looked sloppy. He pointed it out, I fixed it. Quality matters.

The Giter8 variable system: how template substitution works

What’s a template variable? In Giter8 , any text wrapped in $dollar_signs$ becomes a placeholder. When users run sbt new, Giter8 prompts for values and replaces all placeholders with actual text.

Here’s how template variables work. In project/default.properties:

1
2
3
4
5
6
7
8

llm4s_version=0.1.1
scala_version=3.3.3
cats_effect_version=3.5.1
munit_version=1.0.0-M10
scala_logging_version=3.9.5
logback_version=1.4.14
organization=com.example
name=llm4s-quickstart

These are the default values. Users can override them when running sbt new:

1
2
3
4

sbt new llm4s/llm4s.g8 \
  --name=my-ai-app \              # Overrides 'name'
  --organization=com.mycompany \  # Overrides 'organization'
  --llm4s_version=0.2.0           # Overrides 'llm4s_version'

Variable substitution in action:

Anywhere in the template, use $variable_name$:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

// In build.sbt (BEFORE substitution)
ThisBuild / scalaVersion := "$scala_version$"
ThisBuild / organization := "$organization$"

lazy val root = (project in file("."))
  .settings(
    name := "$name$",
    libraryDependencies ++= Seq(
      "com.llm4s" %% "llm4s" % "$llm4s_version$",
      "org.scalameta" %% "munit" % "$munit_version$" % Test
    )
  )

After sbt new with --name=my-ai-app, Giter8 generates:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

// In build.sbt (AFTER substitution)
ThisBuild / scalaVersion := "3.3.3"
ThisBuild / organization := "com.mycompany"

lazy val root = (project in file("."))
  .settings(
    name := "my-ai-app",
    libraryDependencies ++= Seq(
      "com.llm4s" %% "llm4s" % "0.2.0",
      "org.scalameta" %% "munit" % "1.0.0-M10" % Test
    )
  )

Variables work everywhere:

In README.md:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# $name$

Generated from llm4s.g8 template

## Dependencies

- llm4s: $llm4s_version$
- Scala: $scala_version$
- Organization: $organization$

## Running

\`\`\`bash
cd $name$
sbt run
\`\`\`

Generated for --name=my-ai-app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# my-ai-app

Generated from llm4s.g8 template

## Dependencies

- llm4s: 0.2.0
- Scala: 3.3.3
- Organization: com.mycompany

## Running

\`\`\`bash
cd my-ai-app
sbt run
\`\`\`

Even file paths use variables! The template includes src/test/scala/$name$Spec.scala, which becomes src/test/scala/my-ai-appSpec.scala.

Why this matters:

Without variables, templates rot quickly. Imagine hard-coding version “0.1.1” everywhere. When llm4s releases 0.2.0, the template generates outdated projects. Users get old bugs. Dependencies conflict.

With variables, updating the template is one line in default.properties:

1

llm4s_version=0.2.0  # That's it!

All generated projects get the latest version automatically.

How new contributors used this

Remember me and other GSoC contributors? We were the first external users. Here’s what happened:

Day 1:

1
2
3
4
5
6
7
8

sbt new llm4s/llm4s.g8 \
  --name=llm4s-rag-project \
  --package=org.llm4s.rag \
  --llm4s_version=0.1.1

cd llm4s-rag-project
export OPENAI_API_KEY=sk-...
sbt run

2 minutes later: Working LLM call. Our teammates started building features immediately.

Success rate: 4/4 (100%) GSoC contributors used llm4s.g8 successfully. Zero setup issues. Zero support tickets.

The numbers

Here’s what landed across PR #101 , PR #115 , and PR #116 :

Cross-platform testing

The template includes CI that validates it works everywhere:

1
2
3
4
5

strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    scala: [2.13.16, 3.3.3]
    java: [21]

Every PR tests:

• 3 operating systems (Ubuntu, macOS, Windows)
• 2 Scala versions (2.13, 3.x)
• 1 Java version (21)

That’s 6 configurations tested automatically. If the template breaks on Windows? CI catches it before users hit it.

Pre-commit hooks included

The template includes .pre-commit-config.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

repos:
  - repo: https://github.com/scalameta/scalafmt
    rev: v3.7.14
    hooks:
      - id: scalafmt
        name: scalafmt
        entry: sbt scalafmtAll
        language: system
        types: [scala]

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: end-of-file-fixer
      - id: trailing-whitespace

Install with:

1
2

pip install pre-commit
pre-commit install

Now every commit automatically:

• Formats code with scalafmt
• Fixes trailing whitespace
• Ensures files end with newlines

Code quality enforced locally, before CI even runs.

Logging pre-configured

The template includes src/main/resources/logback.xml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <root level="INFO">
    <appender-ref ref="STDOUT" />
  </root>

  <!-- Set llm4s logging to DEBUG for development -->
  <logger name="org.llm4s" level="DEBUG" />
</configuration>

Logs work out of the box. Readable format. Sensible levels.

What happens after sbt new

Let’s walk through the actual user experience:

1. Generate the project

1
2
3
4
5
6
7

$ sbt new llm4s/llm4s.g8 \
    --name=my-ai-app \
    --package=com.example \
    --llm4s_version=0.1.1 \
    --scala_version=3.3.3

[info] Generated project: my-ai-app

2. Enter directory and check structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

$ cd my-ai-app
$ tree -L 2
my-ai-app/
├── build.sbt
├── src/
│   ├── main/
│   └── test/
├── .github/
├── README.md
└── .env.my-ai-app

3. Set API key

1

$ export OPENAI_API_KEY=sk-...

4. Run it

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

$ sbt run
[info] Compiling 2 Scala sources...
[info] running com.example.Main
Prompt: Explain what a Monad is in Scala in simple terms

09:15:32.123 [main] INFO  c.example.PromptExecutor - Starting LLM prompt execution
09:15:32.456 [main] INFO  c.example.PromptExecutor - LLM client initialized for provider: OpenAIClient
09:15:34.789 [main] INFO  c.example.PromptExecutor - Successfully received completion from LLM

Response:
A Monad in Scala is a design pattern that allows you to chain operations while handling context (like errors, options, or async operations) automatically. Think of it as a wrapper that lets you transform values inside without worrying about the wrapper itself.

[success] Total time: 3 s

60 seconds. From nothing to working LLM call.

Lessons learned

Try it yourself

llm4s.g8 is open source. You can see the full template at github.com/llm4s/llm4s.g8 .

Create your own project:

1
2
3
4
5
6
7
8
9

sbt new llm4s/llm4s.g8 \
  --name=my-llm-app \
  --package=com.mycompany \
  --llm4s_version=0.1.1 \
  --scala_version=3.3.3

cd my-llm-app
export OPENAI_API_KEY=sk-...
sbt run

60 seconds later, you’ll have a working LLM application.

What’s next

This template became the foundation for all work - both contributors and end-users in production. But we still had problems.

Remember that error hierarchy from the previous post? We used it in PromptExecutor.scala. But across the entire llm4s codebase, we still had 47 try-catch blocks swallowing errors.

Next post: How we eliminated every single try-catch block using functional error handling patterns. That’s where PR #260 comes in - the safety refactor that fixed a P1 streaming bug and simplified 57 files.

Series Navigation: ← Previous: Production error handling | Next: Error hierarchy refinement →

This is part 2 of a 6-part series on building type-safe LLM infrastructure in Scala. llm4s is maintained by Rory Graves and Kannupriya Kalra . Join the community at discord.gg/4uvTPn6qww .