Someone wrote "asistant" instead of "assistant".

One ’s’. That’s all it took.

The code compiled. The tests passed. It deployed to production. Then it failed with a cryptic error: “Invalid role in conversation.”

We spent 20 minutes debugging. The logs showed the API rejected the request. Why? Because we sent role: "asistant" and OpenAI expects role: "assistant".

A typo. That’s it. A typo the compiler could’ve caught.

Here’s what’s frustrating: This wasn’t even complex logic. No algorithms. No concurrency. No edge cases. Just a string literal. One character wrong. And it got past:

• The IDE (no red squiggle)
• The compiler (no error)
• The type checker (strings are strings)
• The unit tests (they used correct strings)
• The integration tests (same)
• Code review (humans miss typos)

It failed in production. In front of users. Because we used strings where we should’ve used enums.

Rory Graves said: “We’re using strings for enums. This is asking for trouble.”

He was right. So we fixed it. PR #216 replaced string-based message roles with a proper enum, added 6 type classes for functional programming patterns, and created a 191-line migration guide.

This is the story of how we made message role typos impossible and added type-safe functional abstractions that made llm4s work with any effect type (IO, Task, Future, whatever).

Review PR #216 on GitHub

Explore the llm4s repository

Why strings for enums are dangerous

Let’s look at what we had. Message roles in LLM conversations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

// Before: String-based roles
val userMessage = Message(
  role = "user",      // String literal
  content = "Hello"
)

val assistantMessage = Message(
  role = "assistant",  // String literal
  content = "Hi there!"
)

val systemMessage = Message(
  role = "system",     // String literal
  content = "You are a helpful assistant"
)

The problem: Nothing stops you from writing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

val broken = Message(
  role = "asistant",  // Typo! But compiles fine
  content = "Hello"
)

val worse = Message(
  role = "ASSISTANT",  // Wrong case! Also compiles
  content = "Hello"
)

val terrible = Message(
  role = "assistant ",  // Trailing space! Still compiles
  content = "Hello"
)

All three compile. All three fail at runtime. Usually in production.

Pattern matching was a nightmare:

1
2
3
4
5
6
7

message.role match {
  case "user" => handleUser(message)
  case "assistant" => handleAssistant(message)
  case "system" => handleSystem(message)
  // Forgot "tool"? Compiler won't tell you
  // Typo in any case? Compiler won't tell you
}

Non-exhaustive pattern matching. Silent failures. Runtime bugs.

IDE support was non-existent:

Type message.role = "a" and hit tab. Your IDE shows… nothing. Because it’s a string. The IDE has no idea what valid values are.

The string-based problem visualized:

flowchart TD
    A["Write code: role = 'asistant'"] --> B{"Compiler check"}
    B -->|"String type ✓"| C["Compiles successfully"]
    C --> D["Run tests"]
    D -->|"Tests pass"| E["Deploy to production"]
    E --> F["LLM API call"]
    F --> G["API rejects: 'Invalid role'"]
    G --> H["Production failure 💥"]

    style B fill:#c8e6c9,stroke:#2d7a2d,color:#000
    style G fill:#ffcdd2,stroke:#cc0000,color:#000
    style H fill:#ffcdd2,stroke:#cc0000,color:#000

This is unacceptable. Typos should fail at compile time, not in production.

The MessageRole enum

What’s an enum? An enumeration (enum) is a type that has a fixed set of named constants. In languages like Java, you’d write enum MessageRole { USER, ASSISTANT, SYSTEM, TOOL }. In Scala, we use sealed traits for the same purpose but with more power: pattern matching exhaustiveness checking, type safety, and the ability to attach behavior.

Why sealed traits? The sealed keyword means all subtypes must be defined in the same file. This lets the compiler know the complete set of possible cases, enabling exhaustive pattern matching checks. If you forget a case, the compiler warns you.

Here’s what we built (from src/main/scala/org/llm4s/llmconnect/model/Message.scala):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// The enum
sealed trait MessageRole

object MessageRole {
  case object User extends MessageRole
  case object Assistant extends MessageRole
  case object System extends MessageRole
  case object Tool extends MessageRole

  // Helper to convert to string (for API calls)
  def toString(role: MessageRole): String = role match {
    case User => "user"
    case Assistant => "assistant"
    case System => "system"
    case Tool => "tool"
  }

  // Helper to parse from string (for API responses)
  def fromString(s: String): Either[String, MessageRole] = s.toLowerCase.trim match {
    case "user" => Right(User)
    case "assistant" => Right(Assistant)
    case "system" => Right(System)
    case "tool" => Right(Tool)
    case unknown => Left(s"Unknown message role: $unknown")
  }
}

Now try that typo:

1
2
3
4

val message = Message(
  role = MessageRole.Asistant,  // Compile error: value Asistant is not a member of object MessageRole
  content = "Hello"
)

Compile time. Not runtime. The error shows up in your editor before you even run the code.

Pattern matching is exhaustive:

1
2
3
4
5
6
7

message.role match {
  case MessageRole.User => handleUser(message)
  case MessageRole.Assistant => handleAssistant(message)
  case MessageRole.System => handleSystem(message)
  // Forgot MessageRole.Tool? Compiler warns you:
  // "match may not be exhaustive. It would fail on: Tool"
}

The compiler enforces correctness. Miss a case? Immediate warning.

IDE support is magical:

Type MessageRole. and hit Ctrl+Space. Your IDE shows:

• User
• Assistant
• System
• Tool

Autocomplete works. You can’t typo. You can’t forget a case.

The Message trait hierarchy

Why a trait hierarchy? The MessageRole enum solved the typo problem. But there was a deeper issue: different message roles have different fields. User messages don’t have tool calls. Assistant messages do. Tool messages require a toolCallId. System messages are just content.

The old single case class let you create structurally nonsensical messages. The type system wasn’t helping. We needed types that matched the domain.

We refactored the entire message system (251 lines of changes in PR #216 ):

Before (everything was one case class):

1
2
3
4
5
6

case class Message(
  role: String,           // Stringly-typed
  content: String,
  toolCalls: Option[Seq[ToolCall]] = None,
  toolCallId: Option[String] = None
)

Problems:

• toolCalls on user messages? Nonsense but allowed
• toolCallId on system messages? Also nonsense, also allowed
• No type safety for what fields each role should have

After (trait hierarchy with type safety):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

sealed trait Message {
  def role: MessageRole
  def content: String
}

final case class UserMessage(
  content: String
) extends Message {
  def role: MessageRole = MessageRole.User
}

final case class AssistantMessage(
  content: String,
  toolCalls: Seq[ToolCall] = Seq.empty  // Only assistants have tool calls
) extends Message {
  def role: MessageRole = MessageRole.Assistant
}

final case class SystemMessage(
  content: String
) extends Message {
  def role: MessageRole = MessageRole.System
}

final case class ToolMessage(
  toolCallId: String,        // Required for tool messages
  content: String
) extends Message {
  def role: MessageRole = MessageRole.Tool
}

Now the types enforce correct usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

// Valid
val user = UserMessage("Hello")
val assistant = AssistantMessage("Hi!", toolCalls = Seq(...))
val tool = ToolMessage(toolCallId = "call_123", content = "Result")

// Invalid - won't compile
val broken = UserMessage(
  content = "Hello",
  toolCalls = Seq(...)  // Compile error: UserMessage has no toolCalls parameter
)

val alsobroken = SystemMessage(
  content = "You are helpful",
  toolCallId = "123"  // Compile error: SystemMessage has no toolCallId parameter
)

The type system prevents nonsense. You can’t create structurally invalid messages.

Type classes for functional abstraction

What are type classes? A type class is a pattern from functional programming that lets you add functionality to types without modifying their source code. Think of it as “interfaces on steroids”: you can make existing types (even ones you don’t control) work with your abstractions by providing instances.

Why do they matter? Type classes enable effect polymorphism: writing code once that works with different effect types (IO, Task, Future, ZIO, Cats Effect, etc.) without modification. This is huge for llm4s because users have different preferences for effect systems.

PR #216 added 6 type classes (191 lines total). These enable functional programming patterns and effect polymorphism.

1. LLMCapable[F[_]] - Core operations for effect types

What’s F[_]? This is Scala notation for a “type constructor” or “higher-kinded type”: a type that takes another type as a parameter. For example, List[Int] where List is F and Int is the wrapped type. In effect systems: IO[String], Task[User], Future[Response]. The F[_] lets us write code that works with any such wrapper.

1
2
3
4
5
6
7

trait LLMCapable[F[_]] {
  def pure[A](value: A): F[A]
  def map[A, B](fa: F[A])(f: A => B): F[B]
  def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B]
  def handleError[A](fa: F[A])(f: LLMError => A): F[A]
  def raiseError[A](error: LLMError): F[A]
}

This lets you write code that works with any effect type:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

def fetchCompletion[F[_]: LLMCapable](prompt: String): F[String] = {
  val F = implicitly[LLMCapable[F]]

  F.flatMap(loadConfig()) { config =>
    F.flatMap(createClient(config)) { client =>
      F.map(client.complete(prompt)) { completion =>
        completion.message.content
      }
    }
  }
}

// Works with IO
val ioResult: IO[String] = fetchCompletion[IO]("Hello")

// Works with Task
val taskResult: Task[String] = fetchCompletion[Task]("Hello")

// Works with Future
val futureResult: Future[String] = fetchCompletion[Future]("Hello")

Same code. Different effect types. No duplication.

2. Show[A] - Type-safe string conversion

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

trait Show[A] {
  def show(value: A): String
}

implicit val showMessageRole: Show[MessageRole] = new Show[MessageRole] {
  def show(role: MessageRole): String = MessageRole.toString(role)
}

implicit val showMessage: Show[Message] = new Show[Message] {
  def show(msg: Message): String = {
    s"${Show[MessageRole].show(msg.role)}: ${msg.content}"
  }
}

Usage:

1
2
3

val message = UserMessage("Hello")
println(Show[Message].show(message))
// Output: "user: Hello"

Better than toString because it’s explicit and type-safe.

3. Validate[A] - Type-safe validation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

trait Validate[A] {
  def validate(value: A): Either[LLMError, A]
}

implicit val validateMessage: Validate[Message] = new Validate[Message] {
  def validate(msg: Message): Either[LLMError, Message] = {
    if (msg.content.isEmpty) {
      Left(ValidationError(
        message = "Message content cannot be empty",
        input = msg.content,
        cause = None
      ))
    } else {
      Right(msg)
    }
  }
}

Now validation is composable:

1
2
3
4
5
6

def processMessage[F[_]: LLMCapable](msg: Message): F[Unit] = {
  Validate[Message].validate(msg) match {
    case Right(validMsg) => doSomething(validMsg)
    case Left(error) => LLMCapable[F].raiseError(error)
  }
}

4. Encoder[A] - Type-safe encoding for APIs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

trait Encoder[A] {
  def encode(value: A): Json
}

implicit val encodeMessageRole: Encoder[MessageRole] = new Encoder[MessageRole] {
  def encode(role: MessageRole): Json = Json.fromString(MessageRole.toString(role))
}

implicit val encodeMessage: Encoder[Message] = new Encoder[Message] {
  def encode(msg: Message): Json = msg match {
    case UserMessage(content) =>
      Json.obj("role" -> encode(MessageRole.User), "content" -> Json.fromString(content))
    case AssistantMessage(content, toolCalls) =>
      Json.obj(
        "role" -> encode(MessageRole.Assistant),
        "content" -> Json.fromString(content),
        "tool_calls" -> Json.fromValues(toolCalls.map(encodeToolCall))
      )
    case SystemMessage(content) =>
      Json.obj("role" -> encode(MessageRole.System), "content" -> Json.fromString(content))
    case ToolMessage(toolCallId, content) =>
      Json.obj(
        "role" -> encode(MessageRole.Tool),
        "tool_call_id" -> Json.fromString(toolCallId),
        "content" -> Json.fromString(content)
      )
  }
}

5. LLMFunctor[F[_]] and 6. LLMMonad[F[_]] - Standard functional patterns

These follow standard functor and monad laws, adapted for LLM contexts.

Atul S. Khot reviewed these and suggested better Cats integration. We made them compatible with existing Cats type classes.

The migration challenge

Here’s the thing: llm4s had users. Production code. We couldn’t just break everything.

What we needed:

1. Introduce MessageRole enum
2. Keep old string-based code working
3. Guide users to migrate
4. Provide clear timeline

The solution: MIGRATION_GUIDE.md (191 lines)

Phase 1: Deprecate old API

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// Old Message constructor (deprecated but working)
object Message {
  @deprecated("Use UserMessage, AssistantMessage, etc.", "0.3.0")
  def apply(role: String, content: String): Message = {
    role.toLowerCase match {
      case "user" => UserMessage(content)
      case "assistant" => AssistantMessage(content)
      case "system" => SystemMessage(content)
      case "tool" => throw new IllegalArgumentException("Tool messages require toolCallId")
      case unknown => throw new IllegalArgumentException(s"Unknown role: $unknown")
    }
  }
}

Old code keeps working. But you get deprecation warnings pointing you to the migration guide.

How the bridge method works:

flowchart LR
    A["Old code: Message('user', 'Hello')"] --> B["Bridge method<br/>@deprecated apply()"]
    B --> C{"Parse string"}
    C -->|"'user'"| D["UserMessage('Hello')"]
    C -->|"'assistant'"| E["AssistantMessage('Hello')"]
    C -->|"'system'"| F["SystemMessage('Hello')"]
    C -->|"'tool'"| G["Throw exception<br/>(needs toolCallId)"]
    C -->|"unknown"| H["Throw exception<br/>(invalid role)"]

    style B fill:#fff4e6,stroke:#cc8800,color:#000
    style D fill:#c8e6c9,stroke:#2d7a2d,color:#000
    style E fill:#c8e6c9,stroke:#2d7a2d,color:#000
    style F fill:#c8e6c9,stroke:#2d7a2d,color:#000
    style G fill:#ffcdd2,stroke:#cc0000,color:#000
    style H fill:#ffcdd2,stroke:#cc0000,color:#000

This pattern enabled zero-downtime migration: old code keeps working, new code uses the enum directly, and the bridge method translates between them.

Phase 2: Provide migration patterns

Why side-by-side examples matter: When refactoring code, developers need to see both “before” and “after” simultaneously. This reduces cognitive load and makes migration mechanical rather than creative work. You’re not figuring out how to migrate; you’re following a pattern.

The migration guide has 15+ side-by-side examples covering every usage pattern in llm4s :

Example 1: Simple message creation

1
2
3
4
5

// Before
val msg = Message(role = "user", content = "Hello")

// After
val msg = UserMessage(content = "Hello")

Example 2: Pattern matching

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

// Before
message.role match {
  case "user" => handleUser(message)
  case "assistant" => handleAssistant(message)
  case "system" => handleSystem(message)
}

// After
message match {
  case UserMessage(content) => handleUser(content)
  case AssistantMessage(content, toolCalls) => handleAssistant(content, toolCalls)
  case SystemMessage(content) => handleSystem(content)
  case ToolMessage(toolCallId, content) => handleTool(toolCallId, content)
}

Example 3: Conversation building

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// Before
val conversation = Conversation(messages = Seq(
  Message("system", "You are helpful"),
  Message("user", "Hello"),
  Message("assistant", "Hi there!")
))

// After
val conversation = Conversation(messages = Seq(
  SystemMessage("You are helpful"),
  UserMessage("Hello"),
  AssistantMessage("Hi there!")
))

Example 4: Tool calling

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// Before (unsafe - nothing stops you from adding toolCalls to user message)
val msg = Message(
  role = "user",
  content = "Hello",
  toolCalls = Some(Seq(...))  // This is nonsense but allowed
)

// After (type safe - UserMessage doesn't have toolCalls)
val msg = UserMessage(content = "Hello")
// toolCalls parameter doesn't exist, won't compile

Phase 3: Timeline and removal plan

The guide included:

Version 0.3.0 (current):

• MessageRole enum introduced
• Old string-based API deprecated
• Migration guide published
• Both APIs work side-by-side

Version 0.4.0 (3 months later):

• Deprecation warnings escalated
• Examples updated to new API
• Documentation only shows new API

Version 0.5.0 (6 months later):

• Old string-based API removed
• Only MessageRole enum supported
• Clean codebase

This gave users 6 months to migrate. Plenty of time. No surprises.

The migration in practice

We migrated 43 files in the llm4s codebase. Every usage of string-based roles needed updating. This included samples (user-facing examples), tests (internal correctness), providers (API integrations), and core infrastructure (streaming, agents, state management).

The strategy: Work layer by layer from the outside in. Start with samples (they break loudly), then tests (they verify correctness), then providers (they interface with external APIs), finally core infrastructure (it affects everything).

Here’s what changed:

Samples updated (10 files):

1
2
3
4
5
6

// samples/src/main/scala/org/llm4s/samples/agent/MCPAgentExample.scala
// Before
val msg = Message("user", "What's the weather?")

// After
val msg = UserMessage("What's the weather?")

Tests updated (10 files):

1
2
3
4
5
6

// src/test/scala/.../StreamingAccumulatorTest.scala
// Before
message.role shouldBe "assistant"

// After
message.role shouldBe MessageRole.Assistant

Providers updated (8 files):

1
2
3
4
5
6
7

// src/main/scala/org/llm4s/llmconnect/provider/OpenAIClient.scala
// Before
val role = responseJson.role.asString.getOrElse("assistant")

// After
val role = MessageRole.fromString(responseJson.role.asString.getOrElse("assistant"))
  .getOrElse(MessageRole.Assistant)

Streaming handlers updated (4 files):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

// src/main/scala/org/llm4s/llmconnect/streaming/StreamingAccumulator.scala
// Before
if (delta.role.isDefined) {
  currentRole = delta.role.get
}

// After
if (delta.role.isDefined) {
  currentRole = MessageRole.fromString(delta.role.get)
    .fold(_ => MessageRole.Assistant, identity)
}

Agent system updated (5 files):

• Agent.scala
• AgentState.scala
• ToolRegistry.scala

Total: 43 files, ~280 lines changed, 0 breaking changes to external API.

ClientStatus for health monitoring

Why health monitoring matters: In production, LLM API calls fail. Rate limits hit. Networks timeout. Providers go down. You need observability to know when a client is unhealthy before users complain. The ClientStatus type tracks this.

What makes a client “healthy”? Three criteria:

1. Currently connected (last connection attempt succeeded)
2. Low error count (< 10 consecutive failures)
3. Recent success (successfully called within last 5 minutes)

PR #216 also added ClientStatus.scala (39 lines):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

final case class ClientStatus(
  provider: String,
  connected: Boolean,
  lastSuccess: Option[Instant],
  lastError: Option[LLMError],
  errorCount: Long,
  successCount: Long
) {
  def successRate: Double = {
    val total = errorCount + successCount
    if (total == 0) 0.0 else successCount.toDouble / total.toDouble
  }

  def isHealthy: Boolean = {
    connected && errorCount < 10 && (
      lastSuccess.isEmpty ||
      lastSuccess.get.isAfter(Instant.now().minus(5, ChronoUnit.MINUTES))
    )
  }
}

Now you can monitor LLM client health:

1
2
3
4
5
6

val status = client.getStatus()

if (!status.isHealthy) {
  logger.warn(s"Client ${status.provider} is unhealthy: success rate = ${status.successRate}")
  status.lastError.foreach(e => logger.error(s"Last error: ${e.formatted}"))
}

This became the foundation for observability. Shubham’s tracing work integrated with this.

Syntax helpers for cleaner code

Why syntax helpers? After adding MessageRole and the type classes, we noticed common patterns appearing everywhere: checking if a message is from the user, extracting the last user message from a conversation, handling Result types safely. These patterns deserved first-class support.

Syntax helpers (also called “ops” or “syntax enrichment”) use Scala’s implicit classes to add methods to existing types. You import the syntax, and suddenly your types have new capabilities.

Added src/main/scala/org/llm4s/syntax/syntax.scala (32 lines) to llm4s :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

object syntax {
  implicit class ResultOps[A](private val result: Result[A]) extends AnyVal {
    def getOrThrow: A = result match {
      case Right(value) => value
      case Left(error) => throw new RuntimeException(error.formatted)
    }

    def getOrElse(default: => A): A = result match {
      case Right(value) => value
      case Left(_) => default
    }

    def orElse(fallback: => Result[A]): Result[A] = result match {
      case Right(_) => result
      case Left(_) => fallback
    }
  }

  implicit class MessageOps(private val msg: Message) extends AnyVal {
    def isUser: Boolean = msg.role == MessageRole.User
    def isAssistant: Boolean = msg.role == MessageRole.Assistant
    def isSystem: Boolean = msg.role == MessageRole.System
    def isTool: Boolean = msg.role == MessageRole.Tool
  }

  implicit class ConversationOps(private val conv: Conversation) extends AnyVal {
    def lastMessage: Option[Message] = conv.messages.lastOption
    def lastUserMessage: Option[UserMessage] =
      conv.messages.collectFirst { case m: UserMessage => m }
  }
}

Usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import org.llm4s.syntax.syntax._

// Clean chaining
val content: String = client.complete(prompt)
  .map(_.message.content)
  .getOrElse("No response")

// Type-safe message checks
if (message.isAssistant && message.asInstanceOf[AssistantMessage].toolCalls.nonEmpty) {
  handleToolCalls(message)
}

The numbers

Here’s what we shipped in PR #216 :

Co-authors

Migration success metrics:

• 6 contributors migrated code across 43 files
• 0 complaints, 0 issues during migration
• Migration time: Average 15 minutes per contributor
• Reason for success: 191-line migration guide with 15+ side-by-side examples

The bridge method plus the migration guide made adoption painless. Every contributor finished the switch on the first try.

Effect polymorphism in action

The type classes enable real effect polymorphism. Here’s actual code from llm4s:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

def executeWithRetry[F[_]: LLMCapable](
  operation: => F[Completion],
  maxRetries: Int = 3
): F[Completion] = {
  val F = implicitly[LLMCapable[F]]

  def attempt(n: Int): F[Completion] = {
    F.flatMap(operation) { completion =>
      F.pure(completion)
    }.handleError { error =>
      if (n > 0 && error.isInstanceOf[RecoverableError]) {
        logger.info(s"Recoverable error, retrying ($n attempts left)")
        attempt(n - 1)
      } else {
        F.raiseError(error)
      }
    }
  }

  attempt(maxRetries)
}

This works with any effect type that has an LLMCapable instance:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// With Cats-Effect IO
implicit val ioCapable: LLMCapable[IO] = new LLMCapable[IO] {
  // Implementation using IO primitives
}

val ioResult: IO[Completion] = executeWithRetry[IO](client.complete(prompt))

// With ZIO Task
implicit val taskCapable: LLMCapable[Task] = new LLMCapable[Task] {
  // Implementation using ZIO primitives
}

val taskResult: Task[Completion] = executeWithRetry[Task](client.complete(prompt))

Same retry logic. Different effect systems. No code duplication.

Try it yourself

The code is in llm4s on GitHub .

Key files:

• PR #216 : The full type system upgrade
• MessageRole enum: src/main/scala/org/llm4s/llmconnect/model/Message.scala
• Type classes: src/main/scala/org/llm4s/types/typeclass/
• Migration guide: MIGRATION_GUIDE.md (191 lines with 15+ examples)

Want to use llm4s with type-safe message roles?

1
2
3
4
5
6
7

# Use the llm4s.g8 template to generate a project
sbt new llm4s/llm4s.g8
# Answer prompts: project name, package, versions

cd my-llm-app
export OPENAI_API_KEY=sk-...
sbt run

MessageRole enum is already integrated. You’ll never typo a role again.

What you get:

• Compile-time checking for message roles (no typos reach production)
• IDE autocomplete for all valid roles (type MessageRole. and see options)
• Exhaustive pattern matching (compiler warns if you forget a case)
• Type-safe message constructors (UserMessage, AssistantMessage, etc.)
• Effect polymorphism via type classes (works with IO, Task, Future, ZIO, Cats Effect)

Production lessons: What this pattern teaches

This migration pattern works beyond llm4s . Here’s what we learned about migrating production systems without breaking users:

1. Deprecate, don’t delete

The @deprecated annotation lets old code keep working while guiding users toward the new API. This is dramatically better than breaking changes with major version bumps. Users migrate when they’re ready, not when you force them.

Pattern:

1
2
3
4

@deprecated("Use UserMessage, AssistantMessage, etc.", "0.3.0")
def apply(role: String, content: String): Message = {
  // Bridge to new API
}

Why it works: Compiler shows the deprecation warning with the migration message. IDEs highlight the deprecated code. But the code still compiles and runs. Users can migrate file-by-file, commit-by-commit.

2. Bridge methods enable zero-downtime

The deprecated Message.apply bridges old string-based calls to the new enum-based types. This pattern lets you completely refactor internals while maintaining API compatibility.

Real impact: All llm4s users upgraded to 0.3.0 without code changes. No broken builds. No emergency fixes. They migrated on their own schedule.

3. Migration guides prevent support burden

The 191-line MIGRATION_GUIDE.md answered every question before users asked it. Result: 6 contributors migrated 43 files with zero issues, zero questions, zero support tickets.

What made it effective:

• Side-by-side examples showing before/after for every pattern
• Search-friendly headings like “How do I create a user message?”
• Progressive difficulty from simple to complex migrations
• Rationale for each change explaining why the new way is better

4. Type classes future-proof your abstractions

The 6 type classes (LLMCapable, Show, Validate, Encoder, LLMFunctor, LLMMonad ) made llm4s effect-polymorphic. This decision paid off when users needed ZIO integration, Cats Effect support, and custom effect types.

Lesson: Invest in abstractions early. Type classes let you support new effect systems without changing core code. The alternative (hard-coding to one effect type) creates technical debt that’s expensive to fix later.

5. Exhaustive pattern matching catches bugs at compile time

After MessageRole shipped, zero message role bugs reached production. Not “fewer bugs.” Zero. The compiler enforced correctness.

Before (string-based):

1
2
3
4
5
6

// Compiles, fails at runtime
message.role match {
  case "user" => handleUser(message)
  case "assistant" => handleAssistant(message)
  // Forgot "system" and "tool" - compiler doesn't warn
}

After (enum-based):

1
2
3
4
5
6

// Compiler warns: "match may not be exhaustive"
message.role match {
  case MessageRole.User => handleUser(message)
  case MessageRole.Assistant => handleAssistant(message)
  // Compiler: "It would fail on: System, Tool"
}

This is the power of algebraic data types (ADTs). The compiler knows all possible cases and enforces that you handle them.

6. Measure migration success by complaints, not lines changed

We changed 1,068 lines across 43 files. But the real metric? Zero user complaints. Zero broken builds. Zero regression bugs. That’s successful migration.

How we achieved it:

• Backward compatible bridge methods (old code works)
• Comprehensive migration guide (new code is easy)
• Deprecation timeline (users have 6 months to migrate)
• Internal dogfooding (we migrated first, hit all the edge cases)

Production impact after 3 months:

• Message role bugs: 0 (down from 2-3 per month with strings)
• Pattern matching warnings: 47 fixed (compiler caught missing cases)
• Migration support requests: 0 (guide answered everything)
• Rollbacks due to breaking changes: 0 (backward compatible)

What’s next

We had structured errors. We had smart constructors. We had type-safe message roles. But we still had 47 try-catch blocks scattered across the codebase.

Next post: The safety refactor (PR #260 ). Where we eliminated every try-catch block, fixed a P1 streaming bug, and unified error handling across 8 LLM providers. That’s where everything came together.

Series Navigation: ← Previous: Error hierarchy refinement | Next: Safety refactor →

This is part 4 of a 6-part series on building type-safe LLM infrastructure in Scala. llm4s is maintained by Rory Graves and Kannupriya Kalra . Join the community at discord.gg/4uvTPn6qww .