# Building a transformer-based LLM with Effect
Building a transformer-based LLM from scratch is usually the domain of Python (PyTorch) or systems languages like Rust and C++. But Effect is surprisingly capable.
Repo link: [Effect GPT](https://github.com/erayack/effect-gpt)
In this implementation, I didn't just want it to work; I wanted it to be robust, testable, and disciplined. Also explored the usage of Effect in every part of the system. Effect became the backbone that let me write systems level implementation in TypeScript—managing side effects, concurrency, and errors with a level of precision that standard async/await just doesn't offer.
Here is how Effect powers the core of this project.
### Keeping the Core Pure
One of the biggest challenges in complex apps is keeping the "business logic" (in this case, tensor math and model architecture) free from the messiness of the outside world.
I used `Context.GenericTag` to define clear boundaries. The core model code never touches the file system or the console directly. Instead, it asks for services.
```ts
// src/services/Logger.ts
export const Logger = Context.GenericTag<LoggerServiceId, LoggerService>("LoggerService")
export const Metrics = Context.GenericTag<MetricsServiceId, MetricsService>("MetricsService")
export const Seed = Context.GenericTag<SeedServiceId, SeedService>("SeedService")
```
When the model needs to log progress or access random numbers, it uses these tags. The actual implementation—whether it's writing to a terminal via `@effect/platform` or using a mock for testing—is injected later. This keeps the math pure and the architecture clean.
### Dependency Injection without the Boilerplate
Instead of passing huge config objects around or relying on global singletons, the project uses Effect Layers. Layers let us compose our application's dependencies declaratively.
```ts
// src/training/train.ts
const llmLayer = Layer.succeed(LLMService, llm)
const configLayer = Layer.succeed(TrainingConfig, config)
const metricLayer = Layer.mergeAll(LoggerLayer, InMemoryMetricsLive, SeedLayerLive)
```
In the main application, we wire these up to the real production services. But in tests, we can swap them out effortlessly. For example, in `tests/ts/train_loop.test.ts`, I swap out the logger for a silent one and the metrics for a no-op version. This makes testing complex interactions simple and deterministic.
### Streaming Training Pipeline
Training a model involves processing massive amounts of data. Loading it all into memory isn't an option. Effect's `Stream` primitive is perfect for this. It lets us build a pipeline that reads, processes, and batches data lazily, with built-in backpressure.
```ts
// src/training/train.ts
const preprocessed = makeStream()
.pipe(
Stream.mapError(TrainingError.fromUnknown),
Stream.mapChunks(Chunk.chunksOf(batchSize)),
Stream.flattenChunks,
Stream.mapEffect(preprocess, { concurrency }),
Stream.filterMap((value) => value)
)
yield* Effect.scoped(
Stream.runDrain(
Stream.mapEffect(preprocessed, trainExample, { concurrency: trainConcurrency })
)
)
```
The magic here is `Stream.mapEffect`. It ties the heavy lifting (preprocessing and training) to the Effect runtime. This gives us typed error handling and configurable concurrency out of the box. If a file read fails or a tensor shape is wrong, the stream handles it gracefully.
### Errors You Can actually Type
TypeScript's `any` typed catch blocks are a pain. Effect treats errors as first-class values.
I defined a `TrainingError` union type that covers everything from bad user input to matrix multiplication mismatches.
```ts
// src/training/train.ts
const mapShapeError = <A, R>(effect: Effect.Effect<A, ShapeError, R>) =>
effect.pipe(Effect.mapError(TrainingError.shape))
```
Every part of the system—dataset loading, tensor math, CLI parsing—returns precise, typed errors. We don't throw exceptions; we return failure Effects. This means the compiler forces us to handle errors, or explicitly bubble them up. It makes the system incredibly stable.
### Automatic Resource Management
Leaking file handles is a classic bug. Effect solves this with `Scope`.
```ts
yield* Effect.scoped(
Stream.runDrain(
Stream.mapEffect(preprocessed, trainExample, { concurrency: trainConcurrency })
)
)
```
By wrapping the training loop in `Effect.scoped`, we ensure that everything opened during that scope—files, metrics trackers, fibers—is automatically closed when the scope ends, whether it finishes successfully, fails, or is interrupted by the user.
### Reproducible Randomness
Machine learning models need to be deterministic. If you run the same training job twice with the same seed, you should get the exact same weights.
I wrapped the random number generation in a `SeedLayer`.
```ts
// src/services/SeedLayer.ts
export const SeedLayer = (seed?: number): Layer.Layer<SeedServiceId> =>
Layer.succeed(Seed, makeSeedService(seed))
```
When a user provides a seed flag, we inject a deterministic generator. For tests, we use a fixed canonical seed. This allows us to verify the training loop end-to-end without flaky tests caused by random initialization.
### Fearless Concurrency
Finally, Effect makes concurrency approachable. In the training loop, we often want to preprocess data on multiple cores or track metrics without race conditions.
Effect's `Ref` provides atomic mutable state, and `Stream` allows us to control parallelism easily:
```ts
stream.mapEffect(..., { concurrency: 4 })
```
This simple configuration handles all the complex orchestration of fibers, letting us focus on the logic rather than the plumbing.
---
Google Drive
Gist
Clipboard
Sign in with Wallet
