[1/7] Building Qal: A Tiny Language Model for Tigrinya
Building autocomplete for a low-resource language, from data to deployment
THE MOMENT
About two years ago, I sat down to write a few paragraphs in Tigrinya, in the script I grew up reading and writing.
I thought it would take no more than 30 minutes. It took more than an hour. I hovered over the keyboard, hesitated on characters I’d known since childhood, and typed as if I was learning a new language.
That’s when I realized: I hadn’t written Tigrinya script for years.
I didn’t notice because I “text in Tigrinya” daily, just with English letters.
“Kemey hadirka?” instead of “ከመይ ሓዲርካ፧”
It’s easier. Faster. It’s what everyone does. But sitting there, struggling through a few paragraphs, I realized the digital world had quietly pulled me away from my own language.
THE REAL PROBLEM
Tigrinya is spoken by millions of Eritreans and Ethiopians, but the digital tools around it are still catching up.
Researchers, linguists, developers, and community members have been pushing Tigrinya language technology forward for years through keyboards, translation systems, datasets, corpora, OCR work, spelling-checker experiments, and NLP research.
But compared to languages with mature digital ecosystems, everyday support is still limited, especially for longer writing.
For English, everyday language support tools are taken for granted:
Autocomplete finishes your thought before you do
Spell-check catches mistakes before you notice them
Search understands what you meant, not what you typed
Voice typing that never misses a beat
For Tigrinya, some of these tools exist in pieces, but they are still too limited or unreliable to build daily writing habits around.
THE CASCADE EFFECT
When tools are limited, writing becomes effortful and is avoided.
When people don’t write it often, they read it less often.
When reading and writing aren’t daily habits, fluency declines. Characters take longer to recognize. Spelling feels uncertain. Writing slows down. Words that once felt natural require effort or a lookup.
It might not happen overnight. It’s slow erosion.
“ካብ ዓይኒ ዝገለለ ፡ ካብ ልቢ ገለለ ።”
(Out of sight, out of mind.)
— Eritrean Proverb
WHY THIS MATTERS BEYOND CONVENIENCE
About a year ago, I was visiting my friend’s aunt, who told me she was writing a book in Tigrinya. I asked her how long it usually took to write one page.
“All day,” she said.
Her answer sat with me.
Not because she didn’t know what she wanted to say, but long-form writing still asks a lot from the writer. Characters often take multiple keystrokes, words take longer to form, and small mistakes are easy to miss when there is limited support around the writing process.
Autocomplete is limited. Spell-check is minimal. Consistency is mostly manual.
She catches every typo herself.
And she was doing it anyway.
She was not alone either. Another Tigrinya author I spoke with told me a similar story. Even after writing a few Tigrinya books, he still spends days catching spelling errors, fixing grammar, and checking consistency.
Better tools help writers move faster. Better tools help families, learners, authors, and communities keep the language present in daily digital life.
This is about preservation, continuity, and connection across generations.
That is not only about convenience.
WHY I’M WORKING THIS
Someone needs to keep building these tools, and big tech companies won’t prioritize them. Low-resource languages aren’t profitable enough to justify deep investment.
So I’m adding my piece to the puzzle.
Qal builds on the foundation others have created, adding one more piece to the larger effort and vision.
There’s also another reason I am working on this project.
A few months ago, I read Sebastian Raschka’s “Build a Large Language Model from Scratch.” Excellent book. It gave me a strong foundation in how language models work: data preparation, tokenization, embeddings, attention, transformer blocks, and training loops.
But I wanted to turn those concepts into decisions.
That’s why I built something real. Something close to my heart.
Project Qal gave me both: a technical challenge with real constraints and a problem that actually matters to me.
This project forced questions that books don’t answer:
Data: Where do you find data for a low-resource language?
Tokenization: How do you split Tigrinya text into useful tokens?
Architecture: How small can a model be before it stops learning useful patterns?
Training: How do you train a useful model when the dataset is small?
Inference: How do you make autocomplete “fast enough” to feel useful?
Every decision had context. Every shortcut had a cost. And the learning deepens with each choice.
"I hear and I forget. I see and I remember. I do and I understand."
— Confucius
WHAT I’M ACTUALLY BUILDING
Qal is a Tigrinya-first language model I’m building from scratch. Its first concrete goal is a real-time Tigrinya autocomplete system.
Here’s a quick demo of what I have so far:
Try the demo here: Qal autocomplete demo 
I’m starting with autocomplete because it targets the pain I felt directly: slow, hesitant typing.
A useful suggestion at the right moment can keep someone writing instead of stopping, deleting, retyping, or switching back to English letters.
It’s a small intervention, but it compounds:
Better tools → More writing → More familiarity → More presence in daily digital life.
Autocomplete is only the start. The same foundation can later support:
Autocorrect
Spell-check
Question-answering
BUILDING FROM SCRATCH
I’m building from scratch because I want to put Tigrinya first, not an afterthought. Every decision, tokenization strategy, vocabulary size, and model architecture centers on Tigrinya.
But I’m flexible. If fine-tuning a multilingual model proves more effective, I’ll use that instead. The goal is to create a tool that works, not to prove an architectural point.
With limited data, every decision matters. Building from scratch gives me a closer look at those trade-offs and a better understanding of what low-resource language modeling really requires.
THE FOUNDATION
Here’s what I’m working with:
Data: ~70 million Tigrinya tokens from newspapers, books, and the bible
Tokenization: Custom Byte-Pair Encoding (BPE) tokenizer for Tigrinya text
Architecture: Custom GPT-2 style transformer built from scratch
Training: Training setup tuned for low-resource constraints
Goal: Accurate, real-time Tigrinya autocomplete
WHAT’S COMING NEXT
This series is for anyone interested in low-resource language models, or in training and shipping small language models end-to-end.
I’m documenting the full process: what worked, what failed, and which decisions mattered.
Here's the full breakdown:
[1/7] Building Qal: A Tiny Language Model for Tigrinya ← You are here
[2/7] Data Collection - Finding and cleaning 70M tokens for a low-resource language
[3/7] Tokenization - Building a BPE tokenizer from scratch and vocabulary decisions
[4/7] Model Architecture - Design choices and trade-offs with limited data
[5/7] Training & Results - What worked, what failed, and which hyperparameters mattered
[6/7] Inference - Making autocomplete fast enough to feel useful
[7/7] Lessons Learned - Reflections on building language infrastructure for low-resource languages
NEXT UP
How I collected and cleaned 70M tokens of Tigrinya text, and why every token matters when data is scarce.
Building Qal: The Journey■ □ □ □ □ □ □
Why Qal → Data → Tokenization → Architecture → Training → Inference → Lessons
A Note on Open Source
Qal is not meant to be perfect.
It is meant to be a starting point: a practical tool for everyday Tigrinya writing and a learning resource for me and anyone building language models for low-resource languages.
I’m open-sourcing the code, model, and data where possible so others can study it, improve it, or build something better.
This is an individual effort, but it is part of a collective one.
To everyone who has been pushing Tigrinya forward, this project builds on your work.
Thank you.
RESOURCES:
🐙 GitHub: Code and project updates
📩 Substack: Follow 'Building Qal' series
💼 LinkedIn: Connect with me
🌐 Website: More about me and what I'm building