Sinhala deserves real evaluation
Most language models are measured almost entirely in English. This benchmark tests genuine knowledge and reasoning in native Sinhala, with no translation.
How much do compact models really understand සිංහල?
Build open-weight systems of 8 billion parameters or fewer that answer Sinhala multiple-choice questions across Easy, Medium and Hard levels.
Next Up: -
00days
00hours
00min
00sec
why this task
Small enough to run locally
Every component at inference counts towards an 8B budget; models, retrievers, rerankers, embeddings. Efficient ideas win.
Open and verifiable
Open-weight models only, no closed APIs or internet at test time. Every ranked result is reproduced.
how it works
- Register your team (1-4 members) with one corresponding member.
- Develop on the dev set with the public baseline.
- Submit test predictions on the Codabench leaderboard.
- Share your system & code to finalize your submission.
- Present your work at ICTer 2026.