OmniSign

AI-powered Lebanese Sign Language translation, built from scratch for a language that technology forgot.

ongoing · accessibility · computer vision · self-funded

50,000+Signs collectedRaw data from campaigns across Lebanon
80,000+Processed datasetAfter augmentation and cleaning
98%AccuracyInternal evaluation, controlled conditions
Self-fundedFundingNo institutional or grant backing

0:00

Live demo of OmniSign translating Lebanese Sign Language in real time

Public Choice Award, National FYP Demo Day 2025

OmniSign was selected by public vote as the winning project at Lebanon's National Final Year Project Demo Day 2025, competing against university teams from across the country.

Read the full article on RHU.edu.lb

It started with a joke nobody laughed at.

During COVID, I met a deaf man at a clinic. He wasn't struggling to be understood. The staff eventually figured out what he needed through context and gestures.

What I noticed was different. He was trying to tell a joke. You could see it in his face, his timing, the way he paused for a reaction that never came. Nobody understood him enough to laugh with him.

He had every right to share that moment with the people around him. That was when OmniSign became more than an idea.

Our university gave us the option to choose from faculty-approved projects. We chose to find our own challenge instead. It felt overwhelming. But it felt right.

We entered this project assuming data existed. It didn't. We assumed sign language was universal across all spoken languages. It wasn't.

Arabic sign language in Saudi Arabia is completely different from Lebanon. And Lebanon, in a way that surprised no one who knows the country, didn't have a unified sign language at all. Beqaa was different from the South. The South was different from Beirut. Even different sign language schools within Beirut signed differently from each other.

We spent three months traveling the country trying to find common ground. We couldn't. We were close to giving up. Then the team got stubborn. All of us, by nature.

“We decided this was the step we had to take ourselves, not wait for someone else to take it.”

A language with no map

BeqaaSouth LebanonBeirutSchool A, BeirutSchool B, Beirut

50,000 signs, built by hand

50k+
raw signs collected across campaigns in Lebanon
collection
80k+
processed image flows after preprocessing and augmentation
processed
30+
daily words and expressions plus Arabic alphabet support
vocabulary

We held data collection campaigns across Lebanon. Volunteers signed at our homes, family members we taught ourselves. We partnered with NGOs and deaf communities who opened their doors to us.

After processing, cleaning and augmenting, we ended up with over 80,000 image flows covering the full Arabic alphabet and around 30 words and expressions used in daily life. It was a messy process. If it wasn't for the team's chemistry, it never would have happened.

Community partners

Aaramoun Orphanage CenterData collection and community engagement
Sin El Fil ChurchData collection with deaf community members
SignWithNailaInstagram educator and leader in Lebanon's sign language community
Rafik Hariri UniversityBooth space at every campus event for data collection

What the data looks like

Every sign is recorded as hand and pose landmarks through MediaPipe, then stored as JSON per signer and per sign. The alphabet set below is one signer, one frame per letter, from the raw campaign images before augmentation.

Twenty-eight Arabic letters, each shown as a hand sign photographed against a white wall with the letter and its name beneath — The Arabic alphabet in Lebanese Sign Language, one signer, raw frames

Diagram of the pipeline: data loading from features per signer and sign, a TinyTCN model with two Conv1d layers and adaptive pooling, and training with 5-fold cross-validation — Pipeline from the 2sign-model repo: load landmarks, TinyTCN, 5-fold training

Input: MediaPipe Holistic landmarks per frame, with joint angles
Storage: features/{signer}/{sign}/*.json
Classes: Arabic alphabet plus 30+ daily expressions
Split: Stratified 5-fold cross-validation

“This isn't a final year project to them. This is a new life.”

Deaf individuals in Lebanon reach out at least once a month asking how they can help. They don't see a university assignment. They see the beginning of something that could change their daily life.

The community has been requesting clinical and medical phrases as a priority, the situations where miscommunication carries the highest cost.

Lebanon's sign language translators are rare and expensive. Many deaf people can't afford consistent access. Even among translators, regional differences mean understanding isn't guaranteed. OmniSign exists because this gap shouldn't depend on who you can afford to hire.

From gesture to language

OmniSign combines computer vision, neural language modeling and edge computing for real-time gesture-to-text-and-speech translation, with privacy as a structural constraint, not an afterthought.

01Real-time translation
Computer vision pipeline for live video calls. Instant recognition of sign language gestures with sub-frame latency.
02AI language model
Neural networks trained on diverse sign language datasets. 98% accuracy on internal evaluation. Real-world performance varies by conditions.
03Multi-modal output
Simultaneous text and speech. Real-time TTS generates natural spoken language from recognized signs.
04Learning platform
Interactive sign language lessons with progress tracking and personalized paths for ASL and LSL.
05Accessibility first
WCAG-compliant interface with haptic feedback, screen reader support and high-contrast modes.
06ASL and LSL support
Dedicated support for American and Lebanese Sign Language. LSL built from our own dataset, the first of its kind.
07Low latency
Under 100ms end to end using edge computing. Inference runs on-device where possible, with edge fallback.
08Privacy protected
Federated learning architecture. Sign data is processed on-device and never sent to central servers.

The code

Four repositories under the team organisation TwoSign on GitHub.

The team's repositories are private while the project is in development. Ask and I will share access.

The road ahead

Early 2025Started as FYP
Began as a Final Year Project at Rafik Hariri University. Six students chose to find their own challenge instead of picking from faculty-approved projects.
Mid 202550,000 signs collected
Campaigns across Lebanon with Aaramoun Orphanage, Sin El Fil Church and SignWithNaila. 80,000+ processed samples after augmentation.
Late 202598% accuracy achieved
LSTM model trained on the full Arabic alphabet plus 30+ daily expressions. Internal evaluation shows 98% in controlled conditions.
NowWeekly development
Fresh graduates with jobs, putting in weekly one-hour meetings plus coding sessions. Self-funded, community-supported, still building.
NextClinical vocabulary
Medical and clinical phrases, the most requested addition from the deaf community.
FutureScalable architecture
Researching Transformer-based models for better real-world performance and easier scaling to new sign languages.

Frequently asked

Currently Lebanese Sign Language (LSL) and American Sign Language (ASL). LSL support was built entirely from our own dataset, the first of its kind.

The people behind OmniSign

TLTayseer LazCo-founder · Product, Web & PR
Product direction, website, model integration into the mobile app, data collection, community communications, event coordination
LALayth AyacheAI & ML Lead
Computer vision, model training, data pipeline, dataset cleaning and augmentation, community outreach, event planning
ABAbu Baker Hussein El KhatibApp Developer
Flutter mobile app, web app, data collection at campus events and community booths
NENoor El HaririProject Coordinator
Flutter app development, academic documentation, project reports, research coordination
RKRami KronbiComputer Vision Engineer
ML engineering, CV pipeline, data collection, next-phase technical lead
OMDr. Oussama MustaphaResearch Advisor
Academic guidance, research methodology, project oversight