AI + IoT Capstone Project

Every gesture,
understood in real time.

Developed by the SignSync team

SignSync is an Android app that recognizes sign language gestures through your phone camera and translates them into text instantly, built to help SPED classrooms communicate without barriers.

Download the app Try it in your browser
Sign → Text
Instant translation
Real-time
Gesture recognition
On-device
No camera data uploaded
DETECTING… SIGN · REAL-TIME
The Gap

Classrooms move fast. Communication shouldn't be what slows a student down.

In SPED classrooms, sign language is often the clearest way a student can express what they know but not every teacher, aide, or classmate can read it in real time.

01

Limited sign language fluency

Few teachers and support staff are fluent in sign language, creating a daily translation gap in instruction and assessment.

02

No real-time bridge

Existing tools are either too slow, too generic, or not built for a classroom's pace leaving gestures unread until it's too late to respond.

03

Not built for the classroom

Sign recognition research rarely accounts for shared devices, variable lighting, or the physical setup of an actual SPED room.

How It Works

From gesture to understanding, in four steps.

SignSync runs an AI hand-tracking pipeline right on the phone. A dedicated IoT capture unit is part of the project's classroom setup.

01

Capture

The camera captures hand movement and gesture data directly in the classroom.

→
02

Track

Hand landmarks are isolated in real time, filtering out noise from the room around them.

→
03

Recognize

The on-device AI model matches tracked hand movement against known gestures.

→
04

Translate

The result appears as text, readable by the teacher the moment it's signed.

Features

Built around a real classroom, not a lab demo.

Everything below is in the SignSync app today.

Real-time sign-to-text

Open the gesture scanner and see signs translated into readable text as they happen.

Learn & practice

A learning center with tutorials, how-to-sign guides, and a training mode with camera practice.

Translation history

Review past translations and track your learning progress over time.

Built-in AI assistant

An in-app chatbot supports learners and teachers with guidance while they use the app.

Private by design

Camera frames are processed locally on your device with MediaPipe and are never uploaded.

Emergency SOS

A quick-access SOS feature for urgent situations, so help is always one tap away.

Voice sync

Pair voice with text and signs to support conversations between Deaf and hearing users.

Accounts or guest mode

Sign in with email or Google, or continue as a guest to start translating right away.

SPED-classroom ready

Designed with input from an actual SPED setting, accounting for shared use, lighting, and pacing.

Sign Library

A growing library of letters, numbers, and words.

The in-app sign library includes the full A–Z fingerspelling alphabet, numbers 1–10, and a large set of everyday words, each with a how-to-sign guide. Reference images follow the ASL alphabet.

ASL sign for A
Letter A
ASL sign for B
Letter B
ASL sign for C
Letter C
Sign for 1
Number 1
Sign for Hello
Hello
Get the App

Install SignSync on your Android phone.

Free for educational use. Camera processing happens on your device.

SignSync for Android

Version 1.0 · Android 7.0+ · ~130 MB

Download APK
  1. Download the APK file to your phone.
  2. Open it. If asked, allow Install unknown apps for your browser or file manager.
  3. Tap Install, then open SignSync and allow camera access.

Only download SignSync from this page. Need help? Contact the team below.

Try It Live

The same recognition model, running right here.

This runs the same gesture recognizer file that ships inside the app, right in your browser. No install, no server, and nothing is uploaded. Use your camera or upload a photo of a hand sign.

Start your camera or upload a photo to begin
Model not loaded yet
Recognized Sign
—
Hand: — Confidence: —

TECHNOLOGY PREVIEW — THE FULL SIGN LIBRARY LIVES IN THE APP. THIS DEMO RECOGNIZES: OPEN_PALM → HELLO · THUMB_UP → LIKE · THUMB_DOWN → DISLIKE · CLOSED_FIST → FIST · VICTORY → PEACE

Where It Lives

Built for Urdaneta I Central School

SignSync is being developed for direct use in a SPED classroom setting designed alongside the realities of the students and teachers who will actually rely on it.

SPED CLASSROOM URDANETA CITY SIGN-TO-TEXT
Sign language in use

Built with the classroom in mind, not around it.

The Team

Capstone Project - BSIT, University of Eastern Pangasinan

JA
Adviser

Ms. Jessabel P. Alancado

Capstone Adviser

Guiding the research, design, and development direction of SignSync from proposal through defense.

  • Research direction & methodology
  • Technical review & feedback
  • Defense preparation
JB
PC
RB
Development Team

SignSync

Target Completion — December 2026

An AI and IoT-based Filipino Sign Language recognition system developed as a BSIT capstone project.

  • Bodkaen, Jan Allen — Developer - App
  • Bautista, Rainier — Developer - App
  • Casem, Prince Edrian — Developer - Website