Access
KOKO is learning sign language, so a home robot can talk with Deaf people in their own language.
>
Most home technology assumes you can hear it. Voice assistants talk, alarms beep, robots answer out loud. For Deaf people whose first language is sign, that leaves a lot out.
We're teaching KOKO to sign, and we're doing it with fluent Deaf signers who guide what it learns and tell us when it gets things wrong. We would rather show less, and have all of it be right.
Each of KOKO's fingers moves on its own, so it can form the handshapes of the fingerspelled alphabet clearly enough to read: names, places, and words that don't have a sign.
Sign language uses the face, the head and the space around the body, not just the hands. KOKO uses its face display and head direction alongside its hands to carry meaning.
How it's built
Each finger moves independently, so KOKO can form the precise handshapes that signing needs.
A display that shows the facial expressions sign language relies on, like questions, emphasis and emotion.
KOKO faces the person it's signing to and uses head direction the way signers do.
HCI research
A robot that signs raises open human–computer interaction questions. We're studying them in the open, with Deaf researchers and participants, and we'll share what we learn.
How accurately and comfortably do Deaf signers understand KOKO's fingerspelling and signs, compared with a human signer? Which handshapes, speeds and viewing angles break down?
Studies of signing avatars found Deaf users care most about facial expression and natural movement, not just the hands [2]. Can a robot's face display and head movement do that work?
Prior work compared physical and virtual robots for sign language tutoring [4]. In a real home, is a signing robot clearer or more welcome than a signing screen?
Deaf organizations caution against replacing human interpreters, especially for important information [3]. Which everyday moments do Deaf people actually want a signing robot for?
Our commitments
Fluent Deaf signers shape what KOKO learns and review every sign before it ships, and they are paid for their expertise.
KOKO knows a small set of signs, not a whole language. We say clearly what it can't do yet.
For medical, legal and other important conversations, people deserve a qualified human interpreter. KOKO doesn't replace one.
Sign languages are full languages with their own grammar and culture. We build KOKO to respect that, not to simplify it.
We're looking for Deaf organizations, signers, sign language educators and HCI researchers to co-design, study and test KOKO's signing. If that's you, we'd love to talk.
Get in touch