stardance has been extended another month! the new deadline is october 31 :)

You are browsing as a guest. Sign up (or log in) to start making projects!

Philipe_P

@Philipe_P

Joined June 1st, 2026

  • 2Devlogs
  • 1Projects
  • 1Ships
  • 1Votes
Ship Changes requested

Lumen is an assistive communication application designed to make communication more accessible for people who have difficulty speaking. The goal is to reduce the friction between a person knowing what they want to communicate and actually being able to communicate it.

Lumen provides multiple ways to create and speak messages, including a communication board, keyboard input, and gesture-based controls. This gives users different ways to interact with the application depending on their individual needs.

The project has been actively developed for multiple hours, including significant work on the desktop application, user interface, gesture recognition, speech output, communication-board functionality, local phrase storage, and the overall user experience. I also built the project as a usable Windows application rather than leaving it as a prototype that only runs in a development environment.

Lumen includes several major quality-of-life improvements:

  1. Multiple input methods: Users can communicate through buttons, keyboard input, or gesture controls rather than being restricted to one interaction method.

  2. Gesture-based communication: Users can select communication buttons using gesture input and activate the selected button with a simple gesture, providing an alternative interaction method for users who may have difficulty using a traditional interface.

  3. Faster and more convenient communication: Lumen includes frequently needed phrases and communication-board options, along with saved phrases and text-to-speech output, reducing the number of steps required to produce spoken communication.

The project focuses on making an existing everyday task—communicating—less complicated and more accessible. I have put substantial development effort into turning the idea into a functional application that can actually be used and tested.

Try project → See source code →
Open comments for this post

3h 43m 40s logged

Lumen 2.0 is ready for Launch! I made a fully functional website & help page for the software and a Windows download for the software with a browser version with the same capabilities. Please consult the help page on the website (it is very detailed) if you have any questions about Lumen, or you can comment below this post! Use Lumen: https://philipe-pajishvili.github.io/Lumen-Website/

0
0
30
Open comments for this post

22m 29s logged

Welcome to Lumen (V 1.9 Beta): The Free Assistive Communication Software!
Access Lumen: https://philipe-pajishvili.github.io/Lumen/
Current Development Stage: Refinement/Testing Stage

The hours on the Hackatime monitor are not accurate for how long I spent on this (about a week).

Please give it a try and let me know how it goes!

A Quick Introduction to Lumen

Uses AI-powered gesture/pose recognition to help users communicate.

Uses a webcam to detect specific movements and translate them into words.

Includes an advanced keyboard for spelling custom messages.

Can speak selected words and messages aloud.

Includes customizable settings such as word detection speed.

Designed to make communication more accessible, independent, and engaging.

Built with JavaScript, TensorFlow, ml5.js, p5.js, and Teachable Machine.

How to Use Lumen

  1. Launch Lumen

Open Lumen and allow access to your camera.

Position yourself so your upper body is visible to the camera.

2.Use the Pose Model

Lumen comes with a pose-recognition model trained to recognize your left arm.

You can upload your own pose model from Teachable Machine by pasting the model link below the black box.

Move your left arm into the trained position (shown in the image) to select or communicate a word (in blue) or activate a menu.

Note: You MUST wait for the black box to say “Input Detected” before putting your arm down.
3. Choose Your Communication Mode

Use Basic Mode (Default) for quick, preset words.

Switch to Advanced Mode by activating the menu in Basic Mode to access the keyboard and build custom messages.

4.Speak Your Message

In Advanced Mode, use “Say” to have Lumen read your selected word or custom message aloud.

Adjust Word Speed in Settings to change how quickly Lumen detects selections.

Give it a try and let me know what you think! :)

1
0
115

Followers

Loading…