Why 'White Glove'?

I chose this nickname because of the white glove that comes up when you hover over something. I don't know really I just thought it went with it.

What is it?

Whenever I watch a movie like Her or a show like Black Mirror I’m always so jealous of the technology in these futuristic movies and how natural and fluid they seem/integrate with the surrounding environments. Hand gestures and muffled voices are picked up and understood immediately by these AI and computers and then they do exactly what the user thinks. With a slight grunt and wave of the wand and the main character's workspace clears itself and shuts down. Taking in the context of what the user is doing and then performing the correct task based on the same gesture that could be used for something else is really interesting.

So as a developer the first thing that came to mind was how can I build something like this? How about a browser extension that uses the webcam to track the user's hand gestures to control the web browser? This extension would also use the javascript library webgazerjs to track the users' eyes and trigger certain actions depending on what they’re looking at. While thinking about this and researching some things I came across an existing library which basically performed all the functions I wanted. It's called handsfreejs and it is incredible. I was so happy that it already existed and now I got to thinking about the next step, how can I make the physical portion of this?

Looking at the YEEZY STEM player and the Logitech MX 3 mouse I think there’s an interesting intersection on how we can manipulate inputs and interact with applications and other physical objects. Ideally, the physical component would be very simple in design and blend into the desktop/surrounding area without standing out too much.

Physical Version

• Something that heavily resembles the YEEZY STEM PLAYER and acts as a gesture-based mouse.
• The camera looks up through the middle to capture hand movements.
• Environmental sensor for light detection.
• Mini microphone for voice commands
• Built on Pi or Coral AI