J.A.R.V.I.S. Milestone

July 10, 2014

I hit a major milestone today in my little Jarvis project. I got Dragon Naturally Speaking to execute arbitrary shell commands though an SSH session. This is pretty much the last link in the chain of awesomeness. Here’s the flow:

1. I speak the word “hello” into the mic on my earbuds connected to my laptop running Debian
2. That audio goes into a Windows XP guest running under VirtualBox

3. Dragon Naturally Speaking running on Windows performs voice recognition on it and turns it into text

4. That speech is run through a grammar written in Python configured using natlink and Dragonfly

   class WelcomeRule(CompoundRule):
       spec = "hello"                  # Spoken form of command.
       def _process_recognition(self, node, extras):   # Callback when command is spoken.
           spd = Text("spd-say -p -20 -r -50 'Welcome home, Sir!'")
           spd.execute()
           enter = Key("enter")
           enter.execute()
   

5. That grammar sends text-to-speech shell command:

   spd-say -p -20 -r -50 'Welcome home, Sir!
   

   …to the open Putty window already SSH’d back to the Debian host

6. The speech dispatcher sends the text to festival which outputs, “Welcome home, Sir!” though my earbuds.

Thanks to Tavis Rudd for the inspiration.